The futile recommendation letter cycle

It is my pleasure to recommend that recommendation letters be abolished.

In biology, a futile cycle exists when two metabolic processes run simultaneously in opposite directions. Since the pathways oppose each other, they provide no net benefit to the organism*. Collectively, the pathways only expend energy.

Letters of recommendation are the currency in a futile cycle of professional evaluations.

Photo by Signature Pro on Unsplash

Whether applying for graduate school, or a new job, there's a good chance the employer will ask for a letter of recommendation. The applicant secures a commitment to get recommendation letters from 3-5 referees. The referees write and submit their letters. The letter becomes a part of the applicant's application package.

The employer reads the 3-5 letters from each applicant. They use the letters to score or rank applicants.

Like a futile cycle, these letters are good for little more than dissipating energy. At best, they are a time drain. Damaging to the applicant at worst.

It's awkward for candidates to ask for letters of recommendation. Referees, sometimes swamped by requests, have limited time to write them. Employers or committees spend a ton of time reading them.

As a first-generation college student, this whole process was mysterious. I was a mechanic in a race car shop trying to break into academic circles. Professors intimidated me. How on Earth was I going to get a letter of recommendation? Who would I ask? Why would they spend time recommending me? What if they said no? What if the letter sucks? It took immense energy to simply ask for a letter. After I mustered the courage, and my referee agreed, this mysterious transaction occurred. They'd "write a letter" and submit it and I'd either get the position or not.

Now, I see the other side as an employer. I run a mid-sized research laboratory at a high-research activity university. I regularly review and write letters of recommendation for faculty, graduate students, postdocs, and staff. Letters range from short paragraphs to novellas in length. Some letters are glowing. Rarely are they critical. Most are in the middle between these extremes.

To the candidate, the entire process is invisible. Applicants rarely see their letters (unless they write them themselves, more on that in a minute). Search committees or employers rarely explain the role letters play in their evaluation.

There are several serious issues with using letters of recommendation to rank candidates. The letters that make it into application packets are subject to a huge selection bias. The applicant asks people for letters who are likely to write letters that are not bad. There is almost no chance that a letter makes it into a packet that is critical of an applicant. Thus, candid criticism can sink an application package, even if it's constructive.

All letters are anecdotal evidence. Everything in the letter can be true according to the referee's experience. And yet, it does not predict future success for the applicant. There is no guarantee that someone amazing in one situation will work well in the next. Likewise, someone with untapped potential in their previous position may excel in a new position. Context matters. There are almost infinite reasons for this—related to the job or otherwise.

Strong letters have exactly one thing in common: the referee's persuasiveness. The perceived strength of the letter is more about the referee's writing ability than the strength of the candidate. Good referees are masters of picking—and phrasing—which strengths to emphasize. And how to soften weaknesses. Yes, letters can help fill some of the between-the-lines aspects of the applicant. Yet, often what is not in the letter is as—or more—useful than what is in the letter. Thus, experienced evaluators may read between the lines of what is in the letter.

Letters may promote gender and racial bias. For example, the gender or race of a candidate influences the length and descriptions contained within letters. These differences are subtle but affect the applicant's selection success. Such discrepancies may affect male applicants from underrepresented groups the most. (See here, here, and here)

There are power dynamics between referees and candidates. Some referees covertly or overtly use that power to coerce their understudies into performing more work. Likewise, candidates may join a group for the sole purpose of "getting a letter" for their next career step. They are not as interested in the work at hand as the utility of the group to provide a letter in the future (not to bash PreMed students, but this is common among them).

There is a lack of training on how to write letters. Few curricula include best practices for letter writing. What are the components of a letter of recommendation? How does one avoid gender or other bias? Some referees even ask candidates to write their own letters of recommendation. This is especially true for "high profile" referees who already have full plates. They may not know the candidate well. They may not have the time. They may not feel strongly about the candidate. Whatever the reason, they solicit a letter from the candidate, lightly edit it, and sign it. The evaluation committee is none the wiser.

Likewise, each employer or institution evaluates letters differently. These metrics are opaque to everyone but the folks reading the applications. Even within the group of people reading them, differences in interpretations may exist on the sentiments conveyed in the letters.

Recommendation letters are futile—a huge time burden for the referees and evaluators with very little return on investment. They are a relic from the 'good ol’ days' when a letter from the right person opened doors. They feign a level of selectivity and have become surrogates for evaluating the candidate's actual records.

If letters are unreliable, what can we use instead? It may sound obvious: focus on the materials prepared by the applicant. Read the personal statement or cover letter. Is their interest in the position genuine? Do they write clearly? (I'm a huge proponent of clarity of writing equating to clarity of thought.) Is the cover letter authentic? Does it have a life, or is it bland and flat? Do they explain discrepancies in their record (grades, gaps, other anomalies)? Is there evidence of reliability or leadership in their work history?

Some of you might be thinking "I enjoy writing letters for my students" or "I find letters to be useful!" I too enjoy writing letters for quality students. And recommendation letters may have utility in specific instances. For example, letters can contextualize reasons for poor grades. Or explain an applicant's reason for seeking a new position. But these things could and should be in the cover letter, in the applicant's own words. If not conveyed clearly by the referee, nuanced sentiments like these get lost in a sea of mediocre letters. Tired eyes writing letters and tired eyes reading them presents a lose-lose situation.

If recommendation letters are here to stay, perhaps we can re-envision them. The National Science Foundation's model for letters of collaboration might be a good place to start. NSF dictates the following text for letters of collaboration:

“If the proposal submitted by Dr. [insert the full name of the Principal Investigator] entitled [insert the proposal title] is selected for funding by NSF, it is my intent to collaborate and/or commit resources as detailed in the Project Description or the Facilities, Equipment and Other Resources section of the proposal.”

Similar verbiage could be adapted for letters of recommendation. The applicant uploads their cover letter or personal statement. In the statement, they explain their relationship with a referee. And reflect on how the referee perceives them as a candidate. Submission triggers an email to the referees who check a box next to this statement

“The personal statement submitted by [insert the full name of the applicant] for [insert the title of the position] accurately conveys my relationship to the candidate, and I agree they have the potential to succeed in the position described.”

And that's it! It's about as useful as a full letter to the application packet. Yet, it takes a fraction of the time for both the referee and the review committee.

In the meantime, we can't get around the fact that letters are a current obligation. And I’ll keep writing them because I want my students to get new exciting positions! Maybe I'll start including some variant of the following early on in the letter:

I recommend {name of candidate} for {name of position}. But my experience with them is an anecdote. There are a myriad of ways and reasons that you might not have the same experience (better or worse). Their skills are good enough to get them in the door, for the reasons I explain below. But how they perform from here on out will be up to you and how you support them.

Hopefully, recommendation letters will disappear from our ever-expanding analytics-based approaches to admissions and awards. And we can dedicate the energy spent writing letters to other aspects of our work and lives.

*Futile cycles are now known to play important roles in regulating metabolite concentrations.