We recently undertook a project to improve Medium’s engineering interview process. As part of that, I wrote this document to describe qualities that we aren’t specifically seeking in engineering candidates. It was originally published to Hatch, our internal version of Medium, on February 26, 2016. For more information about Medium’s practice of making internal documents public, see Hatching Inside Medium.

This is the actual document used by Medium when hiring engineering candidates and has replaced the version on Hatch as the source of truth. If you apply for a job at Medium, you can use this to see criteria for which you will not be negatively judged if they are lacking.

Engineering interviews: what we don’t screen for

Part two of a three part guide to conducting engineering interviews at Medium.

Preface: Refining our process
Part 1: What we screen for
Part 2: What we don’t screen for
Part 3: Grading rubric

This guide is up for continual improvement. Suggestions are welcome for additional capabilities that should be considered unimportant, as are those for improvements in clarity and precision. However, it is unlikely that any of these categories will be removed, unless there are significant new learnings.

There are any number of qualities, competencies and achievements that we could screen a candidate on. However, many of those qualities traditionally prized by tech companies have no causal relationship to the candidate’s future performance. This document lists some of the capabilities that are commonly part of assessment criteria at other companies, but which we do not specifically look for in candidates.

None of the following areas are negative. However, screening for them can cause us to focus on factors that are tangential or outright unimportant to job performance. There are many fine candidates who will do well at Medium (and current colleagues who are already doing well!), who might be missing all or most of them.

When interviewing, you may record the existence of these qualities — provided they are supported by observable facts — if you feel they substantially contribute to a person’s overall suitability, but you may not count a lack of them against the candidate.

School

A “good” school doesn’t mean the candidate will succeed in their role. The entry practices of many schools are still discriminatory by economic status, race and/or gender. Many excellent engineers did not study computer science in school. Many very accomplished people did not even attend or finish school, cf. Ev.

GPA

A high GPA is nice. It shows the candidate was able to work at a high level for an extended period of time. A low GPA in isolation is not a strong signal of anything. The college experience for many requires making compromises based on the practicalities of life. For senior candidates, a GPA is basically meaningless. (I got a higher GPA than Dan, for instance, but… so what?) For junior candidates, project work is more illuminating.

Previous companies

Many companies show excessive positive bias or deference to candidates who have previously worked at major companies like Facebook, Google or Amazon. While these companies are known to have generally good engineers, they are large enough to have bad ones as well, and their hiring practices are such that many qualified candidates are rejected. A long stint at a major company like these can even in some cases be detrimental as many problems are already solved for engineers, a luxury that is not usually available at a startup. Finally, these candidates may also have distorted views on what constitutes “success” for products, be less comfortable with uncertainty, or have views on deployments and the application development lifecycle that are not aligned with the needs of a smaller company. Candidates with large companies like these on their resumes should be interviewed as we would any other candidate.

Friendship

It is not important for you to feel like you could be friends with a candidate. We are not building a family, we are building a team. A good working relationship, the ability to collaborate, and mutual respect are sufficient. We tend to want to “have a beer” with people who are like ourselves, which can lead to homogenous teams.

Open source contributions

Requiring a candidate to have contributed to open source projects is prejudicial. Many candidates do not have the time to contribute to open source projects because they need to work to support themselves or their families. In addition, many open source projects have toxic cultures that are off-putting to many candidates. Finally, it’s not a great filter to kick out everyone who doesn’t want to work on software for free in their off-hours.

Encyclopedic knowledge of CS algorithms and data structures

It is not important for most candidates to be able to implement a red-black tree from scratch. It is not important for most candidates to know what a Fibonacci Heap is if they’ve never encountered one. It is not important for most candidates to be able to reverse a linked list on the spot. A high level knowledge of time and space complexity, the ability to research, learn and integrate different approaches, and recognition of trade-offs is in most cases sufficient. (In specialized positions, the importance of this might vary.)

Specific technologies

It is not a requirement that the candidate know JavaScript or Go. For all but the most specialized roles, interviews should be technology-agnostic. You should instruct the candidate to use the language they are most familiar with.

Confidence

We want candidates who have the ability to make a decision and back it up with sound reasoning. However, confidence as an isolated trait is not necessarily positive. Many candidates will have difficulty demonstrating confidence in an interview setting (for marginalised candidates, see also stereotype threat). Specifically screening for confidence disproportionately favours white men. Finally, many of the most self-aware people actually appear to lack confidence because they recognise how much they don’t know. The opposite is also true.

Culture fit

Culture fit is a polite way of saying “be like us”. If we’re looking for people who fit in with what we already have, we risk closing ourselves off to new perspectives, and building a homogenous team that suffers from groupthink. Note that culture fit is distinct from alignment of values, and the addition of new perspectives, which are valuable criteria. If someone can #DoHardThings and help us connect to a community who we don’t really understand, they are a potentially valuable resource for us, even if (and perhaps especially if) they make us feel uncomfortable by forcing us to confront our prejudices.