What happens when you stop relying on resumes

If you’re a regular reader of this blog, you know that I’ve come to rely quite heavily on data. I’ve counted typos on resumes, I’ve sifted through a corpus of engineering offers, and I’ve skimmed thousands of recruiting messages to tag them by personalization level. This post, however, is going to be a bit of a departure. Rather than making broad, sweeping conclusions based on a lot of data points, I’m going to narrow in on one story that happened, in part, because of some data I gathered during an experiment. I think it’s a really cool story, and I can only hope that there will be more stories like it that, in time, will enable me to write another post with lots of graphs.

The experiment in question was thus. Last fall, I showed a set of anonymized engineering resumes to about 150 engineers, recruiters, and hiring managers and asked one question: Would you interview this candidate? It turned out that not only did both recruiters and engineers largely fail at predicting who the strong candidates were, but, much more importantly, no one could even agree on what a strong candidate looked like in the first place.

These results were quite startling, and they left me scratching my head about the implications. After all, resumes are such a huge part of how hiring is done. Of course, it wasn’t always this way. From Chinese civil servant exams to Thomas Edison’s somewhat infamous general knowledge tests, filtering for positions requiring specialized knowledge was, up until the last century, accomplished largely through a variety of aptitude tests. Even as the resume rose in prominence and became a ubiquitous arbiter of potential fit, its format has gone through a number of evolutions. Things that were commonly included on resumes at various times (and still are in certain cultures) — a photo, marital status, age, religion, height, weight, blood type, and political affiliation — are no longer en vogue, and it’s certainly not inconceivable that, in the future, asking for a resume will seem just as silly as these now-outdated practices.

So, if resumes aren’t a good signal for hiring engineers, what is? In my blog, I wondered if, instead of a resume, it might be interesting to have people write a bit about something they built that they were excited about. I was excited, in turn, when my friends at KeepSafe actually decided to try this out — for a month, candidates who applied through KeepSafe’s No Resume campaign were evaluated purely on what they wrote about their projects, and this writing sample would be the only thing used to decide whether someone would get an interview. This is the story of what happened.

KeepSafe, in some ways, came to this experiment out of necessity. Like many other small startups, they were feeling the acute pain of hiring engineers in this market. They had a product people loved, a ridiculous user to engineer ratio (39M users for 6 engineers), and the requisite hip South Park office. Along with that, though, they had quite a high hiring bar and consistently found themselves competing for talent with the likes of Google. And more often than not, they’d lose.

To stay in the game, KeepSafe needed to change up the rules and tap into a different pool of people. The hope was that out there were plenty of talented engineers who were getting overlooked because of their lack of resume pedigree but who were passionate and skilled and who would be awesome hires, given the chance to show what they could do.

KeepSafe’s experiment struck a chord, and in the first two days, over 400 people submitted descriptions of stuff they’d built. Awesome stuff like this.

Submissions varied in length. Some were just a sentence. Some were multiple pages and supplemented with links to demos. Not all submissions were awesome. Some were generic, copy-pasted cover letters espousing their interest in the software development life cycle, and some were just disconnected links. Those KeepSafe cut immediately. The rest were a lot tougher to cull.

Though I didn’t participate in the judging, one thing that struck me about the submissions I read was that my normal approach was useless here. Normally, when I look at resumes, I can make a yes/no decision based on proxies like past employers within 10 seconds. With KeepSafe’s submissions, I couldn’t rely on proxies at all. With each person, I had to think about what they built, imagine it, understand it. In addition, I found myself starting to getting a real read on people’s interests and trying to imagine what projects they might want to tackle at KeepSafe, something that’s much harder to gauge with the traditional resume format.

All of this was weird and slow, but it felt really good. And then I realized that unless your achievements fit into a convenient mold, you’ll lose the best parts of you when you try to beat your windy path into a nice clean line. Resumes don’t have an explicit section for building rockets or Minecraft servers, and even if you stick it somewhere in “personal projects”, that’s not where the reader’s eye will go. The sad truth is that if the reader doesn’t like or recognize what they see before they get to the rockets, they will likely never get there at all.

***

By traditional Silicon Valley standards, AJ Alt didn’t look especially good on paper. He hadn’t attended a brand-name school, his GPA wasn’t high enough to stand out, and his only professional programming experience was a multiyear cybersecurity stint at a huge defense contractor, the nature of which he was forbidden from discussing, on a resume or otherwise. His GitHub, full of projects spanning everything from a Python SHA-1 implementation to a tongue-in-cheek “What should I call my bro?” bromanteau generator, hinted at a different story, but most people never got there. While AJ’s government work experience gave him a good amount of cred in the public sector, he found that making the move to industry, and startups especially, was near impossible. It wasn’t that he was blowing interviews. He just couldn’t get through the filter in the first place.

Of the 415 people who applied to KeepSafe through the No Resume campaign, 18 ended up interviewing, and 5 came in for an onsite day of coding. One received an offer. It was AJ, a candidate that Zouhair Belkoura, KeepSafe’s cofounder and CEO, readily admits he would have overlooked, had he come in through traditional channels. Since starting a couple of months ago, AJ has built out an open source Android animation library and was one of the co-creators of a brand new security feature within the app.

When I asked Zouhair if KeepSafe would consider the experiment a success, his answer was a resounding yes. According to Zouhair, “The overall quality of applications seemed to be a lot higher than through our normal resume channels. We also met more people who seemed to genuinely like programming and wanted to talk about it [rather than people just applying because they wanted a job].”

The team has decided that they will continue to hire without resumes for the foreseeable future — the funnel numbers for this process weren’t too different than what you’d see in a more traditional setting, and, with time, as the team gets better at making value judgments based on writing and projects, they will only improve. Resumes work well for companies like Google because their strong brand drives a revolving door of inbound applicants. Copying that process, if you’re a smaller brand, however, can be detrimental. Of course, this experiment had a small sample size, and one hire is more anecdote than gospel, but if replacing the resume with a writing sample is a good way to get at a different, yet highly skilled and engaged talent pool, then it’s worth a shot. As Zouhair told me, “Our culture is that code speaks louder than credentials, and now we have a hiring process that reflects that. [When we made the hire], we had never even seen AJ’s resume. And we don’t need to.”


Interested in applying to KeepSafe without a resume? They are still accepting not-resumes at https://www.getkeepsafe.com/noresume.html. Think resumes are generally a dumb way to filter for engineering ability? Check out what I’m working on now, interviewing.io.

32 Responses to “What happens when you stop relying on resumes”

  1. Lucas Howell

    Recruitment agents are ruining lives of people ! They are putting more pressure on people for the sake of gobbling up more money for clients. For example, they force people to be expert in 10 things for the sake of pleasing their clients. They are just pimps. This business needs to be disrupted.

    Reply
  2. anonymous

    No resume is also not a good idea. Writing a sample about a projects doesn’t say anything more than what a resume reflects. Most of my resumes I see do write about what they have done. The questions interviwer’s ask in the interview are the key in my opinion.

    Reply
  3. Tim

    Well done – achievements are the key, now go the next step.

    Production Statistics is a subjective way to prove reliability.

    Yeah I know – “who even records that data let alone have it for your resume?”

    Well start doing it now! Every day record the things you produced, the things you exchanged for value. These things can be verified thru the company you work/worked for or the clients you produced for.

    The idea is to demonstrate that you can produce and produce consistently. Who wouldn’t hire you then aye?

    Reply
  4. T

    You ever stop to think that the reason typos exist on resumes is because the people who catch them never report them?

    Reply
  5. Alan

    This is spot on. I have a friend from college who was writing computer games when he was 14 but couldn’t write a resume to save his life. He had a hard time finding his first engineering gig after college and was working for a welding company in our home town. I helped him with his resume and he landed a job with a semiconductor manufacturing tool maker. He definitely would have done better with the resume.

    Reply
  6. Dan

    At Talentoday we developed an assessment solution that allows us and anyone else to hire based on a candidate’s personality and motivations (soft skills). We started working with top recruitment agencies like Adecco to allow them to integrate our solutions into their recruitment process. So far more then 3M students from top universities like Stanford, Harvard in United States but also in other 126 countries have been assessed. One thing it’s sure hireing based only on hard skills it’s a thing of the past!

    Reply
  7. Noam

    We’ve been hiring without up-front resumes for years, relying instead on aptitude tests and writing samples as described here. It’s the only way to fly.

    Reply
  8. Alex Hortopan

    I think a good combination of identifying good candidates and it not relying solely on the resume but rather on some online research regarding each individual. There’s a lot of info out there 🙂

    Reply
  9. Anton Babenko

    Asking for free-form description of what people likes to do and have done in the past is a great starting point for most of screening. Ask candidates to show their official resume for next phase of the interview is also giving more details about their experience. I experience that it works equally well for small-to-mid size companies.

    Reply
  10. tie

    That post actually resonates with me very well. My favorite interview question is “Describe/sell me a technology that you really like”. The kind of technology that the candidate chooses, the way he is able (or not) to communicate and defend it, the level of understanding, and the passion he has about it are all very strong clues that are hard to fake.

    Reply
  11. Anon

    What happens when you stop relying on “font-weight: 300;” in your CSS styling on your website?

    Your website no longer displays very thin, single pixel wide, almost impossible to read fonts to your users that make you look like someone who does not care about the readability of the text content you present.

    Reply
  12. DJ

    Not sure this is 100% your process, vs what happens when someone looks at a resume. It sounds like your experiment forced the employers to really look at and truly evaluate each candidate. They put in more effort, and got a better result. Usually when people look at a resume, they just scan the name, the skills they list, the places they work, and maybe they’ll get to the education. The whole process takes 30 seconds? It’s not surprising the results from resumes produces worse results.

    Reply
  13. Graham

    It looks like this process is really weeding out people who are poor communicators, and likely many people for whom English is not the first language. That may well help to find excellent hires (except in those situations where technical skills trump everything else). It probably means that people who think like those doing the hiring (i.e. are similar culturally and socio-economically) also have a better chance, which might not always produce the best result.

    Reply
  14. JD

    What about people who cannot write well? Don’t they deserve jobs?
    Effective communications is always a needed skill, but that doesn’t necessarily mean being able to write a story – especially in a small software shop where everyone can pop around the corner to chat.

    Reply
  15. Chris

    You do need to make an accommodation for non native English speakers. That said, I’ve long believed that typos are the sign of a bad developer. The excuse I always hear in response is “But we don’t pay them to write”. This is pure crap. Code is a written language!! Yes, you ARE paying them to write! The difference is:
    1) Most people in the US learn English from birth but don’t start learning code until they’re in their teens. If you can’t master your native language why should I believe you’ve mastered a language you’ve only been learning for the past 5-10 years?
    2) A typo in English usually is inconsequential because the reader can understand the overall context. A typo in a line of code can cause a failure because machines are not very good at context.

    For this reason engineers should be held to a HIGHER standard than most other roles.

    Grammar: it’s the difference between knowing your s**t and knowing you’re s**t!

    Reply
  16. Mark

    I LIKE the idea of writing an ‘accomplishments’ letter because it lets you see more of the real candidate, not the boring standard view. But, you went through over 400 people to arrive at a single candidate? I wonder if your criteria is also in need of a rethinking? Were all those developers really not a good fit? Is your work and culture so difficult and unique?
    Also, part of the reason your company started this process was because Google snaps up the folks from the Ivy Leage world. But, Google has publicly stated that they gave up on degrees and formal education as a criteria a few years ago, when their data clearly showed no correlation.
    Even now, Google’s process is far from effective and not worthy of emulation. The head of hiring at Google wrote a book about how badly they’ve handled their hiring process. Currently its handled by teams of people, but its still a mess if you ask me.

    Reply
  17. Maurine Pen

    Great Post! I do think Resumes are important in choosing a suitable candidate for the job offer. It is with that resume you can see the profile of a person, starting with their education level, job experiences and the like. How the resume is formatted is also important so it’s important to have grammar and format. I even relearned proper grammar and proper Enlgish at http://preply.com/en/skype/english-native-speaker to further improve my knowledge and skills to land a job that I applied for.

    Reply
  18. Greg

    When I started to read this I thought that it was great and then I put myself into the shoes of the job seeker and thought, “it’s a big enough ordeal without having to write a bespoke piece for every single role I applied for”. I’m thinking that if someone can’t articulate themselves in a CV then for CERTAIN roles they may not be the person you are looking for anyway.

    Reply
  19. Tim

    I agree the resume is not an adequate tool to hire or filter candidates but now you have moved it to writing expertise. So I can’t put together a decent resume what makes you think I can write any better? When you go through the written projects are you again going to be counting spelling errors and improper grammar? Some really good developers and coders are terrible at documentation which is why we get a Technical Writer to document their work. And then we come to those that cannot interview well. They are great at their job but many have few if any people skills and don’t interview well. I think you are on the right track with the “no resume” idea but the follow up would be to not interview the candidate but interview former co-workers and managers. I see so many people get hired where they ask for a reference yet if you talk to the references they were never contacted and neither were any other references.

    Reply
  20. Rhostam Ayyelos

    Hi,
    This approach actually makes a lot of sense. It basically forces what should be part of an interview up to the forefront. Looking at resumes, people tend to get caught up on the wrong things. Not to mention, as a sad reality, I feel people just cut/paste stuff into their resumes. Simple follow-ups to many many “duties” or “projects” over the years yielded, “well, I was on a team that…” I personally chose to include a few words about some of my favorite projects/ challenges completed while working for my various employers. They could range from technical to professional. Resumes are just littered with too much noise, fluff, and misleading clues. Then there is the interview process… I can’t believe how many I’ve sat in and the process ended up being more about hypothetical and personality questions. Sure, these are important in determining as best you can in a small session one’s reasoning capabilities and how they might fit on the team, but how about you verify as best you can they are even technically competent at their jobs? My recent employer has hired and fired (or has people abandon post) no fewer than 5 senior “engineering” staff either in networking or systems administration and architecture – all because they simply didn’t even have basics down (and even I could judge that – a DBA!).

    There are many faults in the system. Resumes are taking the blame thanks to lazy managers and recruiters.

    Cheers, thanks for the story. It certainly is a different approach to consider and something that warrants exploration.

    Reply
  21. Anna Sparks

    Great article, Aline! You are so right and I think your point is particularly relevant in the tech field where there is so much competition. I think there’s a solution to this problem and that is a key qualifications section at the top of a resume. Each of the examples you gave (what applicants told KeepSafe) could be incorporated into the first section of a resume (called Key Qualifications). It would catch the employer’s eye even in this world where resumes are still required. Thanks again for this thought provoking article!

    Reply
  22. Rosalie

    Very interesting article and I like your data-driven approach to optimising tech recruiting. Would this also work for other trades? (E.g. investment).

    Reply
  23. Chang Otillia

    I am a huge fan of this. I call it edge case recruitment and have found that companies that don’t understand it, usually miss out on hidden gems. Not that they would know if it harms them but it’s the best way for start ups to compete with traditional process. I think the creative writing element allows you to peer more excitedly into beautiful minds.

    Reply

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>