Take-Home Challenges
Using Take-Home Challenges in the hiring loop
Take-Home Challenges are problems that candidates can solve at home at their own pace. The task usually mimics the real-world issue they might face when hired. The reviewers expect the deliverable to be source code for a later review.
The setting is remote and asynchronous, and often the candidate is free to pick their tools as well.
Take-Home Challenges: are they unbiased?
Yes.
The setting of take-home tasks excludes all biases of face-to-face types of filters.
The candidate can solve the problem asynchronously and in a familiar environment. There is no ongoing judgment from the interviewer during the process. The candidate can take their time, sleep on a problem, and return to it the next day with fresh ideas. This process closely resembles how they would work in a real-life work environment. All the biases that can arise come later, at the grading stage. For example, reviewers could favor familiar solutions over novel ones. To avoid this, check how well the solution solves the problem before doing any human review.
Take-Home Challenges: are they low-stress for your candidates?
Maybe.
The stress of Take-Home Challenges is the lowest among all filters, but it is not absent.
Their setting removes almost all the stress factors of the face-to-face interviews. The candidates can pick the best time, pace, and tools to operate at their peak performance. It is the strict deadlines and lack of feedback that add stress to this type of interview. The deadlines can hurt the interview efficiency by producing rushed solutions with shortcuts. The lack of feedback can make the candidate worry whether they are doing enough and reaching the bar. That punishes the best candidates by making them invest more time than it is enough to complete the task.
Take-Home Challenges: are they real work?
Yes.
But only if the tasks given to the candidates are not artificial and mimic what they might do when hired.
The job of a software engineer is to code solutions to arising business needs. When done, their code is deployed to a production environment or gets shipped to a customer. After that, the reality determines how well the code solves the original problem. The typical Take-Home Challenge resembles the same process but lacks the last step.
Take-Home Challenges: are they a good predictor of future performance?
Maybe.
The prediction accuracy depends on the task given to a candidate and on the grading process.
Both should mimic the actual processes happening in the company. The task should not be too different from what the candidate would do when hired. The grading process should look like the evaluation of their future everyday work. The closer those metrics are, the better the prediction.
Take-Home Challenges: how to improve your hiring loop
Give real-life tasks to your candidates, and focus on how well they solve the problem.
Take-Home Challenges are already one of the best ways to test your candidate skills. You can improve them further if you address their main weaknesses and pain points.
The first problem is the lack of feedback for the candidate. At work, automated tests and peer reviews give it to the engineer while they write their code. In contrast to this, the candidate has to solve the take-home test in isolation. They need to set the expectations themselves if the task comes without clear test cases. They have to define when to stop and that introduces the fear of "not doing enough." The lack of feedback punishes your best candidates by making them invest more of their time. And if rejected after going this extra mile, it makes it even more unfair in their eyes.
The second problem is the difference between the actual work and the grading process. At work, the code passes tests, peer reviews, canaries, and a final reality check by production. At the interview, the code is often only read by a human engineer. Not only does that introduce biases, but it also changes the interview metric. Reading the code is different from running it, as it doesn't measure how well it will run when shipped.
The final issue is the cost of the take-home challenges. The code reviews require valuable engineering time, often from the most senior employees. The challenges need to be well-defined and have clear test cases. The solutions might come in a variety of languages and might need different skills to review. To provide feedback and answer candidate questions, you need to involve an engineer. If you decide to design, set up, and manage an automated submission testing system, you have to invest a lot of time in it.
These hidden costs of Take-Home Challenges make this method of filtering extremely expensive.
AutoIterative Job Interviews to the rescue!
AutoIterative Job Interviews deliver the value of Take-Home Challenges without their drawbacks.
The platform provides your candidates with real-life problems that they need to solve. On every commit, it deploys their code to a dedicated production environment. It tests the solution with real-life data and gives ongoing feedback to the candidate. The platform focuses on measuring how well the solution addresses the original problem. The metric is whether the candidate can ship code that works in production.
The platform does all that at the constant cost per candidate. It allows your engineers to focus on your product until it is time to meet your candidate face to face.
Start hiring nowWant a second opinion?
Here is what other people say about Take-Home Challenges:
- George Mauer criticizes take-home interviews with a time limit, lack of templates, and hidden requirements.
- Jacob Heuser of LinkedIn says they ditched take-home assignments due to the costs of building and grading them being too high.
- Indeed.com addresses the above with recommendations of what it takes to create a good take-home challenge.
- Angel.co explains the common mistakes in take-home challenges and how to avoid them.
- Candidate's perspective: what it takes to ace a take home interview.