Knowledge Questions

Radar chart comparison of Knowledge Questions and AutoIterative Job Interviews
TL;DR: Asking Quiz or Knowledge Questions could answer if the candidate has studied for the exam, at best. It will also taint your hiring filter with randomness and biases. Jump to recommendations

Using Knowledge Questions in the hiring loop

A knowledge question interview consists of several quizzes about specific tool or technology. It focuses on checking whether the candidate knows the correct answers only. The interview does not test their skills related to the technology in question. The assumption is that knowing trivia facts implies work experience with it. The setting is often face-to-face to prevent the candidate from looking up the answers.

Companies often use such interviews as a cheap initial screening step of the hiring loop. Sometimes, a non-technical person performs the interview to save up on engineering time.

Knowledge Questions: are they unbiased?

No.

This interview type introduces similar biases as other face-to-face interviews.

The interviewers have to detect and avoid their first impressions. Their gut feeling about the candidate and if they like them or not impacts their opinion. They have a limited set of input data and have to decide in a short time frame. In such circumstances, brains use all the possible help from the biases to fill in the gaps.

The interview questions themselves introduce randomness into your hiring loop. Jacob Kaplan-Moss, one of the creators of Django, cleverly puts that:

There’s something like 5,000-10,000 potential trivia questions about Django. When asked a random question, I’m likely not to know the answer. If your trivia question predicts that I don’t know how to use Django—is it a good question?

Knowledge Questions: are they low-stress for your candidates?

No.

Knowledge questions have the same stress factors as other face-to-face interviews.

They need the candidate to recall some bit of trivia that is often present in the documentation. In real life, people rightfully do not bother to memorize such things at all. During the interview, they have to recall them while being under time pressure. Someone they see for the first time decides their fate, and they feel their non-verbal signals. The stress impairs both the memory of the candidate and its assessment by the interview.

Knowledge Questions: are they real work?

No.

Remembering random trivia facts is not what the engineers do at their day-to-day jobs.

The job of the software engineer is to solve a particular business problem with code. They are not playing Jeopardy—they are building your product.

Knowledge Questions: are they a good predictor of future performance?

No.

Unless remembering details about some technology is the core business of the company.

The only thing they check is whether the candidate happens to know the specific bit of trivia. That can indicate if the candidate has studied for the exam but not the work experience. They also favor people with stress resistance more than people with good memory. But they do neither measure nor predict the ability to ship something to production.

Knowledge Questions: how to improve your hiring loop

Stop using them as an interview filter.

If they come from a set of well-known facts, they are pointless as everyone passes the filter. The more obscure facts you include, the more randomness and luck you introduce to the hiring loop. It can produce a good result from time to time, but the better option is to select people by their ability to deliver.

One deceptively attractive quality is that such interviews are cheap on engineering time. The premise is that a non-technical recruiter can do them at the time of the initial screening. But if candidate flow overwhelms your engineers, then first: congratulations! And second: you do not need to decimate the rich input with randomness to throw the talent away. The better option is to harness all that flow, and you can do it without investing engineering time at all.

AutoIterative Job Interviews to the rescue!

We built AutoIterative Job Interviews to be an initial screening of your hiring loop.

They scale up to thousands of candidates without sacrificing the assessment accuracy. They let your candidates shine and show that they can ship code that works to production. And they do it at a constant cost per applicant without demanding your engineering time. Your engineers can continue working on your product until it is time to meet the candidate face to face.

Start hiring now

Want a second opinion?

Here is what other people say about Knowledge Questions: