I wanted to talk about something I’m very passionate about on my new podcast episode, and that’s the science of interviewing candidates.
The other day I was talking to a candidate about one of our software engineering roles. I have a list of questions that I generally like to go through, but I think it’s nice to start with some small talk early on. So we started discussing the weather, commuting, working remotely, just having some chit chat earlier on.
He told me he was in the middle of relocating. I commented that I had been living in the same apartment for the past three+ years, all types of good stuff. I mentioned that I felt the hiring managers would quite like to meet him. He agreed, sent me his CV and we wrapped up the call. Wow, that was a great candidate phone screen, I thought!
Except, I looked at my notes, and I had talked for about half an hour but only found out details about his personal life. I had questions regarding his skills, competencies, and career ambitions that I didn’t get to because I was a bit distracted by the small talk.
These are details that I need to present to my hiring managers. Otherwise, they won’t be sure if the candidate has the technical and personal competencies required.
This story is an example of what researchers call an unstructured interview. Unstructured interviews are incredibly fun but are very weak in predicting candidate potential. Structured interviews, on the other hand, are great at helping you choose who the best candidate is. What exactly are unstructured and structured interviews?
How do you create a structured process? We’ll answer this and much more, on the coffee with a recruiter podcast.
Unstructured interviews
Let’s start with a problem, the problem of unstructured interviews. Unstructured interviews are the most common method companies use in vetting a candidate’s potential, but they predict job performance less accurately. Questions asked during this type of interview do not have a predefined model. In my experience, interviewers tend to make up questions on the go, or they just copy items from other recruiters or their previous company. Think of questions like:
What are your hobbies?
Where do you see yourself in 5 years?
How was your commute on the way to our offices?
In most cases, there’s no predefined idea of what a right answer might look like, so interviewers judge responses on a gut feeling.
In the social sciences, researchers tend to use a rating scale between 0 and 1 to measure correlations between two variables, and a correlation of 0.3 or above is considered meaningful. Interviews with little structure are less able to predict a candidate’s potential. Academic studies on this subject report correlations of 0.38 between unstructured interviews and future job performance. This figure is still a meaningful correlation, but not a strong one, and I’ll discuss an even better way of qualifying candidates.
How harmful are unstructured interviews?
But this is not the worst part. Unstructured interviews tend to increase the probability of biased judgements based on job-irrelevant qualities such as race, gender, age and disabilities.
For example, studies show that even informal interactions with a candidate, for example, a handshake, can impact an interviewer’s decision during an assessment. In an academic paper that appeared in the Journal of Applied Psychology, researchers conducted a meta-analysis of 31 studies on the effects of unstructured interviews on minorities.
The study found that both Black and Hispanic applicants received lower interview results than White applicants in unstructured interviews. On the other hand, structured interviews had lower group differences than low-structure interviews, which implies that a more structured process can result in less bias against candidates.
Structured interviews
The best way to mitigate these problems is to implement structured interviews. This method of interviewing starts with conducting an in-depth analysis of the role you’re recruiting. This analysis can help you decide on a set of competencies that you can look for and qualify when interviewing candidates.
For example, an analysis of a customer service role might show that you need good communication skills, good knowledge of financial services, good interpersonal skills and a willingness to go the extra mile. You then need to create interview questions that help you qualify these competencies. There are three types of questions you can ask – situational, behavioural and probing questions.
Let’s take situational questions as an example. These questions ask the candidate to put themselves in a hypothetical scenario and explain how they would act. Example questions include:
How would you handle negative feedback?
If an angry customer came to you with complaints, what would you do?
These questions should contain a predefined scoring template to pick up the right answers or indicators of what a relevant response is. For example, let’s take the second question on customer complains and rank four answers from worst to best:
Worst answer: sorry, I don’t have time to deal with your complaint
Second worst: Hey man, I’ll get back to you in a bit.
Good answer: Hi, apologies for the inconvenience. I’ll look into the problem and will come back to you in five minutes.
Best answer: Hi, apologies for the inconvenience. I’ll look into the problem, and come back to you as soon as I can with an update.
The reason you don’t want to promise an answer in 5 minutes, is because you might set yourself up for failure and disappointing the customer.
Finally, in structured interviews, it’s essential to ask all candidates the same questions in the same order. Ideally, you also need to train hiring managers to interpret answers consistently. Why should the order of the questions matter? Some questions are easier than others. When you answer a few easy questions, in the beginning, you get motivated, which helps you answer more difficult questions.
If you ask questions of varying difficulty at different moments, then you risk giving some candidates an unfair advantage.
How effective are structured interviews?
Studies show correlations of 0.51 between structured interviews and job performance, so you’re more likely to choose the right candidate when using this method of interviewing. To put that into perspective, the correlation between viagra and sexual performance is around 0.39, which is still higher than unstructured interviews. For Ibuprofen and pain relief, the correlation is about 0.15. This correlation means that structured interviews are more effective than viagra and Ibuprofen.
Various studies prove the utility of structured in different scenarios. One study from the University of Zurich focused on using structured interviews to predict leadership skills. The interview method was able to predict the leader’s annual income, situational leader effectiveness, employee well-being and organisational commitment.
Another study from researchers at Montclair and Rider university showed that structured interviews also serve to reduce biases involved in interviewing applicants who have a physical disability. In non-structured interviews, their research demonstrated that interviewers had a leniency bias, where they evaluated disabled applicants more positively than equally qualified non-disabled applicants. Structured interviews helped reduce this bias. These findings add to the support for the structured interview as a way of increasing fairness in employee selection.
Finally, a second study from Montclair University examined the effects of structured interviews on interviews involving pregnant women, and the open position of either a high school teacher or sales representative. The outcome of the study showed that assessments might have a bias against pregnant women and suggested that a structured interview process reduces this bias.
The tricky part of creating a structured interview process is that it takes a lot of work. You need to put the effort in when analysing a job. You also need to reach an agreement with your hiring managers when deciding on what skills you want to assess, or what type of interview steps and interviewers you’re going to assign. It’s much easier to wing it during interviews and have fun, right?
Well, as difficult as it can be to implement these steps, don’t’ forget that the cost of a bad hire is much greater than the effort of putting in a structured process.
So save yourself the headache of struggling to find the right person and interviewing tons of candidates. Be focused on your approach. That way, you will find suitable candidates in the best way possible.
Thanks again, and stay safe.
Coffee with a Recruiter 