Hiring software engineers, expectations vs. reality

With the transition from individual contributor to Engineering Management comes the responsibility to hire (usually).

There’s a lot of do’s and don’ts out there, and the books you can find are usually written by seasoned managers, at well-known well-funded companies. The pool of talent to chose from is way bigger in numbers and quality when you’re at a hyped startup in “hypergrowth”.

I’d still recommend those books as a starting point for junior managers (like me). Below are a few humbles experience feedbacks, about how reality can sometimes be trickier than expected.

For the context, I had my first hiring manager experiences in a small (~30–70 ppl) startup, definitely not in the top-10 hyped names, but still building a product in a trendy space: autonomous mobility.
In this context, I had to hire for my team, and as a visiting interviewer across departments.
I.e. in a small company, helping find the best colleagues across the board is everyone’s job.

In the beginning, I found being on the hiring side as stressful — if not more — as being on the interviewing side.
There’s something paradoxical in the process: they say that you should always try to find people that are smarter than yourself. But then, if you actually find such a person, how do you make sure you identify them, don’t look like a fool, and give them the motivation to join your team?

Anyway. As obvious as it may seem, the only thing that works against this stress — and also against a lot of other problems — is preparedness. Prepare material, write down what traits and skills you want to identify (good and bad), and have enough material so you have room to improvise — pick and choose — as I found interviews rarely go as planned.

Quick tips to prepare the process, for first-time interviewers

• Layout a high-level agenda for each interview, depending on the role and process. Make sure that everyone in the hiring process is aware of it, and ready to follow it.
• Write down questions and material in a shared space, where the whole team has access and can give feedback. This will allow everyone to see what will be asked and when. I like to have a simple git repo with markdown files, so everyone in the team can contribute and fix without messing unintentionally with the material in use.
• Keep enough room for candidate questions or to explore unplanned topics that may pop up during interviews (5 min is not enough!).
  I usually keep 15 minutes minimum at each stage. Another strategy is to ask them if they have questions earlier than at the end of the interview, so you can adapt more easily. And their questions might also point to things you should check!
• Open your mind, be humble and curious about candidates and their experience, but also precise and fair in your process.

The infamous coding interview

I’ve been through many mental phases on this topic. And tried different flavors. I’m talking here about actual coding, not technical assessments or discussions.
I found it was pushed mostly by non-technical (like HR) people:

We need to make sure they know how to code…

or

We need an objective comparison scale between candidates…

What I found out:

• both are more or less valid concerns, and it’s normal for people with a basic understanding of tech to view “technical assessments” as the holy grail of interviews.
• Depending on the type of profile you search for, and the time you — and the candidate — are willing to invest in the process, they can be useful. They are definitely neither a “silver bullet”, nor a “holy grail”
• The Internet is very undecided between pros and cons on this topic, leaning on the cons in my humble opinion. My take is that it really depends on how you run it and what you expect out of the coding interview process.
• It’s very hard to convince non-technical stakeholders of the process that coding interviews are not as efficient, reliable, and easy to design as they think they are. “The code either works or doesn’t work, am-I-wrong?”

I could try these different flavors of “code interviews” as a hiring manager over the last few years.

1. The homework assignment
2. The live programming interview (spoiler alert: that one is terrible)
3. The pair-programming session

1/ The homework

Expectations: Given some instruction — the same for all candidates — ask them to write some piece of code that looks like a real task. Grading the result gives you a “code-based, mathematical proof” of who’s the best! (“yikes”)…

Reality:

• It always takes longer than expected for candidates, because they want to do it well. It’s not very respectful of their time, especially if you promote “work-life balance” as a value.
• It takes an awful lot of time on your team to review correctly and give meaningful feedback you owe to the candidate. And you need to do it for each candidate! Prefer doing this at a later stage then, if you’re still undecided between a small group of candidates. Also, grading becomes much more difficult and objective past the “the code works well” stage.
• The code could have been written by the candidate’s sister, who is a seasoned developer or even outsourced to StackOverflow. It will take you a lot of attention to detail and energy to figure out it was the case.
• It actually takes a lot of time to design a balanced coding exercise that requires just enough code to judge, not too much in respect to everyone’s time, and is not freely available with solutions on Github.

2/ The live programming exercise

Expectations: Given an instruction, code some piece of working software while someone is “watching you code”. That way everyone knows for sure you wrote the code and we can grade and compare candidates easily.

Reality:

• It’s stressful for the candidate and not a realistic situation. They have to think about coding, usually on a machine that’s not theirs — or worse: on a whiteboard — while looking good, asking good questions, and describing what they’re doing all the time.
• You won’t learn that much regarding technical abilities if you don’t have a super well-crafted exercise designed to assess exactly the skills you want to know about the candidates.
• Same as for the homework, it’s hard to design such an exercise. To run an interview meaningful enough, you’ll need far more than 1h. I found that the ROI is usually pretty bad.

3/ The pair-programming session

Expectations: Solve some task, either from the backlog or imaginary, in a real pair-programming session. (warning: pair-programming != live programming). A good pair-programming session demands 2 active programmers, 1 being the candidate.
While one is writing code, the other is helping and reviewing e.g., and the roles can and should be switched. You can also do the flavor: one writes tests, the other writes the code.

Reality:

• While it can take more than an hour to do a good pair-programming session, that’s still the most enjoyable experience, that is the closest to reality. I recommend the “real task” one. The ROI is far better than the live coding supervision. Plus the candidate gets a peek of reality.
• It can be unsettling for people not used to that kind of practice, but I got some great feedback out of it even from people not accustomed to pair-programming.
• It takes some time and investment from the team, but the reward and observations are immediate and accurate.
• You need interviewers who know how to lead such session to not end up in a “live-supervised-coding-exercise”

My own conclusion on technical and coding interviews so far

What was the most successful was a combination of technical assessment of candidates — targeted questions without coding — at an early stage, added with a pair-programming session with final candidates.

This was also what candidates told me they had the best experience with. It’s less stressful, closer to reality, assessing real skills, and get them to interact with the team, the product, and the codebase (as long as you can use your codebase for that)

Hire for soft skills, train for hard skills

I hold that piece of wisdom as 100% true. All the problems I had to deal with so far were with colleagues that were hired with yellow flags on personality and attitude, but still scored high on technical interviews.
I never found myself discussing a problematic situation about a team member who’s not “good enough at coding”, once hired. 99% of the problems came from ego, mismatch in culture or expectations, communication, stress, and even one time from an alleged psychiatric disorder!

Of course, candidates need to match the minimum technical requirements of the position. But if you have a hard time deciding between candidates, definitely go for the team-oriented, nice, and funny one that has the fewer yellow flags on personality, will diversify and contribute to your team’s culture!

Time-to-hire and pool of candidates

That one is definitely a problem that they don’t write about in books because at successful well-known companies, they have an almost infinite pool of above-average candidates, a team for talent acquisition, and the nice marketing communication around the company and product that helps to bring new candidates continuously.

They can even do KPIs about candidates and interviews, and be therefore data-driven!

In a smaller startup, time is a critical asset, and at some point, you need to decide on a candidate from a limited sample. Knowing that your startup has a high probability to die or pivot in the next 1–2 years, at some point, time spent hiring is more costly and will most likely end up in a not-so-better candidate.

Plus this is time not spent on the product itself.

You need to identify when you reach a local maxima on candidate quality, and decide wether you accept it or invest — a lot more — on reaching the next local maxima.

You would expect that as long as you don’t find the perfect match, you should continue interviewing because the cost of hiring the wrong person is higher than the cost of continuing the process.
The reality is that as long as your company and product are not well-known and attractive — and I mean attractive to the outside population of potential candidates, not to you — candidates will not fight for that position and good ones will prefer better opportunities. You might just continue the process forever and never reach a higher-quality pool of candidates.
So even if it’s true that the costs of hiring the wrong person are huge, you might not have another choice than placing a bet on the best inline, train and trust, and be ready to react swiftly and decisively if the sh** ever hits the fan.

Make the process enjoyable

Stress is a big part of the process for everyone. Candidates and interviewer.
It makes it difficult to see who everyone truly is — unless you’re a professional profiler — as most gestures and words are calculated.

Expectations are to get quick and precise answers to predefined and precise questions.

It might sound obvious, but the best way to get people to open up is to make the process enjoyable for everyone. Instead of making it feel like a police interrogation where everyone gets to ask questions and scrutinize answers, make it feel like a discussion between peers, even during more formal or technical parts. People will share opinions more easily, and it’ll be easier to see what kinds of people are around the table. It’s valuable for both the candidates and the interviewers to see that early.

Closing thought

There is for sure a lot more to say about hiring, but these were a few points I found important as a first-time hiring manager.

Like most management tasks, interviewing is about humans — yes even for programmers — and unlike code, you can’t compile them and get the list of syntax errors out of a command. It’s a relative grading process, that depends on how you designed the questions, the process, the pool of candidates you can expect to reach, the bias and behaviors of interviewers around the table, and so on…

Don’t expect to have a unanimous YES for a candidate across the whole hiring team easily.
And from the experience of having said YES to a terribly unfit person once, I’d go as far as saying that if you have strong unanimity, there might be something wrong with that candidate and you should be careful!

A few books I liked that touch the subject:

• Managing Humans by Michael Lopp
• An Elegant Puzzle: Systems of Engineering Management by Will Larson
• Work Rules! by Laszlo Bock