Microsoft totally changed how it interviews software developers to make sure candidates have the actual skills to do the job

Getty

Microsoft CEO Satya Nadella

• Under new CEO Satya Nadella, Microsoft has tried to reform its corporate culture as it tries to get the whole company rowing in the same direction.
• To that end, Microsoft in 2016 started working on the "Alternative Interview Framework," a new way of conducting developer interviews to reduce bias and do a better job of testing candidates for the skills they'd actually need on the job.
• The team tested the interview process on themselves first, and improved it as they went along.
• After rolling out the process to actual candidates, the team received positive feedback - we spoke to one Microsoft employee who says she chose to work there because the new process made her feel so good about the job.

Under the five-year reign of CEO Satya Nadella, Microsoft has made major strides in reforming its once-cutthroat corporate culture - an effort that's even seeped into how it interviews software developers.

John Montgomery, partner director of program management at Microsoft, tells Business Insider that the core of cultural change is about making sure that teams are working with each other, all over the company, solving real problems for real customers. And so, Microsoft began rethinking its interview process to match that goal.

Traditionally, interviews at Microsoft and elsewhere can be highly technical - like, "reverse a linked list" - and include math games, like asking candidates to figure out how many ping pong balls would fill a 747. Both these types of questions really have little to do with what employees would actually do day-to-day at work - but even Google decided to discontinue its infamous brainteasers, because they didn't actually test for anything worthwhile.

To that end, starting in early 2016, Microsoft's developer division began rolling out the "Alternative Interview Framework," which Montgomery detailed in a Medium blog post. It's designed to better match an applicant's skills to what the job really requires, says Montgomery, who spearheaded the effort as it spread through Microsoft's developer division.

"We wanted it to be more like what people are expected to do at work," Montgomery told Business Insider. "People are expected to collaborate. You can ask questions. We wanted it to be more like what you're expected to do at work."

The 'Alternative Interview Framework'

Under the new process, Microsoft shares the interview questions in advance so that candidates can prepare. During the interview itself, a candidate might run through a real scenario or problem the team is trying to solve.

"When you walk a candidate through that process, every candidate brings something to each of those phases and each of those interviews," Montgomery said. "Some candidates are great at a particular phase but not so great at others. It still makes them great hires. From that perspective of inclusion, it enables us to ask a broader array of questions."

What's more, rather than doing things one-on-one, a candidate talks to two interviewers at once - usually a senior employee and a junior employee. This gives the team two opinions on the candidate to consider, and it can actually make the candidate feel more comfortable as they get double the perspective on the team they're trying to join.

Furthermore, the team uses blind feedback, meaning that all interviewers withhold their opinions on the candidate until the process is complete, at which time they all pool their opinions and come to a final decision. This can help reduce bias: It means that a second- or third-round interviewer won't come into a meeting with the candidate with any preconceived notions from the first or second.

Redesigning the interview process

The team first tested the process on its own members over the summer of 2016, says Montgomery, interviewing each other to see if worked in practice as well as it does on paper.

"We haven't had that much fun in a long time," Montgomery said. "We also tested interview processes on people who are more senior. It's a lot of fun when you're effectively interviewing someone from your own team. We kept on learning and iterating to make it better."

In so doing, says Montgomery, the team tried to apply the principles of product development to refining the new interview process. As they went, the team constantly reassess what was working and what wasn't, iterating on the process on the fly.

In the last half of 2016, the team finally started running its first candidates through this brand new interview process, which has gradually expanded to more of the company since.

There's still more Montgomery would like to improve on. Right now, the interview process is long, requiring candidates to take out an entire day. He thinks it would be useful to figure out if there ways to either shorten the process entirely, or at least spread it out over multiple days and gives candidates more time to reflect and think.

Also, he'd like to improve on making the scenarios more realistic and approachable for people who don't have a background in professional software development, but may still be qualified candidates.

"We want to design an interview process centered around the interviewee," Montgomery said. "It's a manifestation of Microsoft culture and being more inclusive in identifying talent that a normal interview process might not identify."

More empathy

Montgomery believes this new framework is more inclusive: By relying more on finding solutions to real problems, and less on esoteric or in-depth technical knowledge, it's easier for people who didn't come from traditional tech industry backgrounds to shine. Microsoft also works to make sure the interviewers reflect the company's diverse workforce.

Montgomery points out that he himself took an unusual path to Microsoft. He majored in Russian literature, and then worked at trade press magazines and startups before joining Microsoft in 1998.

"I am introverted and I like to have time to reflect," Montgomery said. "It turns out that that's true for a lot of people. Everyone needs time to think. It has helped hugely for us to find people who are a little more thoughtful but good at what they do."

Carol Smith, a senior program manager at Microsoft for the Open Source Programs Office, was one of the first candidates hired through the Alternative Interview Framework, and she says that interview process was the reason why she chose to work at Microsoft.

"The scenario-based interviewing process gave me an excellent sense of what my job at Microsoft and working with my coworkers would be like, much more so than any other interview process I've ever gone through," Smith said. "It was the most difficult interview process I've ever gone through, and it was also the best."

You can read Montgomery's full blog post here.

Get the latest Microsoft stock price here.