« Openess | Main | Oh, the difference a decade makes »

September 15, 2004

Hiring. No False Positives

Other than fundraising, nothing I do as an entrepreneur is as important as hiring. But, hiring is tricky because it's so easy to loosen standards, especially when you're feeling in a pinch for people. But, I always keep two things in mind when hiring, no matter how desperate I feel.

1. a bad employee does far more damage than no employee, no matter the issue.
2. A players hire A players, B players hire C players, and C players hire losers. Let your standards slip once and you're only two generations away from death.

I recently finished How Would You Move Mt Fuji. It's about the role of the Microsoft-style puzzle interview (you know, 'how many manhole covers are there in the US'...). In the book, they mention that Microsoft

"seeks to avoid hiring the wrong person, even if this occasionally means missing out on some good people. The justification is that never before has it cost so much to recruit, maintain, and -- heaven forbid -- discharge an employee"

I like the way David Pritchard, director of recruiting for Microsoft says it.

"The best thing we can do for our competitors is hire poorly. If I hire a bunch of bozos, it will hurt us, because it takes time to get rid of them. They start infiltrating the organization and then they themselves start hiring people of lower quality"

Ding ding ding. We have a winner! Any hiring process should focus on never letting in a bad fit. Even if that means accidentally rejecting a lot of people that would be good fits. Said another way, it optimizes for no false positives, even at the expense of false negatives.

The Google hiring process is notoriously long and complicated. Internally, it's kind of a Liberum veto thing - a single no-vote of the hiring committee means you're not in. Why? Because they put the principle of 'no false positives' to work. They assume that there is a huge talent pool of great people and that they can afford to pass on people that would be great fits in order to make sure they never let someone through who doesn't fit.

Trust me. It's so hard to do. Especially in a startup where you've got much more to do than you have people to do it. But, slip up, even once and it's trouble fast.

September 15, 2004 | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Hiring. No False Positives:

» Google Labs Aptitude Test (GLAT) from Relax, Everything Is Deeply Intertwingled
Joe Kraus has a new typepad about entrepreneurship, and one of his first posts is about Hiring -- No False Positives, a great subject. It's important to remember that there's no one who's great in every environment, so great candidates [Read More]

Tracked on Sep 16, 2004 3:10:28 PM

» Hiring Poorly is Worse than Not Hiring from Business Opportunities Weblog
Joe Kraus, one of the founders of Excite says that hiring a bad employee is far worse than not hiring anyone. He explains: Other than fundraising, nothing I do as an entrepreneur is as important as hiring. But, hiring is... [Read More]

Tracked on Sep 16, 2004 10:19:13 PM

» Hiring. No False Positives. from inluminent/linkblog
Great read, and very true. [Read More]

Tracked on Sep 17, 2004 10:21:46 PM

» Hiring from Information Management Weblog
Joe Kraus on hiring: But, I always keep two things in mind when hiring, no matter how desperate I feel. 1. a bad employee does far more damage than no employee, no matter the issue. 2. A players hire A players, B players hire C players, and C players h... [Read More]

Tracked on Sep 19, 2004 6:54:45 PM

» Hiring - No False Positives from Steve's Synchrono Blog
[Read More]

Tracked on Oct 11, 2004 12:29:49 PM

» No false positives from My blog of HR and technology stuff
Something all recruiters (and managers,CEOs, shareholders etc) fear is hiring the wrong person. When you are very big it does not matter who you hire, if you are big enough you can even screw up with the CEO. (When you do this it seems only the share... [Read More]

Tracked on Oct 18, 2004 3:47:12 AM

» Hiring Practices from David Crow
I long to work for a start-up again, due to personal reasons, I am having to be the responsible one.... [Read More]

Tracked on Oct 19, 2004 2:43:34 PM

» getting the right fit and the right size for project teams from take one onion
A while ago I linklogged the google / hire only good matches meme, from about a month ago. It seemed like such an obvious idea that I was sure I'd heard it before. On reading folklore.org on the early history of the [Read More]

Tracked on Oct 26, 2004 10:02:10 AM

» Hiring: false positives and negatives from Timboy
There's a view of hiring that I think is becoming almost a commonplace in the Valley. It's stated well by Joe Kraus, and also in this [Read More]

Tracked on Dec 8, 2004 1:02:41 AM

» Knowing where your best performers came from...um, yeah...it's important! from Heather's
[Read More]

Tracked on Dec 8, 2004 11:54:09 AM

» Hiring 101 from Relax, Everything Is Deeply Intertwingled
As I updated my Entrepreneurship 101 post, I re-read Hiring: No False Positives by Joe Kraus, and Hiring: False Positives and Negatives by Tim Converse, and I was struck by how challenging hiring is, even if you know what you [Read More]

Tracked on Dec 19, 2004 1:37:23 PM

» IT's hidden truth: everything is too expensive from Open Resource
After just a few days observing an enterprise IT shop I feel as if I may have discovered the universal truth of information technology: everything is too expensive. Every piece of hardware and packaged software, with the exception of some... [Read More]

Tracked on Sep 13, 2005 8:23:23 PM

» Performance hiring from Notes from Classy's Kitchen
Paul Graham thinks you should only hire great hackers, Marc Andreesen thinks you should be very afraid of the law... [Read More]

Tracked on Dec 23, 2005 5:17:02 AM

» "No false positives" - now for the bookmarket from Notes from Classy's Kitchen
The Sunday Times has been having a laugh at the expense of publishers and literary agents by sending in chapters... [Read More]

Tracked on Jan 9, 2006 2:49:01 AM

» gomez article from gomes blog
it's my opinion on that theme [Read More]

Tracked on Apr 26, 2006 8:52:36 PM


Just wanted to post a note to let you know that somebody (at least me) is reading your blog...I've already found your comments interesting and insightful. Can't wait to read more.


Posted by: Jeff | Sep 16, 2004 10:04:13 AM

See also: German insurance giant Allianz will disclose its board members' pay http://quote.bloomberg.com/apps/news?pid=10000100&sid=aOg2TCSxrrs8&refer=germany

Posted by: Michael Nestler | Sep 16, 2004 10:25:37 AM

As someone who has been interviewing with multiple companies in the past months, I can tell you that the process these days might exclude some very qualified individuals. I've been interviewing (coincidentally ad Search Companies since that's my background) into fourth and fifth rounds with tens of candidates. This creates a kind of gamesmanship by the people hiring, which I believe undermines the process.

Posted by: Will Johnston | Sep 16, 2004 3:45:12 PM

In the end, this kind of self-absorbed pretentious hiring process hurts companies. I 100% agree that you must hire A people, sending them though mind-games selects only a certain type of person.

It's not that you're letting some good people slip away, it's that your letting an entire group of good people slip away. These are the people that tend to be the most creative ones, not the puzzle-game smart kind.

Both are needed. Places like Google, Microsoft and Amazon are stacking themselves with one kind of employee, and that type is hiring and self-selecting the new ones. There is no diversity.

In a fast changing industry, diversity and new thinking is critical; Not just solving silly mind puzzles.

Remember, there is a big difference between "book-smart" and "street-smart".

Posted by: Ron | Sep 16, 2004 10:59:50 PM

Good to see you blogging.

Posted by: Whit | Sep 17, 2004 9:39:58 AM

If a company could fire someone during the probationary period without being subject to the threat of multi-million dollar lawsuits, they might be willing to take a chance. Today, it is not worth the risk.

Posted by: Walter Wallis | Sep 17, 2004 3:33:56 PM

One of your former employees doesn't particularly care for puzzle type questions.


Posted by: Zoidberg | Sep 18, 2004 1:24:54 AM

I don't agree with some of your thesis, although it may be because of some fuzziness in terms.

To start, I don't believe that "A people hire A people, B people hire B people, etc.". This assumes that there is one type of person or skill set that is required to succeed and that the 'best' people have the skills to hire other good people. Sounds more like upper class English snobbery than a hiring strategy :).

In my experience, people hire people like themselves. If they hire clones of themselves, you will end up with a group of alike people with the same strengths and weaknesses. Particularly in technical groups, a wider range of qualities is more important than hiring the very best people, particularly when the definition of 'best' is as subject to change as it is in a startup environment.

In several startups I have been involved with, there were virtual clones of the founder, both in personality and skillset. Is this really what is required to win? It may make the founder feel better to have the affirmation of being surrounded by smart people just like him/her, but is this what we need?

As a hiring manager,I actually worry more about false negatives than I do false positives. Too many times I have had my interview teams ding someone for not solving a technical question the same way the interviewer would. Particularly on puzzle questions, where quick thinking and knowing the problem solving trick, is rewarded above a slower, but perhaps better problem solver.

Puzzle questions are an easy out for the hard problem of evaluating people. If I ever need someone to solve puzzles for a living, I will be more than happy to use a puzzle or contrived problem. Until then, I will continue to try and dig deeper, while understanding that hiring people is an imprecise task at best, and we will make mistakes. Part of my job as a manager is to improve or remove the mistakes.

We also have no way of measuring false negatives - those people you reject that could have made the difference between survival and failure. The opportunity cost of the lost contribution of people that were never hired is not measurable. How many people have you missed out on by a hiring process that fears failures more than takes risks on people?

Instead of taking a higher risk hiring strategy, we try to minimize the false positives - still an admirable goal, but one that assumes that there is an interview process that can result in this.

I am not looking for a group of brilliant people who all approach work the same way, but a team that can bring a range of capabilities together to solve business problems. This typically involves a wider range of skills than one might typically assume, both in skills and personalities.

Sorry for the poor writing, this is during a break in a meeting and I don't have time to edit right now.


Posted by: Doc McClenny | Sep 20, 2004 2:56:24 PM

I agree that the difference between good and bad hiring can be the difference between a successful and failed company. However tech companies have evolved complex hiring procedures so they don't have to improve their firing procedures, which, at every tech company I've ever seen, are horrible.

The speculation that this is to avoid "multimillion dollar lawsuits" holds no water. True, executives can pursue those kinds of claims, but that's a special case. The rank and file will just find another job. Most industries outside tech know this well, and don't have a problem with firing people regularly.

The biggest problem with the "no false positives" principle is that there's two issues in hiring: whether someone CAN do the job and whether they WILL do the job. Creating these convoluted puzzle tests just extends the CAN test. There is no WILL test except to hire the person and see what happens.

I agree companies should be selective and thorough in admitting candidates, but the idea that all can be made right through the vetting process strikes me as an idea that only an engineer could love.

Posted by: MB | Sep 21, 2004 10:18:17 AM

I would agree with the position that no employee is better than a bad one. We call this the less-than-zero effect. No employee is zero, a bad employee is -1. How many employees do you really have in the end?

Posted by: Ron | Sep 23, 2004 7:34:26 AM

Taking the approach of "false positive" initially seems to be VERY slow, frustrating and unnecessary. That is, until a company has experienced the negative impact of a damaging hire. This is particularly true when the bad hire is in a management position and propagates the "cancer". I agree that it's better to muddle without, than to bring in a weak hire.

Posted by: Ray | Sep 25, 2004 11:48:07 AM

Just to defend from the people saying,"wow, you'll just hire people who are good at solving brain-teasers," if that's all you're doing, then you deserve to go a very long time before finding the 'right fit'.

But that's not what Amazon, Microsoft, etc., are doing. If that were the case, they'd be just as well-served by holding their fists out and saying,"guess which one holds the penny and you get a job!"

The point of these interviews is to see how a person approaches a problem, and how (or if) they solve it. A huge amount of engineering is estimation and problem solving, so you should test that as well as you can in the interview process. Since you can't have them write a huge application in six hours, you have to fudge it a little bit.

Asking them to write code to reverse a DBCS string and estimate the quantity of rainfall in Washington state in one day is definitely a useful tool in weeding out the non-starters.

Some reasons I've personally disqualified people in "quiz"-type interviews:

1) refusing to write code in the interview (thanks for making that easy!)

2) demonstrating a complete lack of understanding of fundamentals of the problem space. (in one case, not understanding thread safety in an interview for an application server team lead.)

3) resorting to personal insults when challenged on their answer to a problem.

What you're looking for isn't just the ability to answer a "stupid" question or the ability to see "the trick". It's attitude and aptitude. I'd much rather work with someone who's willing to try, be a good sport, and not be a pushover, than someone who lacks a single one of those traits.

Long comment, should have tracked back, sorry.

Posted by: Art | Sep 28, 2004 5:23:36 PM

Luke Hohmann www.lukehohmann.com used to have a paper up called something like "The Roman Evaluation Interview Method". Same idea. He makes the case from a team and trust building standpoint. In other words, hiring a bozo over the objections of your people will spike your team. It's basically the self-selecting team idea by Alistair Cockburn. Having used this method, I can say it works great.

During the heyday, about 1/5th of my time was spent recruiting, staffing job fairs, doing phone interviews, and so forth. Though I'm not a big fan of Mensa-style interview questions, mostly because I suck at logic problems, I did compile a list of "fast rejection" questions to filter candidates. Stuff like "how do you model a many-to-many relationship in a RDBMS" and "what's the difference between static and final?" I had questions for dev, QA/test, tech supp, docs, and I18N candidates. We also used questionnaires and tests (programming, writing, etc.).

Fortunately, that particular company had the foresight to document the interviewing and hiring processes. We had checklists, templates, and support from human resources. We met semi-regularly to refine stuff. We even had a draft a skillset scoring system with topics broken up into categories (e.g. computer science, software engineering, math, communications, creativity).

Posted by: Jason Osgood | Sep 30, 2004 5:12:49 PM

Very interesting blog, Thank You!

As far as your comments on the hiring practices of Microsoft, Amazon, Google and presumably Excite, I would love to see a scientific study of the efficacy of these hiring practices. Interviews should not be about solving puzzles under pressure. I have found many interviewers possess no skills when it comes to interviewing. The questions are either puzzles that were picked from a book or a website or tend to be on the lines of "What is ACID?", "What is MVC Model II?", "Write a string reverse function" or even worse "What is little-endian vs. big-endian?". Rarely are the questions designed to find the candidate's knowledge, elicit deep insights of engineering discipline, understand their creative approach and get to know the temperament of someone, all in an hour or less.

In my opinion, the first thing that an interviewer must do is to stop trying to stump the candidate. S/he should make an attempt to determine the candidate's area of expertise and/or most recently worked on areas and focus on that. This can be pretty hard to pull off when there is no common area of expertise.

After having worked for 22 years with my share of engineers, I can confidently say that interviews are very ineffective at picking the right people. There is no shortcut to working with someone over a period of time and observe their work ethics, team skills, tenacity for solving problems and putting in the extra effort when necessary. I would rather go with a trustworthy reference than base a hiring decision on my limited interaction with the candidate.

As far as Google hiring goes, why is it that it hard to even get called for an interview even when they are constantly soliciting people? Could it be that I have nearly 22 years of experience and thus don't fit their ideal profile?

Lastly, I would recommend reading some studies done on IQ determination and the education system in general(sorry, don't have the references handy). I think a lot of that applies to the hiring process as well.

Posted by: Praki Prakash | Sep 30, 2004 9:17:15 PM

While you are interviewing me, I am interviewing you. I can't afford to work for a company that will ruin my career.

I remember getting referred to a company for an interview. The person that referred me told me that their response was "not technical enough." Well, I'm more technical that the person that referred me. And, I got the same response. I'll tell you that it looked like agism, but it was culture change instead, aka "old people like me don't know XP" or "old people like me push project management." You'd be surprised.

The problem for this employer was that they couldn't ship without my piece of the application. Six months later they still had not shipped. The company? Oh, well, they died on the pike of "not technical enough."

Frankly, everyone even a janitor has their own technology. Not everyone in the company has to be a mathematician, a programmer, or a person that plays team building games.

My rules: never miss a schedule, ship on time, avoid the egos. That probably doesn't make me an A player. But, A player companies can kill themselves. I've seen it enough.

There was a time when there were no A player software companies. A players milked the cash cows. And, frankly, those were the days.

Posted by: David Locke | Jan 16, 2005 7:00:04 PM

www.salleh.blogfa.com pay to read link

please the me call : 1 site good for pay to read email

Posted by: saleh | Jun 6, 2005 3:05:54 AM

I've found your comments interesting and insightful. Can't wait to read more.



Posted by: tech jobs man | Jun 23, 2005 10:32:59 AM

The comments to this entry are closed.