Hiring: Sabermetrics for startups?

If you pitch a venture capitalist for money, be prepared to hear sports analogies. "When are you going to put the puck on the ice" (when are you going to launch). "You need to move the ball downfield" (self explanatory). So, when I saw boxes and boxes of Michael Lewis' book Moneyball stacked up in Accel Partners' offices, I figured it was merely an attempt to brush up on new baseball analogies for their next crop of startups.

However, when I read the book, I was fascinated. On the surface, Lewis' book is a baseball book. But, the more I read, the more I was convinced that there was a great startup lesson contained in its pages. And that's when I understood why Accel was distributing the book.

As I mentioned, the book on its surface, is about baseball. Specifically, it asks the question of how the Oakland As, with the second lowest payroll in baseball can consistently make the playoffs against competition that is far better funded. (Poor entity competing against a rich entity? Sounds like a startup problem to me.)

To set the stage, Lewis explains just how great a discrepency exists between the highest and lowest payrolls in baseball by contrasting baseball with other sports. In football, the ratio of highest to lowest payroll is 3 to 2. This means that the highest paid team pays out 1.5 times what the lowest paid team does. In basketball, that ratio is 1.75 to 1. In baseball, it's 4 to 1. 4 to 1!? I couldn't believe it. Talk about an inherent disadvantage.

So, how could such a financially poor team like the As fare so well? The answer, it turns out is by exploiting a baseball market inefficiency. According to Moneyball, most baseball scouts, the guys who pick talent, look for the same things; things that "feel" right. How fast a player can run? How powerfully can they hit? How fast can they throw? How well can they field? How much do they have the "look" of a baseball player? It's an old game, with lots of traditions and common wisdom. Everyone just "knows" that these things matter.

With all the scouts looking for the same thing, those players who exhibit the commonly looked-for traits are (of course) highly priced. No poor team could afford them, including the As.

But, what if everyone was looking for, and pricing highly, the wrong characteristics? As it turns out, in baseball, the scouts were doing just that. They were highly *overvaluing* players based on a set of "common knowledge" that had very little correlation with winning. Turns out, if you do the math and run the stats, how well you run the bases, how fast you throw and how hard you hit have very little impact on winning baseball games. In fact, the math (apparently) shows that fielding doesn't matter in the slightest.

So, did the scouts believe the math? Of course not. They had tradition and years of experience and wisdom on their sides. They continued to value the players that "felt" right or looked right. The only teams that did believe the math were the teams that couldn't afford not to believe it.

The As were one of those teams. They couldn't afford to pay what the market was highly valuing. So, they set about doing a fact-based statistical analysis and picked players that no one else wanted, but which the math suggested would highly influence the team's ability to win games.

And win they did.

So, what does this have to do with anything startup related? Well, I started to wonder if there was a similar market inefficiency in the people market for technology startups. Were certain characteristics in people highly overvalued relative to their impact on a company's success? Conversely, were there characteristics undervalued in the people marketplace -- hidden gems waiting to be found that could have a radically positive impact on a company's trajectory? If this was the case, how could I identify these so that I could avoid the overvalued people and zero in on the undervalued people?

Well, I don't have an answer just yet, but I'm thinking a lot about it. In general, I think that the technology people market overvalues certain VP-level jobs, typically in marketing and business development relative to these positions impact on a company's success (how often have you seen the "killer" VP of marketing get brought in with some huge salary and equity package to save a company and end up not having much effect?) On the other side, I think the market generally undervalues key engineering hires relative to their contributions.

Why does this happen? For a few reasons, I think. First, most CEOs are not technical (I'm not either). So, they tend to highly value the things they understand (marketing, business dev, sales) and undervalue the things they don't (engineering). Second, people are more attracted to people like themselves. Come from marketing and you'll probably pay more for marketing people. Third, and perhaps most importantly, is the fact that there are no identified fact-based metrics that help CEOs understand how to value engineers. Let me see if I can explain.

In Moneyball, Lewis describes the search for the statistics that matter - those that are highly correlated with scoring runs and thus winning games. And, it turns out that there are two in baseball -- slugging percentage and on-base percentage. Find these characteristics and you've got a strong likelihood that the player will contribute positively to the team.

What are those stats for evaluating a potential engineering hire? People might be willing to pay more for a great engineer (pay what they're actually worth for example) if they could believe with a high degree of confidence that this person will actually make the difference they are paid to make.

I've been wondering if there are such engineering "stats" that are likely indicators of significant future contribution to a startup. Anyone out there got any ideas? For example, does being a commiter to a large open source project have a strong correlation with likely success inside a company? How about a MS degree in Computer science from a top 20 school. Is that a leading indicator? What about coding for fun? If someone codes for fun (as opposed to just for work) is that person more likely to make an unusual positive contribution to the success of a startup?

I'm really curious if there are items that correlate. I'm tired of guessing...

The study of baseball statistics is called Sabermetrics. Has anyone done Sabermetrics for startups?