« Keeping Innovation Alive - The Hackathon | Main | Personal - Joining the EFF Board »

June 21, 2005

Engineer Interview Triage?

A while ago (actually, any post I've done is now "a while ago...") I wrote about Sabermetrics for Startups. I wondered if there was data you could collect in an interview process that would allow you to more accurately determine if someone, particularly an engineer, would be successful in your company.

I don't think I have all the answers, but over the last few months, I've honed in on three questions that I believe have a correlation with three key skills.

I'm not talking about technical skills. There are a lot more people far better than me to judge whether or not someone is technically qualified as a great coder.

I am talking about the intangibles. In particular, I'm talking about three key intangibles -- communicating, tinkering and passion for coding.  In my experience, these things make a huge difference in someone being a great contributor to your startup.

So, here are the questions. They're simple and they aren't pass/fail. But, I think certain answers are more correlated with success.  So, pretend you're in the hot seat, bright lights, uncomfortable chair... you get the idea. Here goes

1. Do you have a blog?
It was Joel Spolsky who wrote a great piece about great engineers being defined not only by their h4x0r skillz, but by their ability to communicate. Here's what the man himself had to say

The difference between a tolerable programmer and a great programmer is not how many programming languages they know, and it's not whether they prefer Python or Java. It's whether they can communicate their ideas.By persuading other people, they get leverage. By writing clear comments and technical specs, they let other programmers understand their code, which means other programmers can use and work with their code instead of rewriting it. Absent this, their code is worthless.

If someone has a blog, you know that they are starting to make communications and writing part of a basic set of habits. You know they value those habits enough to make time for them. A public blog improves the odds that the person sitting across from you (who has great coding skills) can also effectively advocate their ideas both inside and outside the company.

2. What's your home page?
Great engineers make their own homepages. When they hit the "home" icon on their browsers, you're not likely to see My Yahoo or Amazon.  They're disatisfied with their other choices out there and they take matters into their own hands (usually just a large list of links of favorite places to go, laid out "just right"). My friend, Marc Hedlund put it this way, "Jedi Knights make their own lightsabers and great engineers make their own homepages." How true.

I think the trait indicated by making your own home pages is that the person is a "tinkerer". Tinkerers are great inside companies. They're curious. They're often not quite satisfied with the status quo and doing things the way others do. They're the ones that aren't often satisfied with the way your company is doing something. But, rather than complaining or asking, they go ahead and just fix the problem.

It's hard to know if the person sitting across from you is a tinkerer, but if they make their own home page, it's more likely that they are.

3. Do you contribute to an open source project?
One thing you're looking for in a great engineer is a person who is passionate about coding. Passionate doesn't mean all-consumed-and-working-24-7, but it does mean curious, deeply interested and committed. Besides the obvious benefits of being able to review someone's open source code for quality, design patters and architecture decisions, contributing to an open source project has a strong correlation to the person being passionate about code. They're less likely to just be about code-for-cash (not that there is anything wrong about that, it's just not usually right for a very small startup). That intangible, code-as-passion, can make a huge difference to a startup.

So, that's what I think. It's only been a few months of thought. If you've got other ideas, I'm all ears.

June 21, 2005 | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Engineer Interview Triage?:

» Bnoopy: Three questions for recruiting programmers from Steve's Mindfire Blog
[Read More]

Tracked on Jun 22, 2005 5:40:39 AM

» Passing Interview Triage from Jeremy Zawodny's blog
Joe Kraus (JotSpot) write about the three questions he uses to help assess engineers during the interview process. I was glad to see that I had reasonable answers to all three. :-) Do you have a blog? I do. You're reading it. What's your home page? As ... [Read More]

Tracked on Jun 22, 2005 6:15:37 PM

» How to Find People With Passion from Joel Ross
[Read More]

Tracked on Jun 22, 2005 9:22:33 PM

» Interviewing software engineers from heat death of the universe
This has been making the blog rounds the last few days. Joe Kraus has three interview questions can be used to tell whether an engineer might be a good fit for a start-up software company: Do you have a blog?... [Read More]

Tracked on Jun 23, 2005 8:56:39 AM

» Three Questions from Hasan Diwan on the Web
Joe Krauss asks three questions of engineers before hiring them. Namely "Do you have a... [Read More]

Tracked on Jun 23, 2005 2:48:43 PM

» Cercasi sviluppatori PHP from Alberto Mucignat
L'azienda per cui lavoro sta cercando sviluppatori php. Se qualcuno è interessato può inviare il suo curriculum: per piacere è preferibile in formato pdf oppure word, ma senza inutili colorazioni (non vi dico cosa mi arriva certe volte). According to J [Read More]

Tracked on Jul 19, 2005 7:26:28 AM

» Has Blogging Peaked? from Zoli's Blog
Some predict blogging soon peaks or already has. I disagree. This article discusses why blogging for $$$ will crash and burn a'la daytrading, and how then blogging will be back t "normal": a vehicle for self-expression, exchange of ideas, social and p... [Read More]

Tracked on Jul 23, 2005 1:46:55 PM

» Open Call: Developer and Partner from TripTie
I am looking for an experienced developer and partner; someone with a passion and propensity to code. ... [Read More]

Tracked on Aug 7, 2005 9:15:13 AM

» How to hire a product manager? from Harry Nieboer
[Read More]

Tracked on Sep 22, 2005 2:02:38 PM


Does 2 outta 3 count?

Posted by: pFadey | Jun 22, 2005 10:30:24 AM

I don't have a blog. I use google as my home page. I don't contribute to open source projects because there are other things in my life than the computer.

Does this mean I can't communicate, I'm not a tinkerer, and I'm not passionate about what I do?

I don't think so.

These questions are pretty narrow/specific. Having a blog, to me, almost signifies this person is not a good communicator becuase they are using the internet as their means of communcation. Get out of your chair and go talk to someone.

I tinker with tools to help me get my job done, but it just so happens that my browser home page isn't one of them.

I don't do work outside of work becuase there are a ton of way more fun things to do. I really like building apps, but at the end of the day, it's time to go have a beer, go to the gym, be social, watch the Yankees, whatever.

I think someone answering "yes" to these three questions has a better chance of being a heads-down-can't-communicate-nerd than someone who can understand business requirements and turn them into a working piece of software.

Posted by: ben | Jun 22, 2005 11:33:08 AM

I think trying to come up with $n$ questions that reflect a good/bad candidate for *any* position in *any* company, where $n$ is a finite number, is a very tough task! :-)

These 3 questions are good ones.... but am sure some very good people and majority of techies will fail this test! :-(

Posted by: Sumit Chachra | Jun 22, 2005 1:26:53 PM

Ben, the world is flat. Do you get the paradigm shift?
"The playing field is being leveled, says globalization guru Thomas Friedman - from Shanghai to Silicon Valley, from al Qaeda to Wal-Mart."

I live in Bulgaria. Do you know where Bulgaria is? I track 5359 RSS feeds, 797 Orkut communities, 490
Tribe.net groups and many other sources.

Can you compete with me?

Posted by: Dimitar Vesselinov | Jun 22, 2005 1:36:20 PM

I ask one question in every interview: "Forget about this position for a moment. What do you REALLY want to do?"

If they proceed to kiss up and say this job is really what they want to do then they're out.

If I get a blank stare, I know they can't handle pressure AND they're uncreative.

Having an answer to that question, even one that's broad or fuzzy is huge. I want to find people who can think around the corner. I can help them learn the rest.

Posted by: Scott Palmer | Jun 22, 2005 1:48:23 PM

Another exemplary startup post, Joe. As an advocacy blogger, I would insert the Joel Spolsky quote in a broader context: "By presuading other people, you not only promote yourself and your cause, but give your employer leaverage as well."

Posted by: jim sweeney | Jun 22, 2005 2:02:43 PM

I agree with the underlying points about communication and tinkering, but a blog isn't the only way for someone to demonstrate the ability to communicate.

Instead, you could ask a candidate to submit a code sample from a non-work-related project, perhaps with some sort of a design document. (Choosing a non-work-related project will let you see if the person likes to tinker on his/her own time.)

Posted by: Larry Rosenstein | Jun 22, 2005 2:08:38 PM

I know i'm really good at coding. I'm very passionate in this field and I am a tinkerer. I think of myself as a creative problem solver.

I don't have a blog. I don't have time to blog. I do read a lot of blogs though. 600+ blogs on my aggregator. Reading all of them takes a lot of time.

I don't have my own customized homepage. I have Yahoo as my homepage. I do have the bookmarks pane in Firefox open most of the time and I do have a routine list of websites I visit daily to get my daily information.

No. I don't participate in open source projects. I don't have the time.

I guess I'm going to flat-out fail your test. I won't make the cut.

Want to hire great engineers? Post a problem that needs to be solved on your website. You'll weed out 80% of the would-be applicants. You'll be surprised at how few of them would even make the attempt to solve the problem. You'll get a lot of applications with cover letters and resumes but with no solution to your problem.

Posted by: Anonymous | Jun 22, 2005 2:13:26 PM

There is only one question that matters. What have you shipped?

I haven't had a homepage since 1997. I don't contribute much to Open Source because I like to get paid to write code. But I do have a blog.

Posted by: Christopher Baus | Jun 22, 2005 3:41:04 PM

This is a response to Dimitar Vesselinov's comment.
I don't think tracking '5359 RSS feeds, 797 Orkut communities, 490 Tribe.net groups' is any kind of achievement worth harping about.

In fact, it seems like you spend more time feeding yourself information and very little time digesting it.

If you want to talk about your achievements, tell us how you used this information to accomplish something tangible. Till then, don't expect to be taken seriously.

PS: Why assume people don't know where Bulgaria is ? Is it such a crappy, obscure place that you won't expect people to know about it ?

Posted by: Anuj Agarwal | Jun 22, 2005 4:44:59 PM

I have been fortunate enough to have worked
with some very smart and creative people.
They are simply not interested in sharing
every little thing that they do with the
rest of the world.

May be the above questions would help an
interviewer to psyche the candidate and
do some pointless arguments in the name
of a "healthy discussion" with the candiate !

Posted by: AM | Jun 22, 2005 6:46:18 PM

i find it strange that you titled these 3 qualities with examples rather than the underlying concepts.

i strongly agree with your 3 main concepts; communication, initiative and passion (well, thats what i got out of it). and also that you're emphasising that you can't simply *ask* someone if they have these qualities... an employer has gotta look deep for tell tale signs of their existence.

i wouldn't want to start up anything without being surrounded by very passionate people who make me feel lazy... no matter how hard i work :-)

but having a blog does not mean someone has communication skills. i personally find it very difficult to communicate using only text (i like to see/hear feedback as i talk), so i set up a blog to get better at communicating to a blank wall (i can't see the audience so i always feel like i'm writing a message and sending it into the abyss rather than publishing it for potentially the world to read). but it doesn't mean i *am* any good at communicating to the blank wall, nor does my blog reflect my person-to-person skills in any way shape or form. text input froms are a totally different world than real life, and i prefer people who can tell me everything they need me to do over a beer/coffee, not over a LiveJournal thread.

i respect blank home pages (faster browser startups) and lots of keyboard shortcuts. tinkering is good. but tinkering TOO much leads to lack of productivity, which leads to the dark side. (especially if someone blogs as well!). if someone has slashdot as their homepage i think to myself "hmm, they need a few years to mature" :-)

OSS contributions, yeah thats cool. but i prefer the guys who don't have their names plastered all over the mailing lists... i'd rather see them in the CVS activity logs. of course, OSS is not necessary. just that they are in some kind of volunteer-based project and are active.

this article was a good read, i'll certainly be adding this to my RSS feed. i'm about to start up my own consultant business adventure... not as grand as other people around here, but this advise is all very interesting nonetheless (and free!). thanks guys!

Posted by: Sam Halliday | Jun 22, 2005 9:20:02 PM

Thanks for all the comments. One thing I wanted to clarify. These questions are not pass/fail. If the candidate doesn't have a blog or doesn't make their own home page or doesn't contribute to open source, it's not the end of the interview. In fact, while many engineers at JotSpot do the things mentioned above, several do not and I think everyone of them is a total rockstar.

I was just saying that people who do these three things seem to have a higher correlation with success inside the companies I've been a part of or invested in.

The nice part about the questions, in my opinion, is that they point to behaviors which are:

1) not motivated by trying to "succeed" in the interview and are therefore more likely to be indicative of the core motivations of the candidate

2) objectively observable to outsiders (like me)

Posted by: Joe Kraus | Jun 22, 2005 10:23:15 PM

As I noted on Jeremy Zawodny's blog, I have to admit I think it's mostly bunk. Blogs don't tell you how they communicate, just whether they feel like sharing their thoughts openly.

The 'personalized home page' is one I'd really strongly disagree with, as the efficiency of the pure google homepage as a jump-off point is nearly unmatched. Sure, I tried building the 'rss feed homepage' once, but when my feeds expanded past a few dozen, it became useless, and the efficiency of the search engine homepage beckons.

Now, as for open source projects, I'd rather ask, 'Tell me about your programming outside of work.' If they get excited, thrilled about their own personal coding, and can expound on it, then I'm glad, and it tells me more about them as a programmer. If it's an open source project, then you can learn the communication skills by reading their checkin comments. In fact, those are far better than a blog, for knowing how they can communicate.

And to the person who commented about not programming otuside of work, no I wouldn't hire a programmer for a startup someone who doesn't intensely love to code.

-- Morgan Schweers

Posted by: Morgan Schweers | Jun 22, 2005 11:14:33 PM

I don't understand. What makes anyone thing a "great engineer" has to have *anything* to do with the Web? Must everyone capable of writing a coherent sentence necessarily feel impelled to (a) run a website at all, (b) want to put up something two-way on it, and (c) feel egotistical enough to think anyone else is interested in hir daily babblage?
I think that thinking having a blog says diddly says more about you than about anyone you think fails the test.

Posted by: der Mouse | Jun 23, 2005 1:54:22 AM

"What makes anyone thing a 'great engineer' has to have *anything* to do with the Web?"

My answer:
Tim O'Reilly - The Software Paradigm Shift

Web 2.0 Conference 2005

Posted by: Dimitar Vesselinov | Jun 23, 2005 3:08:43 AM

My personal evolution has taken me beyond a homepage filled with categorized shortcuts from which I can make outbound stab-in-the-dark searches to find information.

Much of the information I want now comes to me via my aggregator (I use Bloglines). What remaining outbound searches I make are now made with A9 (using Google's engine) and technorati/feedster/del.icio.us/flickr + tags.

Any valuable engineering tinkerer will have realized the power of aggregation and taxonomy to reduce the drudgery of iterative search. We're all lazy buggers, after all.

Posted by: Brent Ashley | Jun 23, 2005 5:47:06 AM

0 for 3, but with explanation:

1. No blog. On the other hand, a quick search at Google can demonstrate that I have a long history of actually communicating (discussion, not monologues decorated with graffitti) in fora like Usenet and various mailing lists, and a lot of that discussion is on technical subjects, but it is certainly not all technical. A blog is not a bad thing, but it is really a very tilted forum. It's like a theater (or sometimes a temple) instead of a public square or a living room. On the other hand, maybe that's the best one can hope for from a population with a dominant pattern of social discomfort: find the guy who acted as DM for `the D&D games, not one of the quiet players...

2. Historically my start page for web browsing has been blank, for faster browser startup. A page full of links is pretty pointless when there's one I can pull down and have disappear as needed (i.e. a Bookmarks/Favorites/Hotlist menu, depending on which browser I'm using...) With tabbed browsing and support for the starting 'page' being a tab set, I use that in some situations such as the workplace, where a handful of starting points defines my normal web usage. Building one's own starting page might say that one is a tinkerer but it also may indicate a certain egotism that insists on re-inventing the wheel just to be able to say that it is MINE, rather than building on what others have provided as a flexible toolset.

3. I've contributed some bug fixes and similar limited things to open source projects that I use, but the limiting factor isn't "passion for code" at all. A lot of people with real jobs work under terms that make it very difficult and sometimes impossible to commit any significant effort outside the job to anything similar in nature to the job. For exmple, if I were to take a significant role in any OSS project, I'd need specific permission to do so from my management and a specific disclaimer of any rights in that work. That's not because I work for an evil company, but because I work for a company that has learned that they need to protect themselves. In addition, I have a full time job (which often means well over 40 hours per week) plus I have a family. There's the issue of balance and priorities, and it is simply not possible for me to keep up with my existing work and family obligations and give suitable attention and focus to some other project.

I'm not saying that these three questions completely suck, because they really don't, but it would be a gross error to ask them as yes/no questions without digging a little deeper.

Posted by: Bill Cole | Jun 23, 2005 6:45:50 AM

There have been a couple of comments along the lines of "I don't do work outside of work because there more fun things to do." or "I only code for money." The kind of person a startup needs knows that creating software isn't work, it's the most fun thing ever. It comes down to: Do you code because it's a job, or do you code because it hurts if you don't?

I've been using "do you answer questions on newsgroups/mailing lists", "do you have a blog", "what have you coded that you weren't required to code (open source or just personal)" and "what's your favorite software-related book/web site" as intro questions for a while. Of course, you can't really know how good a developer somebody is from a couple of questions, but having interesting answers to those questions has turned out to be a good indicator. (And if the interviewee has contributed to an open source project, it gives me a chance to look at their code, which is a definite bonus)

I know it's not fair to reduce a person to the answers to a couple of questions, but as we're going through hiring-hell right now, I understand intimately the need for some sort of triage.

Posted by: Christopher St. John | Jun 23, 2005 7:46:18 AM

Here's another 0 for 3.

I have excellent communication skills, verbal and written. But I don't blog. I'm a private person, and also I'm not egotistical enough to think that everyone cares about every trivial thought that dribbles out of my keyboard. There was a great article in The Atlantic about "Caring for your Introvert", and it said that extroverts automatically think there's something wrong with anyone who doesn't constantly broadcast every single thing they're thinking. Meanwhile, we introverts are silently thinking "shut up, already!"

My home page is Google. Far more flexible than anything I could put together. I did make an intranet site with that kind of stuff, mostly for HTML/CSS/JS practice.

I do my own side projects, but haven't added anything to an open source project. I think it's a good sign if someone does, of course. Mostly I've been lucky enough to have mind-expanding projects at my various jobs.

I like the spirit behind the method - trying to find the real person and not just the surface credentials - I just disagree with the questions.

Posted by: tom biggs | Jun 23, 2005 8:18:50 AM

Also, to those who mention having a balanced life - house, kids, real-world activities - note that the subject here is startups.

People who run startups often *are* looking for people who don't have outside demands on their attention, so they can work the expected long hours, crash, then come back and do it again the next day. I'm not going to pass judgement on that. To each his own. I prefer a more balanced life, and my family is glad for that.

But that's all irrelevant to someone who's hiring for a startup company. "Can you give us your entire undivided attention? If not, we'll hire the next person who can." One VC guy said he'd check a potential company's parking lot on Sunday morning; if he didn't see cars there, he wouldn't invest.

It's certainly exhilarating to be a part of something like that. I've done it in two-month bursts. But I find that there's more value in paying attention to the whole world, with my joy in programming being a bonus.

Posted by: tom biggs | Jun 23, 2005 8:45:25 AM

Louis Monier. The perfect candidate I would hire. What an inspiration!

Louis Monier On Why He's Going To Google

Posted by: Dimitar Vesselinov | Jun 23, 2005 5:26:20 PM

On a more skill-oriented and experience oriented level, one of my favorite questions is to hand a developer a copy of an invoice and ask the person to sketch out the model for me.

It's amazing to me how many "developers" can't do that. There are lots of developers out there tweaking other people's code but they aren't really creating new solutions. I ask this question early and stop the interview immediately if they can't give me a reasonable model.

Posted by: Ginger | Jun 24, 2005 8:22:11 AM

On a more skill-oriented and experience-oriented level, one of my favorite questions is to hand a developer a copy of an invoice and ask the person to sketch out the model for me.

It's amazing to me how many "developers" can't do that. There are lots of developers out there tweaking other people's code but they aren't really creating new solutions. I ask this question early and stop the interview immediately if they can't give me a reasonable model.

Posted by: Ginger | Jun 24, 2005 9:00:47 AM

Hey, der Mouse,

If you're the same der Mouse I know from Bugtraq, I don't think you have to worry about not having a blog should you apply for a job at Jot.

Just be sure to let us know that it's you ;-)


Posted by: Alex Russell | Jun 24, 2005 11:57:09 AM

The comments to this entry are closed.