Interviewing is a perennial topic because it doesn't make much sense.
Case in point - over the past week I've interviewed for four positions. Three gave me offers within a day, with very little effort on my part. On the other hand, one told me that I had little idea of what I was doing and told me to study up and apply again next year. And, that was after wasting my time with 3.5 hours of interviews. Of the offers I received, two were well over 120k plus great bonus. The other came in under 90k but told me that I could work anywhere in the world, whenever I wanted, as long as I got my work done.
Over that same time, I flat out rejected three recruiters from large engineering companies because their recruitment processes would have cost me $700-1500 just in opportunity costs. And, they weren't even willing to give me the info I needed to help me determine my chance of success or even my expected pay.
Everyone these days seems to believe that their company is so important that they deserve or need a team of top 1000 engineers. When in fact a top 20-percent team who bonds well together would likely be sufficient.
The topic of interviewing is fascinating because there seems to be very little science behind it. Who knows what works?
The opportunity cost is the most frustrating aspect. I've solved several "sample" problems from companies, and been involved in full day interviews writing code on the whiteboard. The worse was spending a day writing some sample code, then doing a panel interview where I was asked to make a change to the code to cover a non-sensical edge case. I was rejected because I "hesitated".
People make several mistakes when analyzing interview techniques:
1. The interviewer and interviewee are after similar and complementary things.
Wrong. The interviewee is trying to find out if the work is interesting, whether or not this would be a good place to work, whether or not he or she could work in that environment with that team and so on.
The interviewer (or rather the employer) is trying to fill a position. This is really important.
Some make the mistake of thinking that if an interview process doesn't evaluate the interviewee accurately it has failed. This is grossly inaccurate. The point of an interview process shouldn't be to look at a single candidate but to look at the process of filling a position, which may well span interviewing many candidates.
A false positive (someone who looks qualified but isn't) for an employer can be incredibly costly. The cost of a false negative is essentially zero as long as the employer can otherwise adequately fill the position.
To spell it out: if an employer gets 50 applications, has phone interviews for 8, brings in 4 for face-to-face interviews and makes two offers, one of which is accepted, the employer has gained the desired result. The fact that a qualified person was rejected along the way (false negative) is essentially irrelevant.
2. Your worth is constant.
Wrong. You said it yourself. You got several different offers. You're implying that if you had been accurately gauged market conditions would have you valued roughly similarly.
Company A might be desperate. Company B less so. You may fill a niche far better at one employer than another. One employer may simply be more cashed up and able to pay a better salary. A given employer may simply suck at negotiating or be under a misconception about market value. The list goes on.
Likewise, your desirability to the employer factors into this and it goes beyond technical skills. If the employer thinks you'll be a great fit and they'd really like to work with you, that improves your worth (to them).
3. Companies are looking for the same thing.
Clearly this is wrong but I do see this attitude come up, typically being implied by showing mismatches in offers as "proof" or similar.
There are many examples of this. For example, all other things being equal I've found that an MIT graduate is much more likely to hire other MIT graduates. The same is true for Stanford, CMU or [insert school here].
Part of this is the "social proof" element (going to a great school and/or working for a top-tier employer can be a huge advantage). But more than that it comes down to cultural similarity, common background and being a known quantity (to some degree).
This is of course different for every employer.
The real problem with interviewing, particularly at larger organizations, is that people who are bad at it are doing it. Interviewing and assessing potential colleagues is a skill and a talent. Some people have it. Some don't.
I've seen another comment here that said you need great engineers to do interviewing. I disagree. Many great engineers seem to be essentially savants who are often ill-equipped for the social discourse entailed in interviewing.
To interview an engineer I firmly believe you need to be an engineer (the same goes for managing engineers) but you don't need to be a rock star. You just have the right additional skills.
The cost of a false negative is essentially zero as long as the employer can otherwise adequately fill the position.
I couldn't quantify them exactly, but I don't think the costs are zero. For one, any candidate that you bring in and reject consumes (in the case of my company) about 4 man hours of developer time. And that's mostly senior/lead developer time. And that's not counting the time we spend discussing the candidate and the general cost of a distraction/context-switch. And then there's the opportunity cost of not having a position filled and work being started. Sometimes the short run matters.
That said, I'd definitely agree that the scale should be tipped in favor of false negatives.
It's better to think in terms of risks not just costs. Interviewers tend to higher people just like them, which means the team is often filled with people that think the same way and come up with the same types of solutions. On the other hand when you get someone in that can 'do the job' but has a radically different perspective they are more likely to bring something new to the table. To be overly simplistic in a team of world class programmers adding someone with a great UI background can be worth a lot more than yet another world class developer. The advantage being diversity does not require a larger paycheck.
Interviewing skills and engineering skills are completely different. You should be an engineer if you want to interview other engineers, but it's likely that a great engineer isn't a great interviewer unless they've been specifically trained and coached on how to interview.
Case in point - over the past week I've interviewed for four positions. Three gave me offers within a day, with very little effort on my part. On the other hand, one told me that I had little idea of what I was doing and told me to study up and apply again next year. And, that was after wasting my time with 3.5 hours of interviews. Of the offers I received, two were well over 120k plus great bonus. The other came in under 90k but told me that I could work anywhere in the world, whenever I wanted, as long as I got my work done.
Over that same time, I flat out rejected three recruiters from large engineering companies because their recruitment processes would have cost me $700-1500 just in opportunity costs. And, they weren't even willing to give me the info I needed to help me determine my chance of success or even my expected pay.
Everyone these days seems to believe that their company is so important that they deserve or need a team of top 1000 engineers. When in fact a top 20-percent team who bonds well together would likely be sufficient.
The topic of interviewing is fascinating because there seems to be very little science behind it. Who knows what works?