A big part of the problem that usually never gets discussed is how the lack of institutional trust between tech companies affects hiring and interviewing. There's no licensure in SWE, so in a sense your reputation is based on the companies you've previously worked for. But it seems like even if you have "name brand" corporations on your CV, other companies have no trust that your experience indicates competency. So engineers are forced to go through the leetcode grind every three years (or whenever they change jobs). Seems highly inefficient to me, especially when it's advocated by technologists who say they want to "automate all the things."
This is definitely true. But, let's be honest here: anyone who's done at least 10 or so standard, ~1-hour technical interviews has probably run into a candidate who looks great on paper, but just can't demonstrate enough basic skill to do the job.
One such candidate I interviewed seemed like they'd be really great for the role: PhD in graph theory, publications, projects listed on the résumé, couple of different programming languages (including ones we used). To me, this person's résumé screamed "solid mid-level developer." I would have probably been willing to pass them at a junior level, had they been able to perform at that level, though.
The interview itself was a pretty familiar story. For the technical portion, I introduced the problem (not a LeetCode-type problem, a more practically-oriented problem), we talked about requirements, drew some stuff on the board, and then got to coding.
I had a feeling when we were going through the requirements discussion that this might not go as smoothly as I'd hoped, but I pushed that feeling aside and did my best to let them shine.
We let people code in any reasonable programming language, but they must write actual code. They can fill in stuff like dummy helper functions, if necessary, but we want to see some kind of running, syntactically correct, and, preferably, at least lightly tested code.
They chose to code in Java, which, while not a terrible choice, seemed to me kind of like they were just handicapping themselves when stuff like (IIRC) Javascript and Ruby were listed on their résumé.
To make the long-ish story a bit shorter, we muddled through trying to implement the requirements we'd talked about earlier in Java, meanwhile the candidate was showing me a distinct lack of familiarity with basic facilities of the language, such as "what sort of methods do lists have that might be helpful here?"
Needless to say, this person did not pass my interview, and we did not end up hiring them. But, I really, really wanted them to succeed. Like I said, on paper, they look great. And, I'm sure they could have gotten through a culture fit interview just fine. I'm just not sure how well they would have done on our team, working on our rather large, pre-existing, and somewhat crufty code bases.
If you can figure out a good way to automate the task of "filter these developers down to the ones who can write some semblance of code," in a way that goes deeper than just "Write some code and run it against our automated test cases," I'd like to hear about that. And, I'm not doubting that it could be done, in theory. For instance, maybe something like the engine behind GitHub's CoPilot could provide a way to analyze and grade the candidate's code on things like style, testability, test coverage, modularity, &c.
But, AFAIK, there's nothing like that out there now, so, a structured process consisting of ~1-hour technical interview sessions, one-on-one with the candidate, attempting as best as possible to simulate the real work environment, is about the best I can think of.
My first degree was civil engineering and I took the California Engineers in Training exam back in my college days and boy was it tough even though I had stellar grades from a top university. You could bring mountains of reference books to the exam but you had no time to use them. You had to solve endless problems based on fundamental techniques you already learned during your education.
Oh, I'm not about to claim that these licensing exams people in other professions take once and then never worry about again (bar exam, medical boards, etc.) compare in difficulty to a typical SWE interview.
But, here me out here:
Suppose the CA bar exam were changed in format to be more like a SWE interview. That is, instead of being a 2 day affair consisting of 5 essays of 60 minutes apiece, a 90-minute performance test[0], and 200 multiple choice questions, let's shrink it down to a format that fits in one day and under 5 hours.
Now, let's nix the performance test right away, because it would be incredibly difficult to shoehorn in any kind of real, practical task in under 90 minutes.
According to [1], it looks like 100 multiple choice questions are allocated 3 hours worth of time. So, basically, our cut down format could be 100 multiple choice and 2-3 essays. So, that's about half the amount of testing that's done currently, more or less.
Now, if you have half the amount of testing available, you have a choice: you can cover half as many areas of law, you can cut down the depth of coverage in each area, or some combination of the two. In any case, what you've done is increase the variability in the test. In other words, passing is now more likely to be influenced by exactly which version of the test you get.
If there's an area of law you're weak in (say, bird law), it might not even be covered. Conversely, if you're a real expert in bird law, you won't get as much of a chance to shine in the new format. That means our new format both allows somewhat more marginal candidates to pass, and gives up some sensitivity in detecting people who are really, really good.
So, in summary, the new format omits any sort of practical task, increases the variability of the test, increases the probability that marginal candidates will get through, and decreases the ability to distinguish truly excellent candidates.
Seems to me that's sounding more and more like a typical day-long SWE interview, isn't it?
