All of these claims from Google that say competition performance hurts or that GPA doesn't matter are missing one huge thing: selection bias.
Google only sees the performance of the employees that it hires, not the performance of the employees that it doesn't hire. Because of this, the data they analyze is statistically biased: all data is conditioned on being employed by Google. So when Google says things like "GPA is not correlated with job performance" what you should hear is "Given that you were hired by Google, GPA is not correlated with job performance."
In general, when you have some thresholding selection, it will cause artificial negative correlations to show up. Here's a very simple example that I hope illustrates the point: Imagine a world where high school students take only two classes, English and Math, and they receive one of two grades, A or B. Now imagine a college that admits students with at least one A (AB, BA, or AA) and that rejects everyone without an A (BB). Now imagine that there is absolutely zero correlation between Math and English - performance on one is totally independent of the other. However, when the college looks at their data, they will nonetheless see a stark anticorrelation between Math and English grades (because everyone who has a B in one subject always has an A in the other subject, simply because all the BBs are missing from their dataset).
The bottom line is that whenever you have some score that is some positive combination of input variables and then you threshold your observed data on a minimum total score (as is the case in hiring at Google or in college admissions), then you will see much stronger negative correlations between your inputs than exists in real life.
And really, whenever you run some selection algorithm, you should hope that (on the margin) there are no correlations between your selection decision and your inputs. If there still is a correlation post-selection, that means your algorithm has left money on the table. So when Google says that programming competitions are negatively correlated with performance and GPA is uncorrelated with performance, what that likely means is that Google's hiring overvalues programming competitions and fairly values GPA.
In fact, if we did a randomized controlled study (the gold standard of proving causation), I think we'd see the expected results. Just imagine - if you grabbed two random programmers from the entire population, one who had won a competition and one who had not, do you really think the competition winner would be the inferior programmer?
Edit: Many other posts here are coming up with plausible sounding narratives to fit the data. "Competition winners are one-trick ponies or loners or write awful code." I encourage everyone to think critically about the data analysis itself.
Great post. You are exactly right - all this study uncovers is that Google put too much bias in favour of programming competition wins in their hiring process in the past. What would be nice to know is exactly how important this factor was.
As I mentioned in another post, I don't think it's that Google sees programming competition success and says "oh boy, we gotta get this one!", but that the experience of those competitions helps the candidates perform better in interviews.
It is very hard to account for that in interviewing. Probably the only effective method would be to ding people with competition experience. Of course, that would be obvious conscious bias and would never happen.
I don't think programming competition wins (PWG) are a negative when selecting candidates out of the potential pool, but Google must have been over-weighting the value of PWG when selecting their candidates. Basically the candidates they hired with PWG must have been weaker overall than the other people they hired. To put it another way PWG might explain 5% of the variations between hires, but if it was given a weighting of 20% then we would see the negative correlation observed.
I also don't think that winning competitions is a negative. I only believe that the experience doing so gives one a bigger advantage in interviewing that in actual work performance.
I also don't believe that the Google hiring committees treat competition winners specially (though of course I could be wrong there). The reason I say that is as a frequent Google interviewer, the interview scores vastly outweigh the value of the resume (to the point of it becoming ignorable). And, I personally don't look at a resume except to see what language a candidate likes to write in, and to see how long they've been out of school (depending on how long, I might ask more a designy question).
Given my anecdotal and inferred evidence, I believe the only way for a competition win to help someone get the job is to make them better at interviews.
Given the stated evidence in the article, combined with my immediately-above belief, the explanation that fits is that competitions help more with interviewing than with job performance.
Since it helps more with selection (passing the interview) than with performance (job review), there is a natural negative correlation with performance for those who were selected.
This is completely consistent with competitions helping people do better with job performance.
All of these claims from Google that say competition performance hurts or that GPA doesn't matter are missing one huge thing: selection bias.
Google only sees the performance of the employees that it hires, not the performance of the employees that it doesn't hire. Because of this, the data they analyze is statistically biased: all data is conditioned on being employed by Google. So when Google says things like "GPA is not correlated with job performance" what you should hear is "Given that you were hired by Google, GPA is not correlated with job performance."
In general, when you have some thresholding selection, it will cause artificial negative correlations to show up. Here's a very simple example that I hope illustrates the point: Imagine a world where high school students take only two classes, English and Math, and they receive one of two grades, A or B. Now imagine a college that admits students with at least one A (AB, BA, or AA) and that rejects everyone without an A (BB). Now imagine that there is absolutely zero correlation between Math and English - performance on one is totally independent of the other. However, when the college looks at their data, they will nonetheless see a stark anticorrelation between Math and English grades (because everyone who has a B in one subject always has an A in the other subject, simply because all the BBs are missing from their dataset).
The bottom line is that whenever you have some score that is some positive combination of input variables and then you threshold your observed data on a minimum total score (as is the case in hiring at Google or in college admissions), then you will see much stronger negative correlations between your inputs than exists in real life.
And really, whenever you run some selection algorithm, you should hope that (on the margin) there are no correlations between your selection decision and your inputs. If there still is a correlation post-selection, that means your algorithm has left money on the table. So when Google says that programming competitions are negatively correlated with performance and GPA is uncorrelated with performance, what that likely means is that Google's hiring overvalues programming competitions and fairly values GPA.
In fact, if we did a randomized controlled study (the gold standard of proving causation), I think we'd see the expected results. Just imagine - if you grabbed two random programmers from the entire population, one who had won a competition and one who had not, do you really think the competition winner would be the inferior programmer?
Edit: Many other posts here are coming up with plausible sounding narratives to fit the data. "Competition winners are one-trick ponies or loners or write awful code." I encourage everyone to think critically about the data analysis itself.
Edit2: From Gwern's comment, this phenomenon is apparently called Berkson's paradox: http://en.wikipedia.org/wiki/Berkson%27s_paradox