Take home tests are the worst. Company says take home test will take 3 hours to complete. They never do. Schedule 2x or 3x the estimate. Especially if you want to impress the reviewer.
You send it over, then the company says no or yes, only to move to new stage.
In the worst case you ruined your weekend and received a no. But the company just took 10 minutes to arbitrarily reject your application.
From the company's perspective:
I've seen applicants receive friends'/roommates'/spouse's help on take home tests. Not a good indicator at all even with a glowing submission.
After doing a handful of these and rejecting several handful of these tests I'd like to add a little to you comment.
I agree that the time it takes is always muchuch longer than what they state.
Companies that offer these tests before doing an initial phone screen get that email deleted. Why would I as an applicant who is applying to 10+ jobs spend time doing this test when I have never even had a chance to interact with a human.
The tests are sometimes not even close to the actual job. As in the job description is for a front end developer and JavaScript knowledge needed and they ask you to write the test using a completely different language. one company (who adversities jobs on here all the time) asked to do some php command line scripting for a JavaScript front end position. How is that in anyway a useful judgement of someone's skills. So wasting people's time is a big deliminator.
An example of a company I experienced that did the take home 'right' did an initial phone screen a couple days after applying. Then did another tech screen which was just basic stuff. After that they asked me to do a take home exercise and while completing it they continued to move forward with the application process including setting up travel arrangements. The take home test was directly related to the job and was given open ended for some creativity if one chose. The onsite final interview was discussing the code, so it would do little good to cheat on it because you need to be able to talk through it.
I didn't even get the job with them but it was actually not a painful experience for once to do a take home test.
Just my two cents, but I believe there is a good way and a terrible way to do it.
The best take-home interview I ever did had a time-limit. And of course, I panicked when getting close to that limit, and tried my best to tidy up my solution but didn't really manage. Then a pop-up appeared asking "Would you like another 5 minutes?" That was just enough time to calm down, tidy up sufficiently, and submit clean code.
The only time I've been given one there was a time limit as it was fairly strict. Frankly I'd have preferred an untimed version but I can see why people would disagree with that.
Yeah, I can see why too, since almost all software is written with a literal ticking time bomb ready to go off if you don't finish it within a certain minute.
This is, of course, sarcasm. Timed programming interview tests are fucking dumb.
I mean, if an employer gives someone a take home test that they expect to take 3 hours and they give them a week to do it, that's reasonable if technically "timed"; but if an employer gives someone a take home test that they expect to take 3 hours and they actually give them 3 hours to do it, that employer sucks at hiring programmers. I won't even bother debating this with the inevitable person who chimes in to rationalize why they do this, because as far as I'm concerned it is a "the sky is blue" statement and any time debating it with people who disagree with this position is completely wasted.
One thing you're missing though is that a time limit allows for a controlled variable when you're comparing candidates.
The test I was referring to I'm almost certain was not intended to be completed in the time allowed. I've used it myself for in person interviews (not take home) and give candidates 3x the time I was given, few of them actually finish it completely. While it just made me feel like an abject failure at the time now I realize a big part of what was being tested was given limited time what bits I focused on. In fact that's exactly what I tell candidates who I give it to.
I agree. I've done really good ones and really bad ones. A good test started in the office with the hiring manager. Really them just watching me write a simple CRUD app in asp.net when it was all the rage. This was for an entry level gig and was actually a really good test. At the end they had an additional feature for you to add from home. It took a couple hrs and ended up being a great test to find decent entry level ppl.
The bad one was about 2 yrs ago. I walked into a conference room for the final round, was sat down at a mac, and asked to write some code in their proprietary DSL without any documentation or anything. It was maddening. I almost walked out, but the salary was stupid high. Didn't get the job. Thinking back on it, maybe it was just a test and they did want me to say it was ridiculous.
I almost walked out, but the salary was stupid high. Didn't
get the job. Thinking back on it, maybe it was just a test
and they did want me to say it was ridiculous.
Thinking about it, I see two realistic possibilities:
- They are being unintentionally stupid by asking you to do this activity.
- If they are intentionally asking you to do a stupid thing and they expect you to revolt in the interview and decry the madness. How does that make you feel about your future day-to-day life at that organization?
In both cases those are not good "A-player" types of people to work for.
No need to torture yourself over more favorable prospects "lost" :)
> Thinking back on it, maybe it was just a test and they did want me to say it was ridiculous.
If that was the case ... then you don't want to work there. I've worked with a few business people that intentionally filtered interviewees by strange characterists. [Aka... graphic designers that were juinor, but didn't have web dev experience (I want to say it was something stranger than that)]
We ask for either work samples or offer the ability to take a take home test. We prefer the former but understand that's not always feasible. If they choose to do the latter we make it clear that it's designed for ~1hr but they can feel free to spend as much or little time as they want. It's not difficult and is really a glorified fizzbuzz, really we're just trying to ascertain if this person can code at all. This is after HR does an initial screen but before one gets into the real engineering group.
I'm a big fan of the work sample method. I have one repo that is polished out and is a good representation of what I am capable of. Most of my code online isn't that nice because I write stuff for fun so making sure I have proper accessibility tags isn't that important.
Now you could debate which is a better representation of my work, the polished dog and pony show repo or the fast and dirty stuff that I push in my ongoing projects.
One take-home test I declined was for a advertised web job that turned out to require lots of PHP knowledge and plugin editing experience.
They seemed offended when I was frank that it would take much longer than the 1 hour they were quoting and that it wouldn't be a real demonstration of skills but more so what I could cram.
OTOH, my current work does a sort of take-home test but it can't be faked/cheated on because you have to record yourself teaching something.
That topic ("People do stuff at home for an interview") is a topic that comes up again and again. My last reply [1] is about a month old and feels still valid.
Don't state that 'take home tests are the worst'. That is - failing to find better words - crap. If that is the ONLY option, I understand that this might be not for you. But - that's not the case here as far as I can tell. You, as a person interested to interview, can opt in. That is awesome.
Now - you might not be the type of guy that would _want_ to opt in, but please refrain from these absolute statements. No, that's not the worst. In fact, it's probably the _best_ option for a number of people (I myself would - if I'd want to interview with this service - opt for the home project).
I fail to understand how this 'bash the home work' attitude comes up again and again. Yes, don't work for free. But if you're doing a 3h whiteboard marathon or work from your own chair? And you pick which one you prefer? I don't get the hate here..
Unfortunately, most companies aren't comfortable replacing an onsite with a work sample (or take home.) So, it's just additive.
It's not "the worst" in relation to other interview options. It's the worst because it's usually in addition to other interview options. I just stopped doing take homes on my last round of interviewing. Just not worth it. It was always just added work. It never replaced a stage of the process.
If you think about it, it makes sense though. Very few companies would hire people directly based on the strength of their github account or their topcoder rank. So, if they won't do that, then what extra information does a take home really provide?
Companies seem to recognize that they want to hire people who do good work and that good work isn't done in an interview setting. But very few companies are willing to just analyze the candidate's work. They want to subjectively judge the person.
Agreed (and see the linked comment please): If it's not a replacement for whiteboard bullshit, then GTFO. That's insane and maybe 'the worst'.
But it's important to remember that a take home exercise cannot fully replace an interview. It can replace the coding part, the whiteboard "and now we ask useless trivia questions" part. But there's no way for this work to replace the "would you fit the team" interview.
Others already discuss this in different subthreads here, but basically I'd expect the company to clearly show how their process works and - ideally - filter by ~social~ criterias first ("You might fit the team, if you can code"). Doing work for free with potentially no feedback or a 'fail' in a later discussion is crap, of course.
There is a reason that nobody filters by social first.
One of the biggest challenges in hiring is that it takes a ton of time to go through many bad candidates before you see any good ones. And time is the one thing you don't have. You're hiring because you don't have enough resources to do the work you already have.
Therefore the name of the game is efficiently rejecting candidates while using up the least amount of your existing people's time. Which means that the most expensive filters should be done last. And the most expensive one is social because judging it takes time from EVERYONE.
And that seems fair.
Why is it only the employee that is expected to take up time? At the end of the day there is likely to be more than one applicant, so your spending 3 hours of your time for 50% chance at best.
The fundamental reason why this is OK is that the company is hoping to pay hundreds of thousands of dollars for the candidate's time. The person who pays the piper, gets to choose how it happens.
But that said, it is still fair. The effort required to hire a new person into a competent company is significantly greater than the effort it takes a competent person to get a new job. As a candidate it doesn't look like this because you see that you personally gave the company more attention than vice versa. However you don't see all of the other people who the company also paid attention to and ultimately rejected.
If you're a competent developer it probably still doesn't look like this because the company usually develops procedures that concentrate the required effort in the hiring manager and/or HR. Therefore you have little idea how much effort is actually spent looking for candidates.
But spend time as a hiring manager and it will be obvious.
its bad for the company because the candidate can work with someone else and produce a glowing submission. I have helped a number of people do these some that have gotten the job.
its bad for the interviewee. They may spend 10-15 hours on this and get rejected for no reason at all. Do you think the person reviewing the submission is putting multiple hours into it. Doubtful.
If the intent is truly(and I mean truly) help individuals that struggle with traditional interviewing techniques then kudos. Demonstrate this by allowing candidates to interview in a way that is comfortable for them(including not taking your take home test) If its the company saying "my time is more valuable than yours. Do this assignment then we'll talk" then no thanks.
If you were able to explain your 'cheated' solution to someone and they could reliably cheat the interviewer, I'd say you .. taught them. GJ!
The interviewee (the vowel count makes me dizzy) can choose what he/she prefers. So that argument is weird. Maybe (if that's your point) it might be beneficial to go for the face to face interview, for direct feedback? But .. some people just _cannot_ do that. You're not talking "Ah, they didn't respond. If you'd just had visited them on-site..." here. It's "opt for the home project or don't apply". Which is empowering the candidate.
I think you are missing the point. If you are ok with a "Cheated" response then you are really hiring a candidate based on their network of experts. An equally or more qualified candidate could submit a less stellar answer only because their dad isn't an emeritus chair at a University.
Or, if they don't actually expect you to spend more than 3 hours on the task, they will compare you to people who have spent 9 hours on it.
Had that recently: they said "spend no more than two hours on it" so I finished in about an hour and 45 that evening and flipped it over.
Then they started asking why my 20 or so unit tests only covered the basics when other candidates had full unit tests in the two hour time frame. I told them the other candidates were simply lying :-)
Very true - it is part of the bane of take home assignments. Even for someone who can code extremely fast, a lot of the exams aren't realistic with time, and then they are compared with those done by people who put in multiples of the asked hours.
I rather put in those extra hours contracting (worst scenario), doing open source work (lots to be done that advances the productivity of tens of thousands of developers, if not more), into my hobbies, or into my friends.
I would rather spend 8 hours at home giving the interviewer a realistic impression of my abilities than 1 hour mumbling over a whiteboard and getting trashcanned because the interviewer mistakenly thinks that whiteboard skills prove anything at all.
As for receiving outside help, if you're judging applicants solely on their homework results, you are Doing It Wrong. The best interview I've ever had gave me a take-home programming assignment by email, then when I came in the lead programmer asked me to explain the program line-by-line and justify various decisions I made. I got the job.
Speaking as an employer: we address this by keeping our coding exercise short, and specifying a time limit. The limit ("60-90 minutes") is not rigidly enforced, but candidates know we're able to see how long they spent, and in practice almost everyone spends more than 60 and less than 90 minutes.
It's tricky. An in-person test would put the candidate in an unfamiliar environment, and makes many people nervous. A take-home test without a time limit opens itself up to the "I'd better spend lots of extra time so I can look impressive" problem. A take-home test with a hard time limit can also make people nervous. This is the best compromise we've been able to come up with (suggestions for improvement welcomed!).
As suggested on this thread, we also don't ask candidates to tackle the exercise until they've had a chance to talk to us on the phone.
This is very much true. We do take-home interview questions where they bring the output to the interview and we talk through their solution in person. We'll ask how long they took as a way to normalize expectations and also adjust the interview question to take more/less time in future instances.
This does two things. In the long run, the time required converges towards where we want it to be (ASSUMING honest answers), and by bringing you in to talk through things, we can easily tell if these thoughts were your own by challenging you on specific parts of the prompt.
This is less than ideal since it costs the interviewee time, and time isn't cheap, but we've found that advance prep removes an even bigger wildcard in interviews: how you respond to interview stress.
Tip: If you want honest answers about how long it took, ask them after they've been hired and working there for a few weeks. Otherwise they will absolutely lie.
"That? Oh, it was no big, probably 20-30 minutes."
Honestly I kind of enjoy take home tests as long as they align with what I'm learning and looking to learn. I've had to do a couple take home interviews as well at this point as part of the initial phone screen, although they revolve more around config management tools/Ruby DSL and buildout work so far. So far the tests I've taken have been relevant to what I'm hoping to move into ("DevOps"/automation) so they've been a pleasure to complete. Not only that, but also a base to build on for playing with additional tools in the chain.
The biggest problem with these though is definitely the time crunch. The first one I hadn't realized how long it would take so rushed through it at the end and made mistakes. Second one gave myself a full week rather than 4 days and that's proceeding to an in-person interview, so you need to be taking as much time as they'll allow in order for it to go smoothly.
Also, another project means more documentation added to the repo so that's pretty nice too. If the project didn't align to my interests, which thankfully happen to be stupidly in-demand right now if you have "senior" experience with them, and the position, I'd nope out of the project right away.
> I've seen applicants receive friends'/roommates'/spouse's help on take home tests. Not a good indicator at all even with a glowing submission.
That's why they use the homework as the basis for the interview. In my opinion, of all things they could test you on during the interview, your own work is potentially one of the more pleasant subjects.
I've had a few take home projects that weren't so bad.
But the last one I did...
A junior just basically shat all over the project and claimed a lot of things.
[The submission was to write a few sample sort functions for a library... the recruiter asked for an app, the paper asked for a library.. I did both... what did I get shat upon for? The user interface/cli that wasn't required.. another thing.. Why did I have 66 commits?]
This is a completely generic response which is not at all tailored to the above poster's specific situation (since I'm not familiar with the details), but the general idea would be something like this:
If you asked someone to do FizzBuzz at home and they sent you a git repo with 60+ commits, you wouldn't be a little puzzled about that? That would seem to suggest they really struggled with it.
I'm not really sure what you mean. We don't know what the task was that he had to solve. I was providing an example of a situation in which 66 commits could be seen as a bad thing.
For a typical whiteboard coding problem, I also wouldn't want to see 60+ commits.
I remember when I first got started with git I committed at an absurd frequency. I don't think it's that unreasonable for someone straight out of college to be relatively new to git, and thus do the same thing.
I remember later on I wrote a script that listened to git hooks and rebuilt my project on a remote server. I was still testing manually at that time, as we all do in the beginning, which resulted in a large number of commits so that I could view the results on the server.
I think it's ok to ask "why do you have so many commits?" but not "why do you have so many commits?!?!?!". It's also not ok to ASSUME that a large number of commits is a bad thing automatically, unless you have reasons far better then any submitted in this thread.
Well I can't say too much without exposing the problem.
But there were about 16 files in total. All of the comparators that were written had multiple tests. (I shot for nearly 100% coverage..although I wasn't going to force stdin emulation, and mock out system.exits) All of the commits that were performed were done in small amounts. (Also, a few were just for transferring work space to other computers) The task in hand would have represented approximately 4 tickets at the bare minimum.
Besides, the number of commits: That problem is resolved by merging a branch back down.
The feedback was written in a way "How do you consider that reasonable in a professional environment?"
(Another thing... I wasn't required to submit via a git repo I was required to submit via a zip file, but I did because I hoped it'd give me a leg up.... Turns out: You're better off not trying)
66 commits sounds perfectly fine for complete unit test coverage, 16 files, etc. Sounds like this was not the sort of person anybody would want to work with if he phrased the question in that condescending way. He'd probably end up being an insane micromanager.
This might even be an underestimate. In my experience, there is a lot of research time that goes into the problem before you really dig in and start coding. I had one take-home project that required a few days just to get my system configured to begin testing code (collecting the dataset [10's of GB], installing libraries, and configuring the system).
>I've seen applicants receive friends'/roommates'/spouse's help on take home tests.
I enjoy overtly mathematical problems, so I've talked through a number of take-home interview questions with friends during the research phase, and even provided implementations for comparison and review after they've returned them. (Most recently on a take-home challenge to generate digits of pi.)
>But the company just took 10 minutes to arbitrarily reject your application.
This is actually my second biggest gripe as an applicant. I spend a couple of hours building and submitting the most compelling application I can for a job. The worst so far was an automated email response that my application had been forwarded for review, and before I finished reading the automated response I got a rejection email from the hiring manager. The emails are literally two minutes apart in my inbox. It is incredibly frustrating to put so much time into applications when they are clearly being summarily rejected.
(For anyone curious, my biggest job search gripe is not receiving any kind of firm decision...ever. I can appreciate that there "is not a good fit at this time", but I'm not going to be sitting here in six months pining for that job I applied for with your company. If I want to show interest, I'll apply again in a year or so. It's actually mildly frustrating when I get a callback three months later.)
I've gotten some pretty wacky projects. Basically, implement a fairly-complex web app. Something that I'd assume (even after doing similar tasks) would probably take me up to 40 hours of work, just because the scope is so crazy. They want user login, multiple features, etc... basically an MVP for a small product.
It's hard. If you submit part of the project, which you usually have to do unless you're unemployed, they never go for it. If you call out of work a day or two to do the project... and you don't get the job anyway...
I recently interviewed for 4 jobs, one of them was one of these type of "you must implement this in 1 hour". It was painful. I didn't finish in time. Apparently the idea was you much choose language X, it will e pretty easy with language X. I chose Y, and they said why did you choose Y, that was stupid. Result - I chose one of my other 3 offers, took the one that made me a staff programmer. And I live in Seattle, ha ha bay area.
My wife is not a technologist. In fact, she's a biologist & nurse who's worked in pharmacoviligance for a series of pharmas & CROs past ~15 years. She was laid off this spring after her company was purchased and went through the whole interview thing. One company had a writing assignment as part of their process. It was after both the phone screening and the in person interviews, and they only gave it to candidates they really liked/wanted, AND it was literally a mini version of the kind of data analysis and reports[1] she'd be doing in the job.
But ... it was something that took about 40 hours to complete, and ruined about a week and a half of family time. Thankfully she got the job, but if she hadn't it wouldn't have been pretty.
I've had an opposite experience, but with just one company: CloudMine. They gave me a personal and technical interview, then a project to add a feature to their existing codebase. They paid me $300 for my work. I was way over time estimates and I didn't get the job, but I will always respect them for that.
The company I work for has a "coding assignment," and it works really well for us. I'm not sure we give a time estimate; I think we just ask for it back in a week or so. The devs reviewing the code don't even know how long it took for the applicant to return, and it isn't a criteria we use to judge them. In any case, I can't imagine it would take anybody more than a few hours.
IMO, it's a less insulting, slightly more realistic version of FizzBuzz.
As far as getting help goes, I'm not sure it matters too much. There's still an on site interview, and we ask a few questions about the assignment, along with some other coding questions, and regular interview stuff. I guess if their friend/roommate/spouse is going to help them code all the time, then it's like we hired two people for the price of one ;-)
We try hard to provide good feedback on each submission (certainly not just a 10 minutes look, even for a truly awful entry).
I think that we should try harder to provide an exercise with a maximum time spent. We also try hard to overlook things that can be attributed to a lack of time but the openness of the exercises we use mean that somebody willing to could spend dozens of hours on it if he/she wanted a truly perfect solution (with unit tests everywhere, handling all API levels & devices in the wild, ...)
I personally thinks that our exercises are more objective than whiteboard tests & better reflect real life work. Actually, we take our inspiration from real problems we had to solve.
If you only supposed to take three hours why not set it up like a take home test? Set up a site where you can download it and then you have three hours to upload the answer?
> Take home tests are the worst. Company says take home test will take 3 hours to complete. They never do. Schedule 2x or 3x the estimate. Especially if you want to impress the reviewer.
My co-worker once found that someone had posted the take home problem on one of those freelance job sites. Needless to say we didn't move forward with the candidate.
I'd even say: if they can get the problem solved for less than their salary and in reasonable time, hire them anyway - you're hiring them to solve programming problems, not to type them.
No, in the worst case, you submit it and the company doesn't even take a look at it. That's not only the worst case, it seems to be the most common as well.
Take home tests are the worst. Company says take home test will take 3 hours to complete. They never do. Schedule 2x or 3x the estimate. Especially if you want to impress the reviewer.
You send it over, then the company says no or yes, only to move to new stage.
In the worst case you ruined your weekend and received a no. But the company just took 10 minutes to arbitrarily reject your application.
From the company's perspective:
I've seen applicants receive friends'/roommates'/spouse's help on take home tests. Not a good indicator at all even with a glowing submission.