As an engineer working on my company’s top of funnel it’s tough. Currently we’ve switched to a short (15-30m) technical problem that we hand grade before candidates get a call. Async technical challenges are obviously gamed but you’d be surprised at how few people both cheat + take longer than 3m to submit the solution
Are you picking a problem candidates can easily cheat on? Could Claude produce the answer in 30 seconds? If so, what's the test actually measuring?
The setups I've seen produce the strongest signal aren't the ones where the candidate can't see the answer that AI produces easily. They're the ones where there is no single answer. Architecture decisions. How you would investigate an outage. How you would tradeoff one constraint against another. The candidate can spend an hour with Claude on it beforehand and it doesn't matter, because the test isn't "did you get the answer," it's "can you talk through the reasoning" and "can I trust your judgement."
Hearing someone explain their thinking on an open problem is a much harder thing to fake. Even if they used an assistant to structure their initial pass, the live unpacking reveals where the thinking is theirs and where it isn't.
> Currently we’ve switched to a short (15-30m) technical problem that we hand grade before candidates get a call.
Funny because I do the same; I don't commit to doing any work/assessments before at least an honest interview round. Businesses have to understand that _they_ are the ones who have to solve for the filtering problem. Most great candidates that you are hoping to snipe are already in a stable job and do not _have_ to jump through hoops to get past a filter.
We’ll skip it based off other signals (referrals notably) of course, but if we offered an interview to every candidate based off their resume we’d never get around to interviewing the good candidates. I don’t like the idea of a technical challenge at the start, but as someone who was discovered way back via skills and not a fancy resume I want to continue offering others the same chance as me.
I have done programming assessments, but I've probably only done 3 in the last 20 years, and those I strictly limited to 1 hour and told the recruiters that I wouldn't spend any longer than that.
I've also flat out refused to proceed with least 5 jobs when recruiters have contacted me, tried to tempt me with a job and then sent through a 3 hour coding test. I've got better things to do with my free time than toy programming puzzles.
