When it comes to take-home assessments, the biggest hurdle is managing cheaters. Offering an assessment that candidates can complete in their own time does bring a lot of benefits. For example, it tends to reduce interview stress, simulates a more realistic work scenario, offers flexibility in timing, and can even demonstrate a candidate's ability to work independently. But, there's a downside to this. The ability to ensure the work is entirely the candidate's own becomes a bit tricky. They could get help from someone else, find the answers online, or even use AI solutions like ChatGPT. So, while the benefits are substantial, the challenge of potential cheating is a significant concern to address.

As such, all online assessment platforms have different approaches to dealing with cheating. Here are a couple of the ways it is done.

1. Proctoring Techniques

Some companies use proctoring techniques to keep a close eye on what candidates do. These are typical strategies for tools like CodeSignal and HackerRank. There are a few ways they do this:

Webcam + Video

You can ask candidates to turn on their webcam, check their ID is really them, and then watch them while they are working. The problem with this is that it can make the candidate feel uncomfortable and increase their stress. Plus, this method doesn't stop candidates from using a webcam and a monitor to make it look like they are working on the challenge.

Watching You Code

Companies like CodeSignal and HackerRank often use an online IDE to watch what keys you're pressing and if you're copying and pasting anything in or out of the test.

The thing is, this kind of beats the point of having a take-home assessment. It forces candidates to use a tool they may not be used to, stops them from using other resources, and has them feeling like they're being watched. From what we've seen, this makes candidates feel just as stressed as they would in a live interview. At least in an online interview, you can chat with someone and get help if needed.

2. Making better questions

Another way to handle cheating is to make questions that are tough to cheat on. The idea is that instead of trying to watch every move a person makes, we make it so they can't find the answers easily and using Google or AI tools in the assessment is not a problem. You can read about some ways to make cheat-proof questions in this blog post. However, platforms like CodeSignal and HackerRank struggle with creating cheat-proof questions as outlined in this article.

Some platforms even have ways to check if a question has been leaked on the internet. If it has, they let the company know so they can change the question. HackerRank does a good job at checking of the question is leaked.

3. Looking for Cheaters

Instead of trying to restrict cheating activity, another alternative is to detect if someone cheated. This can be achieved by comparing a candidate's solution to other submitted solutions or existing solutions available online. This method has proven to be quite effective in spotting instances of cheating.

Tools like HackerRank and CodeSignal are great at this method of detection. However, often times they flag false positives because of the nature of their questions. For example, some LeetCode style questions have very only one main and common way to solve them, so solutions can look similar even if the candidate came up with it themselves. For this reason, this technique is only useful when you create custom and unique questions that have multiple possible solutions.

4. Follow-up Discussions

Last but not least, some companies make their questions in a way that leads to more discussions in later parts of the interview. After passing the take-home assessment, candidates can be invited to an interview to talk about what they did. This can give a lot of insight into what they were thinking when they made certain choices. If they struggle to explain or if they seem too prepared, it might mean they cheated. This method is one of the best because no matter what, you can never be sure if a person cheated on a take-home assessment, since there's always a way around being watched.

What We Do at Hatchways

Here at Hatchways, we believe strongly in avoiding techniques that increase the anxiety of the interview process. For this reason, we avoid strict proctoring techniques like watching candidates code or recording their screens.

Instead we focus on the following strategies:

  • Better Questions: Helping companies create questions that are hard to cheat on. We also using scripts to check the internet for leaked versions of our questions. We promptly alter our questions or notify our partners when a leak is detected, ensuring assessment integrity.
  • Similarity Check: We automatically scan for identical assessments to detect cheating. Our diverse evaluation components, including written elements, make this method particularly effective.
  • Leak Prevention: Leaking a Hatchways question is significantly harder than a LeetCode-style question due to the need to share an entire private codebase and a non-copyable question prompt. As a result, our questions are leaked 5-6 times less frequently than our competitors.
  • Measuring Soft Skills: By incorporating written or soft skill questions into our assessments (optional), we further enhance cheat detection. These questions require personalized and thoughtful responses, which are more challenging to plagiarize and make for an easy way to detect if there has been plagiarism.
  • Designing assessments for follow-up discussions: We make our assessments in a way that leads to good discussions later in the interview process. We believe in turning an online test into a conversation down the line.

If you want to try the candidate experience of a Hatchways assessment, you can do so here!

‍