So @lvh and I have nailed the Latacora hiring challenge down.

At Matasano we had a PHP web app challenge and a custom protocol challenge. We had other stuff, but most of our technical qualification was done based on a web app you tested and a protocol you reversed and tested.

Latacora does both offensive and defensive stuff, and we own the whole deployment, not just the app code.

So our first cut hiring challenge — I’m really happy with it and think it’s better than what we had at Matasano — is a combination Django and AWS challenge. It’s a sort of short pentest of an AWS application environment, where we give you AWS creds and you get us a list of vulnerabilities in the app and the network.

We’re still tinkering with follow-up challenges that are less about finding flaws and more about being able to build stuff up. But the extremely nice thing about assessment challenges is that they’re super easy to score. We want to be able to make technical qualification decisions mechanically and repeatably, so that it doesn’t matter who on our staff reviews candidate challenges.

We still interview! But we start every interview knowing already that the candidate is technically qualified, and they know that too. So our interviews should be way shorter — not an all-day gauntlet — and way less stressful for candidates.