How We Vet a Senior Engineer
for AI Training Work.
The quality of your training data is the quality of the people producing it. Most data vendors hire on volume and weed out by quality scoring after the fact, which means model errors are baked in long before anyone notices. We do the opposite: a deep vetting process up front, then a small bench of people we trust to ship.
This page documents that process in full so you know exactly what you're getting.
The 5-Stage Vetting Loop
Application Screen
5 minPublic profile, prior work, language and stack proficiency, geographic and availability fit. About 80% of applicants don't reach Stage 2.
Take-Home Task
60 minA representative annotation or trajectory task graded against an internal rubric. Calibrated quarterly against client gold sets. About 60% of Stage 1 passes don't reach Stage 3.
Live Calibration Call
45 minA senior calibrator works through 3 sample tasks with the candidate, watching their reasoning out loud. We're testing for judgment, not raw skill. About 50% of Stage 2 passes don't reach Stage 4.
Reference & Background Checks
Two professional references plus a background check appropriate to the engagement's security tier.
Probationary First Batch
First production batch is double-reviewed by a senior calibrator. About 15% of Stage 4 passes don't graduate to standard production.
Summary
End-to-end accept rate from application to production: typically 2–4%.
Calibration after onboarding
Weekly batch sampling against client gold sets.
Quarterly recalibration sessions with the client lead.
Drift detection on inter-reviewer agreement, flagged automatically.
Quarterly performance review per engineer with retention or off-boarding decisions.
What We Look For
Four traits that consistently predict whether someone will produce usable training data.
Six-plus years of production experience
Six or more years of production engineering experience in their primary stack.
Demonstrated verbal reasoning
Demonstrated ability to verbalize reasoning, not just produce output.
NDA and IP assignment readiness
Willingness to operate under NDA and IP assignment.
A real shipping track record
A track record of code or design that has shipped to real users.
