How to Validate a Pre-Employment Assessment

Published November 16, 2026 · 10 min read

Resumes show claims. So do assessment vendors. Almost every pre-employment assessment on the market comes with a claim attached to it: validated, fair, bias free, scientifically backed. Those words show up on a lot of websites. They rarely show up with evidence behind them.
If you are a legal, compliance, or talent acquisition leader responsible for choosing an assessment, that gap is your problem to close, not the vendor's. A test that has not been properly validated, or that produces adverse impact nobody checked for, is not just a bad hiring tool. It is a legal exposure, a candidate experience problem, and a decision you will have to defend later, possibly to a regulator.
Validating a pre-employment assessment comes down to three questions, and you should be able to get a straight answer to each one before you sign anything:
- Does the assessment actually predict job performance, or does it just produce a score?
- Does it disadvantage candidates based on protected characteristics, and has that been tested, not assumed?
- Has an independent third party checked the vendor's work, or is the only evidence the vendor's own marketing?
This guide walks through what each of those questions means in practice, what to ask for, and what good evidence looks like, including how Vervoe approaches validity and bias auditing on its own platform.
What "validity" actually means
In plain terms, validity is the evidence that a score on the assessment relates to how someone actually performs in the role. A test can be reliable (it produces consistent results) without being valid (those results mean anything about job performance). Reliability without validity is just a precise way of measuring the wrong thing.
For a pre-employment assessment, that usually means two things working together:
- Content validity: the tasks in the assessment resemble the real work of the role, not a generic puzzle or personality survey that has no obvious connection to the job.
- Criterion validity: there is actual data showing that people who score well on the assessment tend to perform well once hired, and that relationship has been checked statistically rather than assumed.
This is the core distinction between validation and inference. A resume, a self-reported skills list, or a personality quiz infers that someone might be capable. A role-based assessment, checked against real outcomes, gives you evidence that they are. When a vendor tells you their assessment is "validated," the follow-up question is simple: validated against what, measured how, and can I see it?
What "adverse impact" means, and why it is not optional to check
Adverse impact is what happens when a selection process disadvantages one group of candidates, defined by a protected characteristic such as race, sex, or age, at a meaningfully higher rate than another, even if nobody intended it. The assessment does not need to mention a protected characteristic to produce this effect. A task that happens to depend on a skill unrelated to the job, or a scoring model trained on an unrepresentative data set, can create adverse impact without anyone noticing until someone measures it.
The common statistical check is the four fifths rule: if one group's selection rate is less than 80 percent of the selection rate of the group with the highest selection rate, that is generally treated as evidence of adverse impact worth investigating further. This is standard practice in employment testing, not a Vervoe specific claim, and it is the kind of check any serious vendor should be able to describe and show evidence of running.
Here is the part that catches a lot of buyers out: a vendor can be completely sincere about wanting a fair assessment and still never have actually tested for adverse impact. Good intentions are not a substitute for a measured result.
What a bias audit adds that a validity study does not
Validity and adverse impact testing are often done mainly during development. Good practice is to keep monitoring as roles, candidate pools, and scoring change. A bias audit adds an independent, recurring check on top of that work, carried out by a party with no stake in the outcome. The point of an independent audit is that the vendor is not grading its own homework.
This matters more for an assessment that uses AI to help score responses. An AI model can be influenced by its training data in ways that are not obvious from the outside, and a vendor's internal review of its own model carries an inherent conflict of interest, however careful that review is. An external bias audit exists to catch what internal review tends to miss, and to give you something you can point to that is not simply the vendor's word. For a closer look at how bias enters AI-assisted hiring, see our guide to AI bias in hiring.
Why self-reported validity claims fall short
The current approach for most buyers is to take a vendor's word for it, because getting past the marketing page takes real effort and most procurement timelines do not build in the time for it. That is exactly backward for a decision with this much legal and reputational weight.
A few patterns are worth watching for specifically:
- Black box scoring. If a vendor cannot explain, in plain language, why a specific candidate received a specific score, in terms of the actual tasks they completed, that is a sign the score was never tied back to real evidence of skill in the first place. We cover the growing risk in explainable AI in hiring.
- Generic test banks. A test built once and reused across every role for every client is, by definition, not measuring the specific tasks of your role, which weakens any claim of job relatedness.
- Validity without audit. A vendor who talks about validity but goes quiet when asked about independent bias auditing is giving you half the picture. Validity tells you the test works on average, across the development sample. A bias audit tells you whether it continues to work fairly, in practice, across different groups of real candidates.
A practical framework for validating an assessment before you buy
Before signing with any assessment vendor, ask for documentation, not assurance, on each of the following. If you are building a wider shortlist, our guides to evaluating AI hiring vendors and the best skills assessment platforms for enterprise hiring cover the rest of the evaluation.
1. Job relatedness
Ask how the assessment content was built for the specific role, or role family, you are hiring for. A role-based assessment, built on the real tasks of the job, is inherently easier to defend than a generic test bank, because the connection between the task and the job is visible rather than inferred.
2. Validity evidence
Ask what data supports the claim that scores relate to job performance, over what sample, and how recently it was checked. Treat "we are validated" as the start of the conversation, not the end of it.
3. Adverse impact testing
Ask whether the assessment has been tested for adverse impact across protected groups, using an accepted method, and ask to see a summary of the result, not just confirmation that a test exists.
4. Independent audit
Ask who audits the assessment, how often, and whether that party is independent of the vendor. An annual cadence, from a named independent auditor, is a reasonable minimum bar for any assessment that uses AI scoring.
5. Explainability and traceability
Ask whether a candidate's score can be traced back to specific tasks and specific skills, in language a hiring manager, a candidate, or a legal reviewer could all follow. If a decision cannot hold up when someone asks why, it is not ready to be used at scale.
Here is that framework as a short checklist:
Check | What to ask for | What good evidence looks like |
|---|---|---|
Job relatedness | How the content was built for the specific role or role family you are hiring for. | Role-based tasks that match the real work of the job, not a generic test bank. |
Validity evidence | What data shows that scores relate to job performance, over what sample, and how recently it was checked. | A validity summary with the statistical relationship, the sample, and the date. |
Adverse impact testing | Whether the assessment has been tested across protected groups, using an accepted method, and a summary of the result. | A measured result, such as a four fifths analysis, not just confirmation that a test exists. |
Independent audit | Who audits the assessment, how often, and whether that party is independent of the vendor. | An annual audit by a named independent auditor, especially if AI scoring is used. |
Explainability and traceability | Whether a score can be traced back to specific tasks and skills, in language a hiring manager, a candidate, or a legal reviewer could follow. | A score explained in terms of the actual tasks completed, not a black box result. |
Why this matters more as regulation catches up with hiring technology
The regulatory landscape around automated hiring tools is moving in one direction: toward requiring the kind of documentation this framework asks for. New York City's Local Law 144 requires bias audits for certain automated employment decision tools used on New York City based candidates, along with public disclosure of the results. The European Union's AI Act treats employment related AI systems as high risk, with governance and documentation obligations attached. Other jurisdictions are actively working on similar rules.
None of this is a reason to panic, and it is not a substitute for checking the fundamentals yourself. But it does mean that "we have not been asked to prove this yet" is a shrinking excuse. The buyers asking these questions now, before a regulator asks them, are the ones who will not be scrambling for documentation later.
How Vervoe approaches this
Vervoe's assessments are built around the real tasks of the role, not a generic test bank, so the connection between what a candidate does in the assessment and what the job actually requires is visible rather than assumed. The underlying methodology has been validated using data from millions of candidates, and every score traces back to specific, role-relevant skills, so a hiring manager, a candidate, or a legal reviewer can see why a result came out the way it did.
On top of that, Vervoe's platform undergoes an annual bias audit by Holistic AI, an independent auditor. The point of that audit is the same point raised earlier in this guide: Vervoe should not be the only party checking Vervoe's work, and buyers should not have to take a vendor's internal review on faith.
This is what sits behind Vervoe's build, assess, score, and close the loop approach to assessment: role based tasks (build), realistic conditions for completing them (assess), scoring that ties back to specific skills (score), and an ongoing check on whether the result still holds up across people and over time (close the loop). None of this is presented as a guarantee that bias is eliminated or that any single hire will work out. It is presented as evidence you can ask to see, which is the standard this entire guide has been arguing for.
What to do next
If you are currently evaluating an assessment vendor, the next step is not a longer pitch deck. It is a short, specific list of documents: a validity summary, an adverse impact result, and the name and date of the most recent independent bias audit. Any vendor confident in its own evidence should be able to produce all three without friction.
If you want to see what that looks like in practice, see Vervoe's latest independent bias audit results, or book a demo.
FAQs
It means there is evidence, usually statistical, that scores on the assessment relate to how people actually perform in the job, not just that the assessment produces a consistent score. Validity is about what the score means, not how neatly it is presented.
Adverse impact is when a selection process disadvantages one protected group at a meaningfully higher rate than another. A common check is the four fifths rule, which flags adverse impact when one group's selection rate falls below 80 percent of the selection rate of the group with the highest selection rate. It should be measured directly, not assumed from good intentions.
A bias audit is an independent review of an assessment and its scoring, carried out by a party with no stake in the result. It matters most for assessments that use AI scoring, because an AI model's behavior is not always obvious from the outside, and a vendor reviewing its own model has an inherent conflict of interest.
A vendor's claim is a starting point, not proof. Ask for the underlying data, the sample it was checked against, and whether an independent party has reviewed the assessment, not just the vendor's internal team. Independent evidence is what holds up if the decision is ever challenged.
Yes. Vervoe's platform undergoes an annual bias audit by Holistic AI, an independent auditor, in addition to methodology validated using data from millions of candidates.

Vervoe Team
Writers, researchers, and practitioners from Vervoe
The Vervoe Team brings together writers, researchers, and practitioners across product, talent acquisition, and people science. Together we cover hiring, skills assessment, and how organizations build a workforce ready for what's next, drawn from work with global organizations including Lumen Technologies, Kroll, the NHL, and Australia Post. Vervoe is a skills assessment platform on a mission to make hiring about merit, not background.
Similar articles you may be interested in

Why Job Simulations Resist Cheating by Design
Anti-cheating hiring software works best when integrity is built into the assessment itself. See why job simulations outperform webcam monitoring.

How to design fair job simulations in 7 steps (2026)
A 7 step guide to designing fair, structured job simulations that assess real skills, reduce adverse impact, and stand up to bias-audit scrutiny in the US.

Psychometric Tests vs Skills Assessments: Which Actually Predicts Job Performance?
Personality testing fails to predict job success. Learn why skills assessment tests are a faster, bias-free way to prove ability and hire top talent.