When to trust AI scoring: a calibration guide for HR

60-Second Summary

Treat AI scores like you treat psychometric assessments — useful inputs, not verdicts. Always check validity, reliability, and adverse impact before trusting any scoring output.
Five questions to ask every AI scoring vendor: (1) what does it predict, (2) on what data was it trained, (3) what's the demonstrated error rate, (4) what's the disparate-impact ratio, (5) what audit log will you get?
Acceptable bands depend on consequence. Low-stakes ranking (sourcing): 70%+ accuracy fine. High-stakes (hiring, performance, comp): require 85%+ AND a four-fifths rule pass AND human override always available.
A score with no confidence interval is not a score. If the vendor can't tell you the uncertainty around a number, the number is decorative.

AI scoring shows up everywhere in modern HR — resume-to-JD match scores, assessment grading, performance prediction, attrition risk, engagement health. The output looks definitive: a number between 0 and 100, often color-coded. But underneath, the same tool can be highly accurate, badly calibrated, or actively harmful — and the score itself looks identical in all three cases. This guide is how to tell the difference.

The trust question

Every AI score is a probability dressed up as a verdict. 'Candidate A: 87 match' is shorthand for 'our model estimates that this candidate satisfies our criteria with some level of confidence.' That confidence is the part vendors hide and HR teams need to surface. A 87 from a well-validated model on representative data is informative. A 87 from a model trained on the prior 5 years of your hires — when 91% of your hires were white men under 35 — is propagating bias as math.

Five questions to ask every vendor

Five questions before you trust any AI score

1
1. What exactly does the score predict?
Not 'good fit' — that's vendor-speak. Ask: what was the outcome variable during training? Was it 'recruiter passed to next round' (recruiter judgment, not quality), 'received an offer' (same), 'passed first-year performance review' (better but biased toward continued employment), 'still employed at 3 years' (good but slow signal)?
2
2. On what data was it trained?
Industry, geography, role family, time period, sample size. A model trained on US software engineers from 2018–2023 is not appropriate for Indian customer support roles in 2026. Ask for the training data demographics. If they won't share, that's a red flag.
3
3. What's the demonstrated error rate?
On a held-out test set, what % of the time did the model agree with the ground-truth outcome? Beware vendors who answer in F1 score or AUC without explaining — ask for accuracy at the threshold you'd actually use.
4
4. What's the disparate-impact ratio?
For protected groups (gender, race where collectible, age), what is the pass-rate ratio of the lowest group to the highest? If below 0.80 (four-fifths rule), the tool will trigger US disparate-impact analysis. Vendors should publish this; many don't unless asked.
5
5. What audit log will you get?
For every score, can you retrieve: input features used, model version, confidence interval, comparable scores from peers, and an explanation of the top 3 factors driving the score? If yes, you can defend it. If no, you can't.

Validity, reliability, fairness

These three concepts come from psychometrics — the science behind all employment assessment — and they apply unchanged to AI scoring tools. A trustworthy tool clears all three.

Validity: does it measure what it claims to measure?

Content validity: do the features the model uses relate logically to the outcome? (A model that uses 'attended an Ivy League school' to predict engineering performance has weak content validity.)
Criterion validity: does the score correlate with actual outcomes? Vendors should provide correlation coefficients (r) — anything below r = 0.20 is weak; r = 0.30+ is professional-grade.
Construct validity: does the score generalize to the underlying ability, not just the test? (A coding-test score that predicts coding tests but not job performance has poor construct validity.)

Reliability: does it give the same answer twice?

Test-retest reliability: if the same input is scored twice, do you get the same number? AI tools using non-zero temperature parameters won't — this is a problem for consequential scoring.
Inter-rater reliability: do different graders (or different model versions) agree? Vendors should report this; a model that disagrees with itself across versions can't be calibrated against.
Consistency over time: does the model's scoring drift? Ask for the vendor's drift-monitoring practice.

Fairness: does it produce similar results for similar people across groups?

Demographic parity: do groups pass at similar rates? (Four-fifths rule baseline.)
Equal opportunity: among qualified candidates, are pass rates similar across groups?
Calibration: among people who score X, is the actual outcome rate similar across groups? (A 70% score should mean the same thing for everyone.)

Acceptable error bands by stakes

Use case	Stakes	Minimum accuracy	Disparate impact	Override required
Sourcing prioritization	Low	65%	0.85+	No (recruiter reviews all)
Resume parsing	Low	90% on fields	N/A (data extraction)	Yes (spot-check)
JD-match score for screening	Medium	75%	0.80+	Yes (recruiter reviews all)
Coding/skill assessment grading	Medium	85%	0.80+	Yes (sample-checked)
Interview transcript summarization	Medium	Human-evaluable quality	N/A (content)	Yes (interviewer review)
Performance prediction	High	85%	0.85+	Yes — never automated
Comp recommendation	High	Explainable, range-based	Pay-equity pass	Yes — human decides
Termination / RIF selection	Very high	Don't use AI as primary input	N/A	Always human-led

Do the math yourself

Run the disparate-impact check on your own data quarterly. This is 30 minutes in a spreadsheet, not a project.

The 30-minute disparate-impact check

1
1. Pull the data
All AI scores from the last quarter, with the demographic categories you collect (gender at minimum; ethnicity if collected; age band).
2
2. Apply your threshold
If you use the score to advance candidates above 70, mark each candidate pass/fail at that threshold.
3
3. Calculate pass rates per group
For each protected group, divide passes by applicants. You get pass rates like 38% / 41% / 29%.
4
4. Compute the four-fifths ratio
Lowest pass rate ÷ highest pass rate. Below 0.80 = disparate impact, requires action.
5
5. If failed, diagnose
Common causes: feature in the model correlates with protected status (university, name patterns, gaps in employment, zip code as proxy for race).
6
6. Action
Adjust threshold, change features (vendor), or shift weight away from the score in your decision. Document the analysis and the action.

Red flags

Vendor refuses to disclose the training-data demographics.
Vendor uses 'culture fit,' 'leadership potential,' or other subjective constructs as the prediction target.
Vendor scores video or audio for personality traits, soft skills, or emotion.
Vendor markets 'fully automated' decisions for hiring, performance, or comp.
No audit log per decision.
No published bias-audit results.
Customer-success team can't explain how the score is computed at a feature level.
Pricing tied to number of candidates rejected (creates perverse incentives).
The same input gives different scores on different days without an explainable version change.

FAQ

Frequently asked questions

Are AI scores better than human judgment?

Sometimes, in specific domains. Structured AI scoring of coding tests can be more consistent than human grading. AI summary of a stack of resumes is faster and surfaces patterns. AI scoring of interview videos for personality is uniformly worse than no scoring at all. The right question isn't 'is AI better' but 'is THIS AI tool better than the human process it replaces, on the specific outcome we care about, for the specific population we're scoring.'

What if the vendor says their model is proprietary and they can't share details?

Decline to buy. In 2026, every credible vendor of AI scoring for HR can provide: training data composition, accuracy on a held-out set, bias audit results, and an explanation of features per score. If they can't, either the product isn't mature enough to be trusted with consequential decisions, or they're hiding something.

How often should we re-audit?

Quarterly for active tools. Annually for low-use tools. After any major vendor model update. Whenever an incident occurs. Many jurisdictions (NYC LL 144, EU AI Act) require annual audits as a legal minimum.

What if our bias audit shows a problem but the tool is the best on the market?

Three options: (1) raise the threshold so the disparate impact disappears, accepting smaller candidate pool, (2) weight the AI score less in your decision (e.g., from 50% of decision to 20%), (3) replace the tool. Doing nothing is not an option — you've now documented knowledge of the impact, which dramatically increases legal exposure.

Can we just turn off the AI on candidates from groups where it underperforms?

No. Differential treatment by protected status is itself discrimination. The fix is to improve the tool or use it consistently with a lower weight, not to apply it selectively.