NYC AEDT Bias Audits: What an Annual Audit Actually Looks Like

60-Second Summary

AEDTs require annual independent bias audits, public summary disclosure, and candidate notice.
Audit uses impact-ratio analysis across sex, race/ethnicity, and intersectional categories.
An audit is not a one-time compliance task — it's an annual operating cadence with vendor cooperation.
Many vendors publish a 'model audit' that is NOT your audit. You need a deployer-specific audit using your data.

NYC Local Law 144 has been in force since July 2023 and is the most operationally precise AI-in-hiring law in any major US jurisdiction. Despite that, most employers using AEDTs in their NYC hiring funnel are out of compliance — typically because they treat the vendor's published audit as their audit, which it almost never is.

What an AEDT is

An Automated Employment Decision Tool is any computational process that issues a 'simplified output' (score, ranking, classification, recommendation) substantially assisting employment decisions, including hiring and promotion. Scope is broad: resume parsers that rank, structured assessment scorers, video-interview rankers, and ATS features that 'recommend' candidates can all qualify.

What the audit covers

Required audit calculations

1
Selection rate by group
For each scoring category and each demographic group: % of candidates in that group selected (or scored above threshold).
2
Impact ratio
Selection rate of each group divided by the selection rate of the most-selected group. <0.80 (the 'four-fifths rule') triggers disparate impact concern.
3
Sex categories
Male, Female, and (where data allows) non-binary.
4
Race/ethnicity categories
Per EEO-1 categories: Hispanic/Latino, White, Black, Asian, Native Hawaiian/Pacific Islander, American Indian/Alaska Native, Two or More.
5
Intersectional categories
Sex × race intersections (e.g. Black women, White men) — required since 2023 amendments.

What 'independent' means

The auditor must not have been involved in using, developing, or distributing the tool, and must not have an employment or financial relationship with the employer or vendor that would create a conflict. Practically: your audit firm cannot also be your vendor's consultant, your tool's developer, or your in-house data science team using the tool. Most large accounting firms and specialised employment-law consultancies have stood up AEDT audit practices since 2023.

Why a vendor's model audit is not enough

Vendor model audit

Run on vendor's training/test data
Uses generic candidate pool
Same for every customer
Sometimes useful as a starting point

Your AEDT audit (LL144)

Run on your actual hiring data
Uses your candidate population
Specific to your job categories
Required by law if you can't rely on aggregate vendor data

LL144 allows reliance on the vendor's audit only if (a) the vendor provided audit covers historical data from their employer customers and (b) you cannot reasonably contribute your own data. Most employers can contribute their data; most should.

Running this as a yearly cadence

Quarter 1 — confirm AEDT scope: every tool used in NYC hiring funnel in the last 12 months.
Quarter 1 — engage independent auditor; agree data extract and demographic linkage method.
Quarter 2 — provide 12 months of selection data; auditor produces draft impact ratios.
Quarter 2 — review with legal; address any subgroup with impact ratio <0.80.
Quarter 3 — publish audit summary on careers site (specific elements required by rule).
Quarter 3 — update candidate notice: 10 business days before tool use, with opt-out where required.
Quarter 4 — retain audit + supporting data per the 2-year minimum; plan next year's audit.

What candidate notice must contain

Notice that an AEDT will be used, the job qualifications/characteristics the AEDT will use, and (on request within 30 days) the type and source of data, the data retention policy, and an alternative selection process if requested. Build a standard template, legal-reviewed, deployed in your ATS and on the careers site.

Enforcement reality

Civil penalties are $500 for a first violation and $500–$1,500 for each subsequent violation, with each day of non-compliance counted separately. Class-action exposure under related state-law theories is the bigger risk. The reputational risk — a public AEDT audit showing your tool failing the four-fifths rule for a protected group — is bigger still.

Written by Pawan Joshi.Sources cited inline.

First published 23 Jun 2026See site changelog →