What it means
- An auditor independent of the people who built, sold or use the AI system tests whether its outcomes differ unfairly across groups (for example sex, race or ethnicity, age, and in India also caste, religion, region or language where lawful data exists).
- The audit uses real or representative data, calculates measures such as selection rates and impact ratios, and documents results.
- Where law requires, a summary of results is published and affected people are notified before the tool is used on them.
- ISO 27001 reuse: None. New for AI, not covered by ISO 27001.
Why it matters
- NYC Local Law 144 is the main driver today. Employers and employment agencies using an automated employment decision tool (AEDT) for candidates or employees in New York City must:
- have an independent bias audit done within one year before use,
- publish a summary of the results (including selection or scoring rates and impact ratios by sex, race and ethnicity, and intersectional categories),
- give candidates notice at least 10 business days before use.
- Enforced by the NYC Department of Consumer and Worker Protection since July 2023, with civil penalties per violation.
- Indian HR-tech and SaaS companies selling screening, assessment or ranking tools to US employers will be asked to support or supply this audit.
- In India, the India AI Governance Guidelines encourage algorithmic audits and automated bias detection, and a pending private member's bill proposes bias audits.
Checklist
- Identify AI systems where a bias audit is legally required (for example AEDTs used for NYC roles) or contractually expected.
- Select an auditor who is independent: not involved in building or using the tool and with no financial interest in it.
- Agree the audit scope: protected categories, data source (historical or test data), metrics (selection rate, scoring rate, impact ratio), intersectional analysis.
- Prepare data with demographic fields handled lawfully and securely (SC-AI-011 Lawful basis and consent for personal data used in AI).
- Run internal fairness testing first so the external audit has few surprises (SC-AI-017 Fairness and discrimination testing of outputs, SC-AI-013 Data bias and representativeness checks).
- Publish the audit summary and distribution date where required; keep it current (annual refresh for NYC LL144).
- Send candidate notices in time and offer the alternative process or accommodation information required.
- Track remediation for adverse findings and re-test.
- For vendors: decide in contracts who commissions the audit and who publishes (SC-AI-007 Third-party and vendor AI due diligence).
Evidence examples
- Bias audit report signed by the independent auditor, with methodology and impact ratios.
- Auditor independence declaration and engagement letter.
- Published audit summary on the careers page with date.
- Candidate notice templates and send logs.
- Remediation plan and re-test results.
- Contract clauses allocating audit responsibility between vendor and employer.
Common pitfalls
- Using the model developer's own team and calling it "independent".
- Audit done once and not refreshed within a year.
- Too little demographic data to compute meaningful ratios, with no plan to fix it.
- Vendor and customer each assuming the other will commission the audit.
Satisfies
- BindingNYC Local Law 144 (AEDT)
Also relevant to
- India AI Governance Guidelinesguidance mapping (algorithmic audits and automated bias detection; no obligation IDs)
- AI (Ethics and Accountability) Bill 2025guidance mapping (pending bill; no obligation IDs)
Standards crosswalk
Clause references only. Check the standard itself for the exact wording.
ISO/IEC 42001
- 9.1 monitoring, measurement, analysis and evaluation
- A.5 AI system impact assessment
- A.6.2.4 verification and validation
ISO/IEC 27001
No direct equivalent
SOC 2
No direct equivalent
Key takeaways
- NYC Local Law 144 makes independent bias audits mandatory for hiring tools used for NYC roles; Indian HR-tech exporters are in scope through customers.
- Independence is the point: the auditor must not have built, sold or used the tool.
- Do internal fairness testing first; the external audit confirms it.