Level 5 · Talent screening — employment, a defect that closes, a limit that gets declared

🟡 Partial — The risk loop is re-derivable today from the committed outputs of the public vldemo-talent-screening repo (one tag per beat), and it runs on a CPU in under a minute on the real Campus Recruitment dataset (n=215, CC0) — you run it yourself. The one blocking control (model-feature-leakage) closes at V2 → gate GREEN; the data/fairness findings (data-sample-adequacy, selection-rate-disparity-gender) stay ADVISORY, measured and documented but not certified — the sample is capped at n=215 and no retraining widens it. James approves green with that residual consciously accepted and quarterly post-market monitoring (Art.72). The talent tags resolve on github.com/Venturalitica/vldemo-talent-screening.

TL;DR

A talent-screening classifier (the demo employer is Example Talent Corp) reads a job applicant’s profile and predicts shortlist / reject. That decision gates a person’s access to a job, so the system is EU AI Act high-risk — and notice the route. Level 3’s retina screener was high-risk through Art. 6(1) + Annex I (the product-safety route); Level 4’s credit model through Annex III §5(b) (the creditworthiness route). This one is high-risk through Annex III §4 — employment / worker management: AI used to recruit, select, or screen applicants. James (Quality) and Martha (engineer) run the familiar loop on the real Campus Recruitment dataset (CC0, n=215) — and this level’s lesson is telling apart a defect you fix from a limit you declare. The one BLOCKING control closes: model-feature-leakage is RED at V1 because the model trains on salary, a feature present only for placed candidates (NaN→0 otherwise) and therefore collinear with the label — total leakage, a safety defect (Art.15). Martha flips mitigate: true, the pipeline drops salary, and the control flips GREEN (a real RED→GREEN by a code change) — it is the program’s ONLY blocking control, so the gate closes GREEN. Alongside it, two controls are genuinely MEASURED but stay ADVISORY (enforcement: audit, non-blocking): data-sample-adequacy (n/500 = 0.43 — Campus Recruitment is capped at 215 rows, there is no more to collect) and selection-rate-disparity-gender (selection-rate parity by sex, citing Ley 15/2022 Art. 23 — with n=215, ~24 women in the test cohort, it is underpowered). Certifying them as blocking would manufacture a threshold the sample cannot sustain, so the engine measures them, documents them, and does not block. At approval, James consciously accepts that residual — with a recorded motive and quarterly post-market monitoring of selection disparity (Art.72) — and the system ends up declared, treated, GREEN, requested, and approved. Finally Aitor, the external auditor, debuts: he runs froga verify --pubkey <key> with only a public key, confirms the signature is authentic and reads the honest GREEN/approved state, with the advisory residual on record — but he is not a notified body and does not certify. The one line to carry out: the engine certifies what it can measure reliably, and honestly documents what it cannot — without manufacturing either a permanent red or a certification the sample cannot support.

If you have not met the spine, the overlay idea, or the prior two overlays, read these first:

Level 4 · Consumer credit — AI Act + DORAThe previous overlay: a credit model high-risk via Annex III §5(b) (the credit route) plus DORA, closing with a legitimate approve-with-residual-on-record. Level 5 keeps the three-role structure (PO Nerea + QM James + dev Martha) and the honesty rule, but tells apart a CERTIFIABLE defect (which closes the gate) from a data/fairness limit that can only be documented and accepted — never certified.

More than the AI ActThe concept this level runs on: a sectoral regime stacked on the AI Act spine; one system, N regimes; the same signed bundle projected onto multiple catalogs. Here the 'overlay' is the employment dimension — national labour law + RGPD anchor the live obligation, not the (deferred) AI Act high-risk gate date.

Prerrequisitos

Level 2 (the high-risk spine), More than the AI Act, and ideally Level 3 and Level 4 (the prior overlays), plus What is a risk? for the RED / AMBER / GREEN vocabulary. A note on materials:

You run it yourself. The pipeline trains on Campus Recruitment (CC0, n=215, bundled in the repo) on a CPU in under a minute — no GPU, no downloads, no credentials. You can clone, check out a beat, and re-derive the verdict locally (§3). The committed evidence + nightly gate remain the source of truth — a local re-train may vary with dependency versions; the gate never re-trains.
Nerea (portal) declares the system; James (portal) reviews and approves; Martha (CLI + git) executes the pipeline and signs the evidence. New this level: Aitor, an external auditor who verifies with only a public key (§5).

By the end you'll be able to…

Explain why this system is high-risk through Annex III §4 (the employment / worker management route), and contrast it with Level 3’s Art. 6(1) product route and Level 4’s Annex III §5(b) credit route.
Tell apart a certifiable safety defect (model-feature-leakage, closes by a code change, a real RED→GREEN) from a non-certifiable data/fairness limit (data-sample-adequacy, selection-rate-disparity-gender: measured, documented, non-blocking because n=215 cannot support a reliable blocking threshold).
Explain the governance decision behind enforcement: audit vs enforcement: gate — why turning a structurally underpowered finding into a gate would manufacture a threshold with no reachable fix, and why honesty means measuring and documenting it rather than blocking or manufacturing a green.
Read an approval with conscious acceptance of the advisory residual: James accepts the data/fairness limit with a recorded motive and post-market monitoring (Art.72), and the cycle closes GREEN.
Describe how national labour law (ET Art. 64.4.d) and RGPD Art. 22 anchor the live obligation, and why national legislation is a NationalLegislation authority, rank 0 → no presumption of conformity (unlike a harmonised standard).
Explain why Aitor can verify the signed evidence with a public key and read the honest GREEN/approved state, but is not a notified body and does not certify.

The lessons of this level

The framing — the employment overlayA talent-screening classifier high-risk via the employment route (Annex III §4); which laws apply and why the certifiable defect is told apart from the data/fairness limit that only gets documented.

The arc, by handoffsThe one blocking control (feature leakage) CLOSES by a code change: RED→GREEN. The data/fairness findings stay measured and advisory; the cycle closes with James's approval and the residual consciously accepted.

In depth — Option B, Aitor, and Article 10Why the governance decision (Option B) separates the certifiable defect from the data limit, Aitor's debut (verifies with a public key and does not certify), and Article 10 deep with Articles 9/13/14 revisited.

Practice & recapReproduce the leakage's RED→GREEN arc and the approval with the advisory residual accepted, and self-check what you learned.