Level 4 · Practice & recap
Run it yourself
Section titled “Run it yourself”Use the V1 milestone chip (in the arc) to clone the repo and check out the starting point, then install the project dependencies:
uv sync # install deps from pyproject.toml + uv.lock (includes fairlearn)With the project environment active, reproduce the arc. Each block below is the REAL step from the steps/ script —the same one that builds the published repo, not a hand-written copy—; the “copy” button gives you the exact command:
1 · Measure V1 (unmitigated) → AMBER. First compile the gate contract; then add the base model and run it. The point fails (≈ 0.1050 > 0.092), but the bootstrap 95% CI crosses the line from the lower edge (ci_low ≈ 0.0425, measured OUT-OF-FOLD over all 1,000 rows, 5-fold) → INCONCLUSIVE:
froga compilegit add -Agit commit -m "compile: assessment plan OSCAL (contrato del gate, antes del modelo)"
patch: params.patchpatch: dvc-evaluate.patchpatch: train.patchpatch: evaluate.patchpatch: compliance-eval.patchfroga runmay failgit add -Agit commit -m "modelo: logreg base V1 sin mitigación (train/evaluate + dvc.yaml) — run: V1 evidencia base (gate ÁMBAR infrapotenciado; seed=42)"
Read the evidence the run produced — your OWN signed bundle: .froga/bundle.json → control_results[unfair-credit-exclusion] shows actual_value ≈ 0.1050, threshold 0.092, passed: false, and power.ci_low ≈ 0.0425 / power.ci_high ≈ 0.1702 — the interval straddles the threshold → AMBER / inconclusive (neither a conclusive red nor a clean green), with real power (n=1,000, not n=200).
2 · Treat the risk (turn on the fairness-aware learner) → GREEN. The treatment is a single versioned change: replace train.py —the diff below is exactly what you change— and commit it with the script’s message. The estimator goes from a plain logistic regression to fairlearn’s ExponentiatedGradient(DemographicParity):
git checkout -b tratamiento/mitiga-paridadpatch: train.patchfroga runmay failgit add -Agit commit -m "treatment: aplica mitigación de paridad demográfica (fairlearn DemographicParity) sobre el sexo — ISO 23894 §6.5"
Step 2 is the honest core: with the SAME power (1,000 rows, OOF) the gate lands GREEN, because this time the whole bootstrap CI (ci_high ≈ 0.0645) sits below the threshold — the engine reports that honestly as soon as the evidence proves it, no sooner and no later. Step 080 reproduces loan-v1.0.0-amber; step 100 reproduces loan-v2.0.0-green.
3 · Periodic review: the approval chain. The arc does not end at beat 5’s approval. The froga.yaml declared review_interval: P6M: past six months, the system enters its periodic review (ISO 23894 §6.6). Here there is nothing to re-measure —the double governance residual (proxy + opacity) is a decision, not a number—, so the review and the re-approval are governance acts over the same V2 bundle, with no froga run in between:
Reason: «Re-aprobación tras la revisión periódica (P6M) sobre la MISMA evidencia vigente: se re-confirma la aceptación consciente del doble residual (proxy-discrimination above-appetite sin tratar + opacidad por encima del apetito). El aprobador efectivo del expediente pasa a ser esta re-aprobación (cadena aprobar→revisar→re-aprobar).»
The review (froga review) reopens the cycle —the engine moves the system to «under review» and suspends the approval— and the re-approval (froga approve) re-closes it. The interesting effect is in the record: there is no longer one approval but a chain — approve → review → re-approve —, and the engine marks the effective approver, the last approval act covering the current evidence (the re-approval). Step 080 reproduces loan-v1.0.0-amber; step 100, loan-v2.0.0-green; and the re-approval lands at loan-v2.0.0-reapproved.
What you just saw
Section titled “What you just saw”Your second overlay, this time in finance: one consumer-credit scoring model under two regimes at once — EU AI Act high-risk (via Annex III §5(b), the use route, contrasting retina’s Art. 6(1) product route) and DORA, because the operator is a financial entity, so the model is an ICT asset. The fairness gate was honest AMBER → GREEN: measured OUT-OF-FOLD over all 1,000 rows (5-fold, not a 200-row held-out), the untreated V1 was inconclusive (point 0.1050 above the gate, but the bootstrap CI straddles from the lower edge, 0.0425), and the fairlearn DemographicParity treatment dropped the number (point 0.0065) and closed the power (the whole CI lands below the line, ci_high ≈ 0.0645) — so the engine certified a verdict its evidence could, this time, actually carry. Reviewing V1, Nerea identified a second risk (A.1.c): proxy-discrimination —postal code as a proxy for ethnic origin— which she left above-appetite untreated; it does not change the gate colour, but it opens a gate in the approval. The cycle closed with a legitimate approve — request + approve consciously accepting the double governance residual (the untreated proxy and the opacity residual — fairness is no longer part of what’s accepted, it’s resolved), the reason recorded in the portal record (cl. 11 turns from Gap to Covered) — because §5(b) credit self-declares (no external body), unlike retina’s pause. Art. 12 went deep as git-native, ex-ante record-keeping (the commit graph is the log; the signed acts in .froga/acts/ and the pipeline_lock_digest), and DORA contributed a signed Register of Information (xBRL-OIM) — a deliverable, never a verdict. The line to carry away: the fairness verdict is AI Act / ISO (GREEN after treatment); DORA is documentation — never “DORA gobernada”. And a sixth beat: the periodic review (P6M) reopens the approval over the same bundle and forms a chain approve → review → re-approve; the engine exposes the whole chain and marks the effective approver —the party of record ultimately accountable— with froga status and the portal now in agreement (issue #667).
Self-check
Section titled “Self-check”Nerea declares risk.proxy-discrimination while reviewing V1 but does NOT treat it. Why does the gate colour not change — and what does that risk then compel?
Because the gate colour is painted by measured controls (control_results), and a risk that is identified but untreated carries no treat: block and no measure, so it contributes no control_result — there is no number to compare against a threshold, so the gate stays as it was (AMBER from V1’s disparity, measured OOF and still untreated at this point). What the proxy does is not paint the gate red: being above-appetite (individual impact HIGH × likelihood LIKELY, over the individual appetite), it opens a gate in the approval — the engine requires that someone accept it consciously, with a reason, or the system is not approved. This is the A.1.c beat of the ISO 23894 process (§6.4.2, risk analysis): the risk surfaces when you examine the model, not when you declare it up front, and its residual travels to the governance decision instead of vanishing.
In the periodic review, James re-approves the SAME bundle without measuring again. What is the 'approval chain' that appears then, and who is the 'effective approver'?
On re-approving over the same bundle (reviewing a governance residual does not require re-measuring: there is no parameter to change), the record stops having one approval and gains a chain of acts: approve (beat 5) → review (the P6M review) → re-approve. The engine exposes it whole —no approval is discarded, it is the trace Art. 12 traceability demands— and marks the effective approver: the last approval act covering the current evidence, i.e. the re-approval. In this demo both approvals are James’s, so the effective one is his re-approval (the most recent commit), not the original approval. What closes issue #667 is that froga status (the CLI) and the portal now agree on which is effective — both point to the re-approval —; before, they diverged (the CLI showed the original, the portal the re-approval), giving two different “ultimate accountables” for the same system. Contrast with Level 2: there the review did send you back to measure, and because there was new evidence the chain started over (a single act over the fresh bundle); here, with no new evidence, the chain grows over the same bundle.
The engine assembles a DORA Register of Information for this system. Why is that a 'deliverable, not a verdict', and why must you never say the system is 'DORA gobernada'?
Because DORA is binding sectoral law, not a harmonised AI-risk-management standard cited for Article 9 — so it sits outside the engine’s verdict-bearing authority chain and confers no presumption. The DORA output is therefore a signed deliverable + cross-map: the engine projects what was measured and declared into the ESAs’ xBRL-OIM Register of Information (B_01.01 entity / B_07.01 ICT function / B_05.01 third-party from the ML-BOM) — documentation a compliance officer hands a supervisor, the finance counterpart of retina’s MDR crosswalk. It is not a statement that the system “is DORA-conformant”. Concretely, no measure in the manifest cites a dora.* clause (DORA is only an advisory frameworks tag), so there is no per-clause verdict to teach — you teach the Register, not clause coverage. The conformity verdict on the model’s fairness belongs to the AI Act / ISO 23894, and in this demo it closes GREEN after treatment (measured with real power, OOF over all 1,000 rows). Calling the system “DORA gobernada” because it carries a Register is exactly the over-claim to avoid: DORA is ICT operational resilience, not model governance.
The fairness gate is AMBER at V1 and turns GREEN after the fairlearn treatment. Why is AMBER → GREEN the honest verdict here, rather than a manufactured RED → GREEN story or an AMBER forced by lack of data?
Because the cohort is measured with real power: all 1,000 rows, each with an OUT-OF-FOLD prediction (5-fold StratifiedKFold — every row is predicted by a model that never saw it train), not a 200-row held-out. At V1 the measured demographic_parity_diff ≈ 0.1050 sits above the < 0.092 gate, and the bootstrap 95% CI runs from 0.0425 to 0.1702 — it straddles the line from the lower edge — so the interval cannot prove the breach with certainty and the engine fires inconclusive() → AMBER. The fairlearn DemographicParity treatment then drops the number to ≈ 0.0065 and this time the whole CI sits below the threshold (ci_high ≈ 0.0645) — the treatment demonstrates the residual, with the same power that left V1 AMBER — so the verdict turns conclusive GREEN. Fabricating a RED or a GREEN the data could not support would both violate the engine’s honesty rule: it never claims more certainty than its evidence carries — and it does not stay AMBER once the evidence genuinely proves the residual either. AMBER and clean-GREEN (as in retina, n ≈ 1,279) are the same honest rule applied to whatever evidence is on hand at each point in the arc.
Article 12 is taught here as 'git-native record-keeping'. What does that mean, what does it enable, and where does its honest boundary lie?
It means the commit graph is the log: governance events are commits carrying signed acts, the lifecycle state lives in the signed acts in .froga/acts/ (NNNN-request.json, NNNN-approve.json) rather than a mutable database field, and the signed bundle pins exactly what was approved via the pipeline_lock_digest (model + data + params + threshold + residual). It is a deliverable / paper-trail, not a gating control — there is no “Art. 12 RED”; its quality is completeness and tamper-evidence. What it enables is the right to human review of an automated credit decision (GDPR Art. 22 / CJEU SCHUFA, C-634/21): a reviewer can git diff the model, data, threshold, and signed residual to see precisely what was decided and why. Its honest boundary: git-native Art. 12 covers code / model / dataset / decision reproducibility, not infrastructure resilience (backups, RTO/RPO, failover, pen-test, incident triage) — those are operational ICT records that belong to DORA’s world, not to the reproducibility log.
Where to go next
Section titled “Where to go next”You have now met your second overlay, the honest AMBER→GREEN arc, the untreated proxy risk of the A.1.c beat, a legitimate approve that consciously accepts the double governance residual (proxy + opacity), Article 12 git-native, and the DORA deliverable-not-verdict rule. To go deeper on the regimes this level touched: