Level 4 · The arc, by handoffs

The arc, by handoffs

Six beats. As in previous levels, all three roles are present from the first: 🧭 Nerea at the top (portal · product), 🛡️ James in the middle (portal · quality), 🧑‍💻 Martha on the right (CLI + git). The lesson of this arc is AMBER → GREEN —a real, honest inconclusiveness that genuinely resolves once measured with real power (OUT-OF-FOLD over all 1,000 rows, not a 200-row held-out)— with a proxy risk that surfaces when reviewing the model and is left untreated, closing with a legitimate approval that consciously accepts that residual. And a new sixth beat: the periodic review that reopens that approval and forms an approval chain with an effective approver — the party of record ultimately accountable.

The hard data in each beat —the commands, the commit message, the diff and the gate colour— is not hand-written: every GuionStep block reads it from the repository’s steps/ script. Were a step to not exist, the build would fail; what you read here is exactly what the repository does. The commands and commit messages are the verbatim Spanish artefacts of the repo you clone.

1Declare the §5(b) + DORA system and the exclusion risk

🧭 Nerea

Nerea is the Product Owner at NovaCredit, a credit institution deploying a scoring model. She opens the portal and runs the authoring missions that generate the governance manifest froga.yaml. Three things matter:

1. The tier is high, declared by a human, via the use route. The classify-system mission fills in the classification block with the basis: “EU AI Act Annex III §5(b) (creditworthiness evaluation / credit scoring)” — plus “DORA Art. 6 (ICT risk of the financial entity)”. The high-risk trigger here is Annex III §5(b), not Art. 6(1). The engine does not infer it; Nerea declares it and owns it.
2. Two regimes are declared, and one is the overlay. The declare-standards mission writes applicable_standards with the spine (eu/ai-act@2024, eu/pren-18228@2026, iso/23894@2023) plus eu/dora@2022 — the DORA overlay — with a dora.entity block (the LEI, the entity type credit_institution). Declaring eu/dora@2022 is what makes the engine assemble the DORA Register later.
3. The risk is unfair credit exclusion. The identify-risks mission adds risk.unfair-credit-exclusion — “the model may unjustly exclude protected groups by inheriting historical patterns of discrimination.” The control that will confirm the residual is demographic-parity difference — a blocking fairness gate (< 0.092).

The missions open a PR in the system’s repository. The turn passes to James, who reviews and merges it. In the script, the identity, the framing and the risk programme land as three froga.yaml manifest commits —the diff of the last one shows the risks with their numeric gates fixed before any model exists:

Identidad del sistema (froga init + patch)🧭 Nerea

Contexto, estándares aplicables y encuadre (crédito Anexo III §5(b) + DORA)🧭 Nerea

Programa de riesgos con gates numéricos y firmantes (antes del modelo)🧭 Nerea

🛡️ James

James receives the PR opened by Nerea’s missions, reviews that the manifest is correct — tier high via Annex III §5(b) (creditworthiness) plus DORA Art. 6 (ICT risk), standards eu/ai-act@2024 + eu/pren-18228@2026 + iso/23894@2023 + eu/dora@2022 with a dora.entity block, unfair-credit-exclusion risk in full vocabulary — and merges it. The declared froga.yaml is now in the repository.

↓ relevo calidad → dev · high-risk §5(b) + DORA overlay declared

🧑‍💻 Martha

There is nothing to measure yet, but Martha leaves the reproducer ready in git: the uv + DVC project, the real Statlog German Credit dataset (UCI #144, CC-BY-4.0) with its Croissant manifest, and the featurization step are already script commits:

Repo, identidad, proyecto uv (uv init + deps), DVC, pubkey y README del reproductor🧑‍💻 Martha

Commands⧉ copy

15

• git init -q -b main
• git config user.email marta@demo.seigarrena.dev
• git config user.name "Marta Demo"
• uv init --bare --name vldemo-loan-scoring --python 3.12
• patch: pyproject-index.patch
• uv add "dvc>=3" pyyaml froga "venturalitica==0.6.11" "mlcroissant>=1.0" pyarrow "scikit-learn==1.8.0" fairlearn pandas joblib
• patch: gitignore.patch
• patch: gitattributes.patch
• patch: readme.patch
• uv sync
• uv run dvc init --no-scm -q
• uv run dvc config core.analytics false
• froga pubkey --out .froga/PUBKEY.txt
• git add -A
• git commit -m "init: repo, proyecto uv (deps reales) y README del reproductor"

Dataset German Credit + manifiesto Croissant + versionado DVC🧑‍💻 Martha

Pipeline de rasgos sobre German Credit — escribe Y corre (dvc repro)🧑‍💻 Martha

With the manifest declared and the scaffolding in git, the turn passes to measuring (beat 2).

Venturalítica·demo.venturalitica.ai

DeclaredNext: run and measure

Venturalítica·demo.venturalitica.ai

Which Annex III category applies?

Education and vocational trainingHR and hiringEssential public servicesCredit scoringHealthcare servicesWelfare services

Which MDR class applies to the device?

Class I (low risk; self-certified)Class I sterile (NB for sterilization)High-risk AIClass I with measurement (NB for metrology)High-risk AIClass IIa (medium risk; NB required)High-risk AIClass IIb (medium-high risk; NB required)High-risk AIClass III (high risk; NB + design dossier)High-risk AI

Any additional regime?

DORA (financial entity)GDPR (personal data)MDR (medical device)ET Art. 64.4.d (employment algorithms)Law 15/2022 (equality)

Venturalítica·demo.venturalitica.ai

Significant risks

IDTitleLikelihoodIndividual impactSocietal impactOrganizational impactAssign treatment to (optional)

1

ID

Title

LikelihoodRareUnlikelyPossibleLikelyAlmost certain

Individual impactLowMediumHighCritical

Societal impactLowMediumHighCritical

Organizational impactLowMediumHighCritical

Assign treatment to (optional)

2

ID

Title

LikelihoodRareUnlikelyPossibleLikelyAlmost certain

Individual impactLowMediumHighCritical

Societal impactLowMediumHighCritical

Organizational impactLowMediumHighCritical

Assign treatment to (optional)

3

ID

Title

LikelihoodRareUnlikelyPossibleLikelyAlmost certain

Individual impactLowMediumHighCritical

Societal impactLowMediumHighCritical

Organizational impactLowMediumHighCritical

Assign treatment to (optional)

4

ID

Title

LikelihoodRareUnlikelyPossibleLikelyAlmost certain

Individual impactLowMediumHighCritical

Societal impactLowMediumHighCritical

Organizational impactLowMediumHighCritical

Assign treatment to (optional)

5

ID

Title

LikelihoodRareUnlikelyPossibleLikelyAlmost certain

Individual impactLowMediumHighCritical

Societal impactLowMediumHighCritical

Organizational impactLowMediumHighCritical

Assign treatment to (optional)

2Measure V1: honest, underpowered amber

🛡️ James

Waiting. Nerea has declared the system and James has merged it; now James waits for Martha’s first measurement. He does not touch the CLI: when Martha reports the gate colour, he will see whether the evidence proves the residual or only estimates it.

↓ relevo calidad → dev · tag loan-v1.0.0-amber — OOF over 1,000 rows: the CI still crosses the line

🧑‍💻 Martha

Martha measures the system as it stands — the inherent risk, untreated. First she compiles the gate contract —which controls will be measured and with what threshold—:

Compilar el plan de evaluación OSCAL (antes del modelo)🧑‍💻 Martha

Commands⧉ copy

• froga compile
• git add -A
• git commit -m "compile: assessment plan OSCAL (contrato del gate, antes del modelo)"

Now she adds the base model —a plain logistic regression with no fairness mitigation— and runs the evidence:

Modelo logreg base V1 — escribe Y corre (gate ÁMBAR honesto, OOF n=1000)🧑‍💻 MarthaExpected verdictAMBER

📍 reproducir loan-v1.0.0-amber

git clone https://github.com/Venturalitica/vldemo-loan-scoring
cd vldemo-loan-scoring
git switch -c try loan-v1.0.0-amber

⧉ copiar

Commands⧉ copy

• patch: params.patch
• patch: dvc-evaluate.patch
• patch: train.patch
• patch: evaluate.patch
• patch: compliance-eval.patch
• froga runmay fail
• git add -A
• git commit -m "modelo: logreg base V1 sin mitigación (train/evaluate + dvc.yaml) — run: V1 evidencia base (gate ÁMBAR infrapotenciado; seed=42)"

The unfair-credit-exclusion control comes back AMBER, and why it is amber is the whole point:

• actual_value = 0.1050, gate < 0.092 → the point estimate fails (passed: false): the untreated model genuinely discriminates, measured with real power.
• The power block reports a bootstrap 95% CI of [0.0425, 0.1702], measured OUT-OF-FOLD over all 1,000 rows (690 men, 310 women; 5-fold StratifiedKFold — every row is predicted by a model that never saw it train). The lower edge, 0.0425, falls below the 0.092 line.

So the interval straddles the threshold from both sides: the engine cannot prove with certainty that the disparity breaches the limit, so it fires inconclusive() → AMBER, not a conclusive red. This is honest in a different sense than a small sample: with 1,000 rows the power is real — the interval straddles because the untreated residual is genuinely close to the threshold, not because data is missing. The bundle is signed and anchored regardless — an AMBER result is evidence, recorded honestly.

This AMBER is not a 'soft red' — and it is not engineered

It is tempting to want a clean RED→GREEN story (Level 3 had one). But fabricating a RED the data cannot support would violate the engine’s honesty rule: it never claims more certainty than its evidence carries. With 1,000 rows measured OOF, power is no longer the problem — the interval straddles because the untreated residual is genuinely close to the line. So the honest verdict is AMBER — and here, unlike an underpowered sample, it is the treatment, not more rows, that can resolve it.

The risk is real and the evidence inconclusive. But before treating, Nerea takes another look at the model — and finds something the gate did not see.

Venturalítica·demo.venturalitica.ai

●Evaluated · AMBER

3Review V1: a proxy risk surfaces (left untreated)

🧭 Nerea

Reviewing the V1 evidence, Nerea does not just look at the gate colour: she looks at the model. And she sees something the fairness gate did not capture — the postal code enters as a feature. In areas with residential segregation, the postal code correlates with ethnic origin: the model can learn to discriminate on a protected attribute without using it explicitly. This is the A.1.c beat of the ISO 23894 process (§6.4.2, risk analysis): a risk that surfaces when you examine the system, not when you declare it up front.

Nerea declares it in froga.yaml as risk.proxy-discrimination — individual impact HIGH, likelihood LIKELY —, and makes a deliberate choice: she does not treat it yet. Removing the postal code would degrade the model and it needs more analysis before acting; so the risk stays identified but untreated, above the individual appetite (above-appetite):

Riesgo de discriminación por proxy identificado (above-appetite, sin tratar)🧭 Nerea

Note what does not change: with no treat: block and no measure, the risk contributes no control_result, so the gate colour does not move — it stays AMBER. An above-appetite untreated risk does not paint the gate red by itself; what it does is open a gate in the approval: someone will have to accept it consciously, with a reason, or the system is not approved. That is the second residual that will travel to the governance beat.

↑ relevo dev → producto · A.1.c — the gate colour does NOT change; it opens a gate in the approval

🛡️ calidad

🧑‍💻 Martha

Waiting. Identifying the risk is a governance act by the Product Owner; Martha will see it reflected in the manifest and know the approval will carry an added condition. Her next step is still the fairness treatment.

Venturalítica·demo.venturalitica.ai

1 residual for the decision

Above-appetite proxy risk, untreatedproxy-discrimination (postal code as a proxy for ethnic origin) — individual HIGH × LIKELY; no measures → the gate colour does not change

4Treat the risk: the number improves, the power does not (still amber)

🛡️ James

Waiting. James has seen the AMBER — the gate is inconclusive, measured OOF over all 1,000 rows. He does not touch git; he waits for the re-measured evidence, where he will read whether the treatment resolves the inconclusiveness.

↓ relevo calidad → dev · tag loan-v2.0.0-green — the treatment demonstrates the residual: conclusive GREEN

🧑‍💻 Martha

This is the treatment (ISO 23894 §6.5). As in Level 2, the treatment replaces train.py: the estimator goes from a plain logistic regression to a fairness-aware learner with fairlearn’s DemographicParity constraint (ExponentiatedGradient), a versioned code change in git that the engine attributes the risk treatment to. The diff is the treatment, and the same step runs the gate again:

Abre la rama de tratamiento + mitigación de paridad demográfica fairlearn (§6.5) — escribe Y corre (gate VERDE)🧑‍💻 MarthaExpected verdictAMBER

📍 reproducir loan-v2.0.0-green

git clone https://github.com/Venturalitica/vldemo-loan-scoring
cd vldemo-loan-scoring
git switch -c try loan-v2.0.0-green

⧉ copiar

Commands⧉ copy

• git checkout -b tratamiento/mitiga-paridad
• patch: train.patch
• froga runmay fail
• git add -A
• git commit -m "treatment: aplica mitigación de paridad demográfica (fairlearn DemographicParity) sobre el sexo — ISO 23894 §6.5"

And this time the gate turns GREEN, and the reason is the honest finding of the level:

• actual_value = 0.0065 — the point estimate drops far below V1’s 0.1050, and far below the < 0.092 gate.
• The power block: bootstrap 95% CI [0.0010, 0.0645] — the upper edge sits below the 0.092 threshold.

The CI no longer crosses the line: it lands entirely below it. The engine fires adequate() → conclusive GREEN. Read it precisely: the treatment is a real, versioned intervention, and on the same 1,000 rows measured OOF, the number drops AND the power proves it. No extra data was needed — the same sample size that left V1 in AMBER certifies V2 GREEN, because the treated residual sits genuinely far from the threshold.

Why AMBER → GREEN is the honest verdict here — the same rule as retina

Same machinery, now the same conclusion as retina. Retina’s sensitivity was measured on a held-out cohort of n ≈ 1,279: its CI was tight and its lower bound cleared the line → a provable clean GREEN. Credit fairness here is measured OOF on all 1,000 rows: in V1 the CI straddles the threshold (the untreated residual is genuinely undecided) → AMBER; in V2, treated, the CI lands entirely below it → GREEN. The engine reports GREEN not because anyone wanted amber, but because this time the evidence genuinely proves the residual. AMBER and clean-GREEN are the same honest rule applied to the same amount of evidence, before and after treatment — the difference is the treatment, not the data volume.

Treated, signed, conclusively GREEN — and the untreated proxy risk from beat 3 is the only thing left for the decision.

Venturalítica·demo.venturalitica.ai

Undeclared

Declared

Assured

Tested

Treated

Approved

Marketable

5Governance: approve with both residuals on record and close

🛡️ James

This is the close — and the contrast with retina is the point. Martha’s treated, signed GREEN evidence is in the repo, with the request already emitted (Martha’s column). James reviews the bundle, verifies Martha’s signature, and sees fairness resolved: the treatment demonstrated the residual, measured OOF over all 1,000 rows — no amber left to accept. But two governance conditions remain, the ones the gate never measures: proxy-discrimination stays above-appetite, untreated, and the opacity residual exceeds appetite with no technical control bounding it.

James approves — the approval accepts those two governance residuals at once: the untreated proxy risk and the opacity residual. The approval is a git-native, signed act: a DSSE act .froga/acts/NNNN-approve.json —signed ECDSA-P256 and verified against James’s key declared in signers:— records who approved; the commit that adds it is plain text, no GPG signature (do not expect the «Verified» badge when you click it on GitHub), but the act itself is cryptographically signed; the reason that accepts both residuals lives in the portal record (the CLI does not take --reason and drops it with a warning); and alongside the act the evidence is signed (bundle.json.sig and the conformances). With the approval in history, conformance is re-emitted and prEN 18228 clause 11 (risk management review) turns from Gap to Covered on reading the approval act:

Aprobar (aceptación consciente del doble residual)🛡️ James

📍 reproducir loan-v2.0.0-approved

git clone https://github.com/Venturalitica/vldemo-loan-scoring
cd vldemo-loan-scoring
git switch -c try loan-v2.0.0-approved

⧉ copiar

🖊️ James approves

Reason: «Aceptación consciente del riesgo residual: proxy-discrimination sigue above-appetite sin tratar (se acepta con motivo) y el residual de opacidad excede el apetito; la equidad medida OUT-OF-FOLD (n=1000) cierra VERDE tras el tratamiento fairlearn. Revisión P6M.»

The engine erases neither residual: it anchors them in the approval, attributable and verifiable. That is the difference between consciously accepting a residual and hiding it.

Why this approve is legitimate — and retina's was not

A consumer-credit system under Annex III §5(b) is high-risk, but its conformity route is self-declaration: there is no external notified body for §5(b) AI systems (unlike an MDR Class IIa device, which legally cannot close on its own attestation). So the quality role may close the cycle by approving — provided the approval is honest about what it accepts. Here it is: fairness is already GREEN, demonstrated; the residuals left are above-appetite proxy and above-appetite opacity, and the approval is conscious risk acceptance with both on the record. That is a legitimate self-declaration — the honest counterpart to retina’s honest pause. The difference is not effort; it is the regime’s conformity route.

↓ relevo calidad → dev · the signed act .froga/acts/NNNN-approve.json records who approved; the reason with both residuals lives in the portal record

🧑‍💻 Martha

Martha closes her part of the cycle from the CLI. With V2 GREEN signed, she emits the conformance and reconstruct over the applicable standards and requests approval:

Conformance + reconstruct firmados (entitlement de pago + 6 normas)🧑‍💻 Martha

Commands⧉ copy

• patch: entitlement.patch
• froga conformance --standard eu/pren-18228@2026 --outmay fail
• froga conformance --standard iso/23894@2023 --outmay fail
• froga conformance --standard eu/ai-act@2024 --outmay fail
• froga conformance --standard eu/dora@2022 --outmay fail
• froga conformance --standard eu/pren-18283@2026 --outmay fail
• froga conformance --standard eu/pren-18282@2026 --outmay fail
• froga reconstruct --outmay fail
• git add -A
• git commit -m ".froga: conformance + reconstruct firmados"

Solicitar aprobación (gate verde; residual proxy/opacidad pendiente de aceptación)🧑‍💻 Martha

📍 reproducir loan-v2.0.0-request

git clone https://github.com/Venturalitica/vldemo-loan-scoring
cd vldemo-loan-scoring
git switch -c try loan-v2.0.0-request

⧉ copiar

git commitsolicitud de aprobación (el acto lo commitea froga request)

The approval is James’s (above): Martha requested it with the act before, she cannot grant it herself. Once James’s signed act .froga/acts/NNNN-approve.json lands —with both governance residuals consciously accepted—, Martha re-emits conformance so that froga derives the cycle from the git history, now with the real approval:

Re-emitir conformance tras aprobar (cl.11 = Approved)🧑‍💻 Martha

Commands⧉ copy

• froga conformance --standard eu/pren-18228@2026 --outmay fail
• froga conformance --standard iso/23894@2023 --outmay fail
• froga conformance --standard eu/ai-act@2024 --outmay fail
• froga conformance --standard eu/dora@2022 --outmay fail
• froga conformance --standard eu/pren-18283@2026 --outmay fail
• froga conformance --standard eu/pren-18282@2026 --outmay fail
• froga reconstruct --outmay fail
• git add -A
• git commit -m ".froga: re-emisión de conformance + reconstruct tras aprobar (cl.11 = Approved)"

The system ends up declared, treated, GREEN, requested and approved with the double governance residual (proxy + opacity) on record: the end of the arc, with James’s approval anchored at the loan-v2.0.0-approved tag above.

Venturalítica·demo.venturalitica.ai

Undeclared

Declared

Assured

Tested

Treated

Approved

Marketable

Venturalítica·demo.venturalitica.ai

Approve with the double residual on record

Fairness is now GREEN (treatment demonstrated, OOF over all 1,000 rows) — two governance residuals remain: above-appetite untreated proxy-discrimination + opacity above appetite. James accepts both consciously, with a reason (Annex III §5(b) self-declaration).

Untreated proxyproxy-discrimination stays above-appetite — requires conscious acceptance with a reason.

OpacityThe opacity residual exceeds appetite, with no technical control bounding it — requires conscious acceptance.

Reason *

CancelApprove

6Periodic review: the approval chain and the effective approver

🛡️ James

The credit froga.yaml declared review_interval: P6M: a semi-annual review (ISO 23894 §6.6, the continual oversight of Art. 14). Past the six months, the portal offers the review and James runs it — and, as with any review, it reopens the cycle:

Revisión periódica P6M (la aprobación queda suspendida)🛡️ James

📍 reproducir loan-v2.0.0-review

git clone https://github.com/Venturalitica/vldemo-loan-scoring
cd vldemo-loan-scoring
git switch -c try loan-v2.0.0-review

⧉ copiar

🖊️ James reviews

The engine derives «under periodic review» (froga status: the system is under periodic review (ISO 23894 §6.6); compliance must re-approve it): the approval above is suspended. But here the arc differs from Level 2’s: the evidence does not change. There is nothing to re-measure —the double residual (untreated proxy + opacity) is a governance decision, not a number to re-run—, so there is no froga run in between. James reviews the same V2 bundle and re-approves it as is:

Re-aprobar sobre el mismo bundle (cadena con aprobador efectivo, #667)🛡️ James

📍 reproducir loan-v2.0.0-reapproved

git clone https://github.com/Venturalitica/vldemo-loan-scoring
cd vldemo-loan-scoring
git switch -c try loan-v2.0.0-reapproved

⧉ copiar

🖊️ James approves

Reason: «Re-aprobación tras la revisión periódica (P6M) sobre la MISMA evidencia vigente: se re-confirma la aceptación consciente del doble residual (proxy-discrimination above-appetite sin tratar + opacidad por encima del apetito). El aprobador efectivo del expediente pasa a ser esta re-aprobación (cadena aprobar→revisar→re-aprobar).»

The approval chain and the effective approver (#667)

On re-approving over the same bundle, the record no longer has one approval but a chain: approve (beat 5) → review (this beat) → re-approve. The engine exposes the whole chain —no approval is lost, it is the trace Art. 12 traceability demands— and marks the effective approver: the last approval act covering the current evidence. Here both approvals are James’s, so the chain is James → (James reviews) → James; the effective act is the re-approval (its more recent commit), not the original approval. What closes issue #667 is that froga status (the CLI) and the portal now agree on which is effective: both point to the re-approval (froga status: management approval … Diego Demo, with the re-approval’s sha). Before, the CLI showed the original approval and the portal the re-approval — two different “ultimate accountables” for the same system. Now the party of record is unambiguous: whoever signed the current re-approval.

The system returns to «approved», now with the full chain in the record and the effective approver anchored at the loan-v2.0.0-reapproved tag above.

↓ relevo calidad → dev · the review reopens the approval over the SAME bundle → chain approve→review→re-approve with an effective approver (#667)

🧑‍💻 Martha

Waiting — and this time Martha’s turn is not to act. Reviewing a governance residual (the untreated proxy, the opacity) does not require re-measuring: there is no parameter to change nor gate to re-run, the V2 evidence is still current. All of the beat’s action lives in the portal, between James’s review and re-approval. Martha will see it reflected in the record’s approval chain, with no commit of her own in between — the exact contrast with Level 2, where the review did send you back to measure.

Venturalítica·demo.venturalitica.ai

1 review pending

Periodic risk review (P6M due)The froga.yaml declares review_interval: P6M. Six months have passed since the approval; the portal offers the review. On review, the cycle turns to «under review»: the approval is suspended until re-approval.

Venturalítica·demo.venturalitica.ai

Re-approve over the same evidence

The review did not change the evidence: the same V2 bundle, the same double residual. James re-confirms the conscious acceptance — and in doing so becomes the effective approver of record.

Double residual re-confirmedproxy-discrimination above-appetite untreated + opacity above appetite — re-accepted consciously with a reason on the re-approval.

Reason *

CancelRe-approve

---

Next: In depth — the honest AMBER→GREEN arc, git-native Art. 12, and DORAThe honest AMBER→GREEN arc and OOF statistical power, the Article 12 activity log as a commit graph, and the DORA Register of Information deliverable-not-verdict rule.