Skip to content
Venturalítica

Run the KAG (sei assess)

🟡 Partial — The KAG is a deterministic mock in v1; the real service (Python via API) is a future line.

ISO 23894 §6.4.2 requires that risk identification be continuous: risks are not declared once and frozen — the register grows as the system evolves. The risk program (the AssuranceProgram, today a section — risk: — of sei.yaml itself) is that living register. The KAG (Knowledge Assessment Graph) is the component that detects candidate risks not yet declared in it. sei assess runs the KAG against the current manifest and emits, in an advisory and non-blocking manner, the risks the KAG proposes but the team has not yet registered.

What sei assess does

Section titled “What sei assess does”

sei assess reads the project’s sei.yaml (intended purpose, modality, task type), passes that information to the KAG, and crosses the proposed risks against the ids already declared in the risk: section of sei.yaml. New risks — those the KAG proposes but that do not appear in the risk program — form the growth backlog of the register:

Run the KAG
sei assess --repo .

When there are new risks, the output lists each one with its identifier and title:

sei assess: the KAG proposes 1 undeclared risk(s) in the AssuranceProgram (growth backlog):
  + risk.age-discrimination — «Age-based Credit Discrimination»
Curate them: add them to the AssuranceProgram (with impact/likelihood/provenance) and commit — the commit is the identification act (ISO 6.4.2).

When the register is up to date:

sei assess: the KAG proposes no new risks — the risk register is current.

The exit code is always 0 (advisory). A KAG-proposed risk does not block sei run or sei compile; its purpose is to flag the gap so the team can decide whether to curate it or document why it does not apply.

Living-register flow

Section titled “Living-register flow”

The full register growth cycle, in accordance with ISO 23894 §6.4.2, has four steps:

  1. Detection — sei assess lists the growth backlog.
  2. Human curation — the team evaluates each proposed risk: adds it to the risk: section of sei.yaml with impact, likelihood, treat, and provenance: proposed_by_kag, or decides it does not apply and documents that decision.
  3. Commit — the commit that adds the risk to the risk program is itself the dated and attributable identification act. sei reconstruct retrieves it later as “①Identification”.
  4. Recompilation — sei compile regenerates assessment_plan.oscal.yaml with the new risk. The engine re-measures on the next sei run.

The provenance proposed_by_kag is stored in the sei.yaml risk program and travels to the signed evidence bundle, where sei reconstruct displays it distinguishing it from risks declared directly by the team.

Example: emergent age-discrimination risk

Section titled “Example: emergent age-discrimination risk”

The loan_growth scenario illustrates this cycle. The initial risk program (the risk: section of sei.yaml) declares five risks (unfair credit exclusion, data governance, opacity, model robustness/security, and insufficient human oversight). Mid-development, sei assess proposes an additional emergent risk, risk.age-discrimination, that the initial register did not cover:

T0 — before curating the register
sei assess --repo .
# → proposes risk.age-discrimination (not yet declared)

The team curates the risk in the risk: section of sei.yaml, under the risks: list:

sei.yaml (excerpt — emergent risk under risk.risks)
risk:
  # … appetite, criteria, overall_residual_criterion, previous risks …
  risks:
    - id: risk.age-discrimination
      title: "Age-based Credit Discrimination"
      affects: [sys.creditscore]
      sources: [ctx.historical-bias]
      provenance: proposed_by_kag          # identified by the KAG, curated by the team
      impact: { individual: HIGH, society: MEDIUM }
      likelihood: POSSIBLE                 # ISO 6.4.3 analysis — POSSIBLE×HIGH = HIGH
      treat:
        - method: REDUCE
          action: "Constrain demographic parity of the decision also by age group."
          controls: [eu/ai-act@2024#art-15]
          residual_likelihood: UNLIKELY
          measures:
            - id: age-discrimination
              metric: demographic_parity_diff
              constraint: "< 0.03"
              severity: high
              enforcement: gate
              lifecycle: [validation]
              article: "15"
              inputs: { prediction: prediction, dimension: age_bucket,
                        age_bucket_method: quantiles, age_buckets: 3 }

After curating and committing, recompile and re-measure:

T2 — after incorporating the emergent risk
sei compile --repo .
sei run --repo .
# → the age-discrimination control FAILS (age DP ≈ 0.14 ≫ 0.03)
# → gate RED: mitigating gender alone left the age dimension unfair

This result is the ISO iteration: the register grew, the engine detected the empirical gap, and the team must apply a new treatment (V3, which mitigates gender and age simultaneously). At the end of the cycle, sei reconstruct shows:

  ① Identification   <sha> <date>  «GROWTH: +risk.age-discrimination»
                     · identified by: KAG (proposed, curated by human)

The git history of sei.yaml (the risk program) is the identification audit log, dated and attributable, in accordance with ISO 23894 §6.4.2.

For the complete step-by-step flow, including dvc exp exploration and V3 treatment, see the tutorial First system with Venturalítica.

References

Section titled “References”