Skip to content

fix(C01,C03,C05,C11,C12,AppC): correct requirement level assignments#846

Open
vtknightmare wants to merge 1 commit into
OWASP:mainfrom
vtknightmare:vtknightmare/fix/level-calibration-corrections
Open

fix(C01,C03,C05,C11,C12,AppC): correct requirement level assignments#846
vtknightmare wants to merge 1 commit into
OWASP:mainfrom
vtknightmare:vtknightmare/fix/level-calibration-corrections

Conversation

@vtknightmare
Copy link
Copy Markdown
Collaborator

Level calibration against L1=baseline / L2=standard / L3=high-assurance.

  • C01 1.2.4: L3 -> L2. Dataset versioning (DVC, MLflow, Git-LFS) is standard MLOps practice, not high-assurance.
  • C03 3.2.4: L1 -> L2. Automated deployment blocking on safety evaluation failure requires CI/CD pipeline infrastructure not assumed at baseline.
  • C05 5.6.1: L2 -> L3. KV-cache partitioning requires access to inference framework internals most operators do not have.
  • C11 11.4.2: L1 -> L2. Calibrating rate limits to a specific inversion threat model requires threat modeling work not present at baseline.
  • C12 12.5.1: L2 -> L3. Validating consent scope covering data subjects whose data materially influenced a response requires determining training data influence per inference, an open problem.
  • AppC AC.9.1: L1 -> L2. Signed SLSA/in-toto provenance attestations require a mature MLOps provenance pipeline, not present at baseline.
  • AppC AC.9.2: L2 -> L3. Restores level progression after AC.9.1 move.
  • AppC AC.5.2: L2 -> L3, plus conditioner. Most commercial AI coding tools do not surface training citations.

@vtknightmare vtknightmare requested a review from ottosulin June 2, 2026 13:55
@RicoKomenda
Copy link
Copy Markdown
Collaborator

The description lists 8 level changes, but the diff only has 3 (C03 3.2.4, C05 5.6.1, C11 11.4.2). The C01, C12, and Appendix C changes aren't here. Did you mean to push more commits, or should the description be trimmed to match?

@vtknightmare vtknightmare force-pushed the vtknightmare/fix/level-calibration-corrections branch from 908f0a6 to 24f9db9 Compare June 2, 2026 16:03
@vtknightmare
Copy link
Copy Markdown
Collaborator Author

The description lists 8 level changes, but the diff only has 3 (C03 3.2.4, C05 5.6.1, C11 11.4.2). The C01, C12, and Appendix C changes aren't here. Did you mean to push more commits, or should the description be trimmed to match?

Tooling bug on my end. The script applied edits to the working tree before branching, and shared files across PRs meant some ended up on the wrong branch. Rebuilt from a clean main; all 8 level changes (C01 1.2.4, C03 3.2.4, C05 5.6.1, C11 11.4.2, C12 12.5.1, AppC AC.9.1/9.2/5.2) are now in the diff. The level decisions themselves are unchanged.
Force-pushed.

@RicoKomenda
Copy link
Copy Markdown
Collaborator

Thanks, the full set is here now. C01 1.2.4, C12 12.5.1, C05 5.6.1, C11 11.4.2, and AppC AC.9.1 all look well-calibrated.

One pushback and one note:

  • C03 3.2.4 (L1->L2): this splits oddly from 3.2.3. If safety evals with pass/fail thresholds run at L1, but blocking deploy on failure is only required at L2, then L1 evaluates without acting on the result. I'd keep 3.2.4 at L1 alongside 3.2.3.
  • AppC AC.5.2: the L3 move is fine, but "where the AI tool supports citation retrieval" makes the control vacuous when the tool doesn't support it. Worth reconsidering the conditioner.

@ottosulin
Copy link
Copy Markdown
Collaborator

LGTM, with the exception that I agree with Rico on C03 3.2.4 change should be reverted.

@vtknightmare vtknightmare force-pushed the vtknightmare/fix/level-calibration-corrections branch from 24f9db9 to 927e48f Compare June 2, 2026 19:44
@vtknightmare
Copy link
Copy Markdown
Collaborator Author

LGTM, with the exception that I agree with Rico on C03 3.2.4 change should be reverted.

Agreed on both. 3.2.4 is back at L1 - you're right that splitting the eval run and the blocking gate across levels means baseline deployments evaluate without acting on the result, which defeats the point. On AC.5.2: conditioner removed, L3 move stays. At L3 the expectation is that orgs select tools that actually support citation surfacing; the conditioner was an unnecessary out.

Level calibration (L1 baseline / L2 standard / L3 high-assurance).

- C01 1.2.4: L3 -> L2 (dataset versioning is standard MLOps).
- C03 3.2.4: kept at L1 (review feedback: splitting the eval run (3.2.3) and
  the blocking gate (3.2.4) across L1/L2 means baseline deployments run evals
  without acting on the result; both controls belong at L1).
- C05 5.6.1: L2 -> L3 (KV-cache partitioning requires inference-framework
  internals not available at L2).
- C11 11.4.2: L1 -> L2 (threat-model calibration is not a baseline activity).
- C12 12.5.1: L2 -> L3 (per-inference consent-scope over training influence
  is an open research problem).
- AppC AC.9.1: L1 -> L2 (signed SLSA/in-toto provenance needs a mature MLOps
  pipeline).
- AppC AC.9.2: L2 -> L3 (maintains progression after the AC.9.1 move).
- AppC AC.5.2: L2 -> L3, conditioner removed (review feedback: 'where the AI
  tool supports citation retrieval' makes the control vacuous at L3; orgs at
  that level are expected to select tools that support this capability).
@vtknightmare vtknightmare force-pushed the vtknightmare/fix/level-calibration-corrections branch from 927e48f to 3a2b378 Compare June 2, 2026 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants