Skip to content

fix(C01,C02,C03,C10,AppC): improve testability and remove ambiguous l…#848

Merged
RicoKomenda merged 1 commit into
OWASP:mainfrom
vtknightmare:vtknightmare/fix/testability-clarifications-and-appendix-c
Jun 2, 2026
Merged

fix(C01,C02,C03,C10,AppC): improve testability and remove ambiguous l…#848
RicoKomenda merged 1 commit into
OWASP:mainfrom
vtknightmare:vtknightmare/fix/testability-clarifications-and-appendix-c

Conversation

@vtknightmare
Copy link
Copy Markdown
Collaborator

…anguage

Preface rule: 'conformance can be objectively validated through testing, inspection, or audit.'

  • AppC AC.3.6: remove 'tokenized' (NLP tokenization is not the same as data-security tokenization; the word was technically incorrect)
  • AppC AC.4.3: define 'critical security finding' as CVSS >= 9.0 or organizational severity policy; without a threshold the control is not auditable
  • AppC AC.6.2: scope fine-tuning to orgs controlling model training infra; SaaS tool users cannot implement fine-tuning
  • C01 1.2.1: replace 'assessed and addressed' with minimum documented deliverable (impact assessment + chosen mitigation)
  • C01 1.4.5: add 'documented set of adversarially relevant input subgroups' so evaluation scope is verifiable
  • C01 1.4.6: require chosen defense and tuning rationale to be recorded alongside model artifact; 'appropriate' was not auditable
  • C02 2.1.6: replace 'regional legal constraints' (200+ jurisdictions) with 'regional content-policy classification' (configurable attribute)
  • C02 2.3.1: remove 'illegal requests' (jurisdiction-dependent); replace with 'requests violating organizational content policy'
  • C03 3.2.5: add minimum test criteria (prompt injection rejection and tool output sanitization) so 'covers' is verifiable
  • C10 10.3.3: clarify Origin and Host are validated independently; was ambiguous about whether failing one header was sufficient

@RicoKomenda
Copy link
Copy Markdown
Collaborator

Same as the other PRs in this batch: the diff has 5 of the 10 changes listed (the Appendix C edits, C01 1.2.1, and C02 2.1.6 aren't here). Were those meant to be pushed?

The present ones are mostly good. C01 1.4.6, C03 3.2.5, and C10 10.3.3 are clear improvements. Two notes:

  • C02 2.3.1: dropping "illegal requests" makes sense, but the appended "Requests violating organizational content policy are also covered" is vague and overlaps 2.3.3, which already covers policy-violating inputs. Might be cleaner to drop the sentence.
  • C01 1.4.5: "retained per the audit log requirements" doesn't have a clear anchor in C01 (no general audit-log requirement there). Also worth keeping the old threat examples (mimicking trusted style/demographic), they made the intent concrete.

…uage

Preface rule: conformance must be objectively validatable via test/inspection/audit.

- AppC AC.3.6: drop 'tokenized' (NLP tokenization is not data-security
  tokenization; the term was incorrect here).
- AppC AC.4.3: define 'critical security finding' as CVSS >= 9.0 or the org
  severity policy, so the merge gate is auditable.
- AppC AC.6.2: scope fine-tuning to orgs that control training infra.
- C01 1.2.1: replace 'assessed and addressed' with a minimum documented
  deliverable (impact assessment + selected mitigation).
- C01 1.4.5 (review): keep the concrete adversarial examples (they make the
  intent clear), add testable timing (prior to deployment and after any
  significant model update), and drop the unanchored audit-log phrase.
- C01 1.4.6: require the chosen defense and tuning rationale to be recorded
  alongside the model artifact.
- C02 2.1.6: replace 'regional legal constraints' with an auditable attribute
  (regional content-policy classification).
- C02 2.3.1 (review): drop 'illegal requests' (jurisdiction-dependent) and do
  NOT append a vague policy clause — policy-violating inputs are already
  covered by 2.3.3.
- C03 3.2.5: add a minimum testable bar (prompt-injection rejection and tool
  output sanitization).
- C10 10.3.3: clarify Origin and Host are validated independently and either
  failing is sufficient to reject.
@vtknightmare vtknightmare force-pushed the vtknightmare/fix/testability-clarifications-and-appendix-c branch from 21d185e to eb4e65b Compare June 2, 2026 16:03
@vtknightmare
Copy link
Copy Markdown
Collaborator Author

Same as the other PRs in this batch: the diff has 5 of the 10 changes listed (the Appendix C edits, C01 1.2.1, and C02 2.1.6 aren't here). Were those meant to be pushed?

The present ones are mostly good. C01 1.4.6, C03 3.2.5, and C10 10.3.3 are clear improvements. Two notes:

* C02 2.3.1: dropping "illegal requests" makes sense, but the appended "Requests violating organizational content policy are also covered" is vague and overlaps 2.3.3, which already covers policy-violating inputs. Might be cleaner to drop the sentence.

* C01 1.4.5: "retained per the audit log requirements" doesn't have a clear anchor in C01 (no general audit-log requirement there). Also worth keeping the old threat examples (mimicking trusted style/demographic), they made the intent concrete.

Same partial-diff fix: AppC edits, C01 1.2.1, and C02 2.1.6 are now included.
On 2.3.1: dropped the appended sentence, it overlaps 2.3.3 as you said. Just removes "illegal requests":
Verify that every inbound prompt is scored by a content classifier for violence, self-harm, hate, and sexual content against configurable thresholds, and that prompts exceeding those thresholds are rejected or sanitized before reaching model context.
On 1.4.5: kept the threat examples, dropped the audit-log phrase (no anchor in C01), added testable timing:
Verify that models used in security-relevant decisions (e.g., abuse detection, fraud scoring, automated trust decisions) are evaluated, prior to deployment and after any significant model update, for systematic bias patterns that an adversary could exploit to evade controls (e.g., mimicking a trusted language style or demographic pattern to bypass detection).
Force-pushed.

@RicoKomenda
Copy link
Copy Markdown
Collaborator

All ten changes are in now, and both earlier points are addressed: 1.4.5 keeps the threat examples and drops the audit-log reference, and 2.3.1 drops the overlapping sentence. The new ones are solid too. AC.3.6 fixing the "tokenized" misnomer, AC.4.3's CVSS >= 9.0 threshold, and C02 2.1.6 moving off "regional legal constraints" are all good calls. No remaining concerns from me.

@RicoKomenda RicoKomenda merged commit 4d4007a into OWASP:main Jun 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants