test(core): harden the rule suite with realistic negative cases#18
Open
PunGrumpy wants to merge 5 commits into
Open
test(core): harden the rule suite with realistic negative cases#18PunGrumpy wants to merge 5 commits into
PunGrumpy wants to merge 5 commits into
Conversation
Extend the no-root-user, no-secrets-in-env, and no-add-remote rule tests with realistic compliant shapes (multi-stage final USER, ARG build-secret passthrough, multiple ENV vars, local archive ADD with --chown) and confirm genuine secret-shaped keys (AWS_SECRET_ACCESS_KEY, API_KEY) still trigger a diagnostic.
Extend no-version-key, require-resource-limits, require-restart-policy, and use-depends-on-condition with realistic multi-service compliant compose shapes (deploy.resources.limits, deploy.restart_policy, and long-form depends_on with a service_healthy condition).
Extend clean-package-cache with an apk-based realistic compliant shape (apk add --no-cache) to cover the non-apt package manager path. prefer-slim-base and avoid-dev-dependencies already carry realistic compliant coverage from plans 003/004.
Extend absolute-workdir, require-healthcheck, prefer-copy-over-add, use-exec-form, and require-labels with realistic compliant shapes (variable WORKDIR, multi-stage HEALTHCHECK, flagged archive ADD, ENTRYPOINT exec form, OCI label keys). Document the two known rule-specificity false positives as test.todo (not fixed): use-pipefail flags a quoted regex alternation as an unguarded pipe, and avoid-run-cd matches 'cd' inside unrelated path segments/words via an unbounded \bcd\b match against inst.args.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Hardens the rule suite with realistic negative cases across all 25 rules, so the next rule-specificity bug is caught by CI instead of by an audit. Test-only — no source changes, no changeset. Implements plan 006. Depends on 003 and 004.
Why
The suite was not assertion-free, but every negative case tested the trivially-compliant form (
USER node,FROM node:22.2.0-alpine,ENV NORMAL_VAR). Meanwhile every bug fixed in plans 003/004 lived in a realistic shape the suite never constructed: multi-stage builds,--chownflags, registry ports, digest pins, stage aliases, ARG-driven bases. This closes that gap systematically.What changed (all in
packages/core/test/rules.test.ts, purely additive — 0 deletions)use-multi-stageon a real 2-stage build,clean-package-cacheonapk add --no-cache,require-labelson OCI labels,use-dockerignoreonCOPY --chown . ., compose rules ondeploy-block configs,order-layerson manifest→install→source ordering.no-root-useron a multi-stage file whose final stage lacksUSER;no-secrets-in-envonAWS_SECRET_ACCESS_KEYandAPI_KEY(fake placeholder values);pin-image-versiononregistry.example.com:5000/team/app(untagged behind a port);no-add-remoteonADD --chown … https://….test.todo(not fixed — out of scope, no approved plan):use-pipefailflags a regex alternation inside a quoted argument (RUN grep -E "foo|bar" …), andavoid-run-cdmatchescdinside path segments/strings (/opt/cd,abcd). Both have the same root cause — matching raw text with no shell-quote awareness — and would likely be fixed together by a small tokenizer pre-pass. Mirrored inplans/README.md's deferred list.Two
${VAR}negatives (ENV API_KEY=${API_KEY}→ silent,WORKDIR ${APP_HOME}→ silent) assert intended behavior — both rules have explicit variable-skipping guards in source (!value.startsWith("$")and\$in the absolute-path regex), so these document the contract rather than cementing a false negative.Verification
ultracite check0.src/rules/: security.ts 99.5→100%, compose.ts 99.3→100%, performance.ts 91.6→93.7%.Stacking
Base is
advisor/004-shared-image-ref-parser(PR #16), which stacks on 003→002. Sibling of PR #17 (005). GitHub auto-retargets tomainonce #14→#16 merge. Merge after #14, #15, #16.🤖 Generated with Claude Code