Skip to content

development.md

Latest commit

 

History

History
291 lines (206 loc) · 21.1 KB

development.md

File metadata and controls

291 lines (206 loc) · 21.1 KB
Raw

Development Guide

This guide is for contributors and maintainers working in the AuroraFlow repository. The README is the project overview; this file covers local setup, verification, source layout, and the maturity boundaries to keep in mind while making changes.

Development stance

AuroraFlow documentation should describe implemented behavior first and roadmap behavior second. If a feature is not backed by source, tests, scripts, or workflow configuration, call it planned rather than current.

Important current boundaries:

  • Self-healing SAT is diagnostic and guarded; it does not blindly or autonomously update selectors.
  • SELF_HEAL_REGISTRY_MODE=read can load active selector records and candidate history when a registry runtime is configured.
  • SELF_HEAL_REGISTRY_MODE=write_pending can persist SAT history observations and reviewable pending promotion records.
  • Reviewed approve, reject, conflict, and rollback workflows mutate selector registry records only; they do not rewrite source files.
  • Redis is an optional durable selector-store backend owned by consumers/operators; prefixes are namespace hygiene, not authorization.
  • The observability stack is suitable for local development and CI smoke validation. Production deployment requires environment-owned credentials, TLS, storage, retention, network policy, and operations support.
  • The package build emits src/**/*.ts declarations and JavaScript into dist; the package also includes curated docs/ and schemas/.

Prerequisites

  • Node.js >=20 <25
  • npm
  • Docker, for Redis/Testcontainers integration tests and the local observability stack
  • Playwright browsers, for smoke, examples, and E2E tests
  • ShellCheck and actionlint, when running the same workflow linting locally as CI

Install dependencies and browser binaries:

npm ci
npx playwright install --with-deps

Source layout

Path Responsibility
src/index.ts Root package export surface.
src/playwright.ts auroraflow/playwright entrypoint: Playwright test/expect fixture extension and lifecycle re-exports.
src/pageObjects/pageObjectBase.ts Base page action wrapper, screenshots, self-healing failure path, and page action telemetry.
src/helpers/pageFactory.ts Page object instance cache.
src/framework/selfHealing/ Configuration, DOM snapshots, candidate extraction/scoring, failure artifacts, guarded validation, run-level budget, reviewed promotion governance, and artifact schemas.
src/data/selectors/selectorRegistry.ts Typed selector registry repository over a Redis-compatible store contract.
src/utils/redisClient.ts Redis runtime configuration, connection lifecycle, namespacing, retries, and key scanning.
src/framework/observability/ No-op/default telemetry facade, OpenTelemetry adapter, attribute builders, flakiness reports, SLO dashboards, and alert evaluation.
src/framework/runtime/ AuroraFlowContext dependency container, closeAuroraFlow lifecycle/disposer registry, and the Playwright fixture helper.
src/utils/logger.ts Structured logging configuration and redaction defaults.
tests/suites/unit/ Fast unit tests.
tests/suites/contracts/ Package, workflow, infrastructure, documentation, and schema-adjacent contracts.
tests/suites/integration/ Redis/Testcontainers and OTLP integration coverage.
tests/suites/e2e/examples/ Playwright-backed example and smoke scenarios.
.github/workflows/ Quality, examples, security, and E2E matrix workflows.
observability/ Local stack configuration and reference production manifests.

Common commands

Quality gates

npm run format:check
npm run lint
npm run typecheck
npm test
npm run test:unit
npm run test:contracts
npm run test:integration
npm run schemas:check
npm run build

Check manifest/lockfile consistency before Dependabot or dependency PRs:

npm run lockfile:check

If it fails, regenerate the lockfile with npm install --package-lock-only, commit package.json and package-lock.json together, then rerun the check.

Run the repository verification target:

npm run verify

npm test is the unit-only fast path. It uses Vitest's thread pool without per-file isolation and a 30s per-test timeout; it must not require browsers, Docker, Redis, or an OTLP collector.

npm run test:contracts validates package, workflow, infrastructure, and documentation contracts without Redis/Testcontainers or browser setup.

npm run test:integration is reserved for real integration coverage (Redis/Testcontainers and OTLP export). Use AURORAFLOW_REDIS_INTEGRATION_REQUIRED=true npm run test:integration when Redis evidence must block instead of skip. Set AURORAFLOW_REDIS_INTEGRATION_EXTERNAL=true with AURORAFLOW_REDIS_* connection settings to reuse an already-running Redis instead of starting the default Testcontainers Redis instance.

npm run verify runs repo-local actionlint bootstrap, formatting, linting, typechecking, unit tests, contracts, Redis/OTLP integration, ShellCheck, and workflow linting. Schema validation is deliberately outside verify; it runs once as a CI/release evidence gate (npm run schemas:check).

TypeScript keeps its strict baseline. The first phased type-aware ESLint slice rejects unnecessary type assertions; validate external values at their boundary instead of casting them to silence the checker. noUncheckedIndexedAccess and exactOptionalPropertyTypes remain deferred to dedicated boundary-hardening migrations rather than a broad drive-by change.

Contract assertion policy

Contract specs are semantic-first:

  • raw toContain/toMatch and bare boolean toBe(true)/toBe(false) stay banned in tests/suites/contracts/**.
  • Use parsed workflow/JSON/Compose models where practical instead of asserting raw YAML, Markdown, or JSON text.
  • Public compatibility or safety wording checks are allowed only when they protect user-facing contracts; route them through tests/helpers/contractAssertions.ts with explicit rationale.
  • tests/suites/contracts/workflows/test-taxonomy.contract.spec.ts enforces the matcher ban.
Command Cost tier Scope
npm test / npm run test:unit Fast local Unit tests only; thread pool without per-file isolation and 30s per-test timeout; no browser, Docker, Redis, or OTLP dependency.
npm run test:contracts Static/contract Package, workflow, infrastructure, and docs contracts.
npm run test:integration Real integration Redis/Testcontainers and OTLP export.
AURORAFLOW_REDIS_INTEGRATION_REQUIRED=true npm run test:integration Blocking real integration Same Redis/OTLP integration suite, but Redis startup/connect failures fail instead of skip.
npm run test:coverage Coverage One complete unit coverage run enforcing the global src/** floor and every risk-weighted per-file floor.
npm run test:e2e Browser Playwright browser projects.
npm run test:e2e:guarded Guarded browser proof Parallel Chrome proof for guarded self-heal at the default gate.
npm run verify Full local gate Static checks, unit, contracts, integration, ShellCheck, and workflow lint. Schema validation is a CI/release evidence gate (npm run schemas:check), run once outside verify.
npm run test:mutation / test:mutation:check Scheduled/manual advisory Scoped mutation baseline for calibration-critical code; killed mutants that survive or become inapplicable fail the evidence lane, but it remains outside verify. See mutation & property baseline.
npm run benchmark:failure-path Manual/browser Warning-only safe-action failure, DOM snapshot, SAT extraction, and artifact-write timing; not part of verify. See failure-path performance baseline.

Risk-weighted coverage floors

npm run test:coverage enforces per-file floors for high-risk modules in addition to the global erosion guard. Floors live in configs/vitest.coverage-global.mts and sit just below measured coverage, so a failure means real erosion, not noise. Covered surfaces include the OpenTelemetry adapter, the Redis client and selector store, and the promotion workflow.

src/framework/selfHealing/domSnapshot.ts carries a documented exemption: ~90% of it runs inside page.evaluate (browser context), which node coverage cannot reach and which the guarded Chrome proof exercises instead. Its node-reachable privacy/normalization helpers are unit-tested; the line/statement floor guards that surface only. Structured-candidate coverage lives in the node-side candidate locator, DOM extraction, and guarded-validation suites.

Mutation & property baseline

Calibration-critical code (scoring, config, guarded validation, retry, Redis CAS) has a scoped mutation/property baseline that measures assertion catch-rate, not just execution. Property tests run inside npm run test:unit with fixed seeds and are reproducible from the seed; Mutation Baseline (Advisory) runs npm run test:mutation:check on scheduled/manual Quality Gates evidence lanes and fails when a previously killed mutant survives or becomes inapplicable. It remains outside verify and branch protection. No new test dependency is used. See mutation & property baseline.

Failure-path performance baseline

npm run benchmark:failure-path measures bounded DOM capture, SAT candidate extraction, representative artifact writes, and aggregate post-error safe-action diagnostics on a fixed, network-free Chrome fixture. The default command compares medians with the committed baseline but reports trends without failing; refresh the committed observation only with npm run benchmark:failure-path:record. No latency threshold participates in verify or required CI before maintainer approval. See failure-path performance baseline.

CI gate topology

Branch protection depends on two stable aggregate gates, Quality Gate and Security Gate, not on individual matrix-leg or internal job names. Each aggregate runs with if: always(), verifies the expected success-or-skipped result of every upstream lane for the current event, fails closed on a missing/cancelled/unexpectedly-skipped lane, and publishes a result table to the job summary.

Gate Cost tier Runs Responsibility
Lockfile Drift Fast dependency check Manifest/dependabot changes, Dependabot PRs, main, scheduled, and manual quality workflow runs Fails early with clear remediation when package-lock.json is not synchronized with package.json; the one explicit lockfile gate (the shared setup no longer dry-runs it before every npm ci).
Node Compatibility (Node 20/22/24) Fast matrix Pull requests, main, scheduled, and manual quality workflow runs npm ci and unit tests only — runtime-sensitive behavior across supported Node versions; no lint/typecheck/schema (those are runtime-insensitive and run once), no Docker, Redis, OTLP collector, or browser install.
Static Analysis (Node 22) Static + Docker integration Pull requests, main, scheduled, and manual quality workflow runs Format, lint, typecheck, contracts, Redis/OTLP integration with AURORAFLOW_REDIS_INTEGRATION_REQUIRED=true, schemas, ShellCheck, and workflow lint — the canonical runtime-insensitive gates, run once. actionlint is installed once and reused.
Coverage (Unit Floors) Coverage Pull requests, main, scheduled, and manual quality workflow runs One complete unit coverage run enforcing the global src/** floor and risk-weighted per-file floors for high-risk modules once on Node 22. The former critical-only coverage pass was a strict subset with identical floors and was removed.
E2E (Chrome) Browser heavy Browser-relevant changes, main, scheduled/manual runs, or full-e2e/risk:e2e PR labels; skipped (no runner) otherwise Runs the full Chrome project once — the union of the former smoke, guarded self-heal, examples, and risk subsets, so no Playwright test ID runs twice on the same event — then evaluates self-healing governance on that run's artifacts.
Observability smoke jobs Docker optional Observability Lite on observability-relevant changes plus a daily floor; Observability Full Stack weekly/manual only Keeps collector-only Lite receipt evidence separate from the weekly full seven-service reference-stack validation; neither is a routine gate on unrelated pushes.

Browser suites

npm run test:smoke
npm run test:examples
npm run test:e2e

The exhaustive E2E Matrix workflow runs one job per installed browser binary (desktop and mobile WebKit share a single webkit install in one job) on a daily schedule and manual dispatch — no per-run sharding, because Playwright is already fully parallel and browser/dependency setup, not test time, dominates a job. Post-merge Chrome coverage comes once from the Quality Gates E2E (Chrome) lane rather than the full matrix on every push; run E2E Matrix manually after a risky merge if cross-browser confirmation is needed before the next daily run. The E2E (Chrome) lane is path-filtered: it runs the full Chrome project on browser-relevant changes and is skipped (no runner) otherwise.

If a constrained local machine times out during accessibility smoke checks, reproduce with a larger Playwright timeout before changing source:

npx playwright test --config=configs/playwright.config.ts \
  --project='Google Chrome' \
  --grep @smoke \
  --timeout=60000 \
  --workers=1

Package surface

npm run build
npm run pack:dry-run

tsconfig.build.json uses src as the root and excludes tests, examples, and scripts. Package contracts expect dist, docs, schemas, README.md, LICENSE, and the root playwright.* compatibility shims as the packaged files.

Redis development

Integration tests use Testcontainers and an ephemeral redis:7.2-alpine container when Docker is available. For iterative local debugging, start the repository Redis service:

npm run infra:redis:up
npm run test:integration
npm run infra:redis:logs
npm run infra:redis:down

Default local Redis behavior is skip-friendly: if Docker/Testcontainers cannot start Redis, the Redis integration spec reports an explicit skip so non-Redis contributors are not blocked. Required mode is blocking: AURORAFLOW_REDIS_INTEGRATION_REQUIRED=true npm run test:integration fails on Redis startup, connection, or evidence failures and is the CI/release mode used by Static Analysis (Node 22) and the release dry-run. External mode is opt-in: AURORAFLOW_REDIS_INTEGRATION_EXTERNAL=true AURORAFLOW_REDIS_HOST=127.0.0.1 AURORAFLOW_REDIS_PORT=6379 npm run test:integration reuses an existing Redis with a per-run key prefix.

Production Redis use is outside local development ownership: operators own TLS, auth, ACLs, backups, restore drills, no-eviction capacity planning, retention, and incident response. Keep the Redis production runbook aligned with selector-store behavior when changing Redis keys, commands, TTLs, or cleanup workflows.

Keep selector registry changes aligned with:

  • src/data/selectors/selectorRegistry.ts
  • src/utils/redisClient.ts
  • docs/architecture/data-layer.md
  • docs/operations/redis-production-runbook.md
  • tests/suites/integration/framework/data/redisIntegration.spec.ts

Self-healing development

Supported runtime modes:

  • SELF_HEAL_MODE=off
  • SELF_HEAL_MODE=suggest
  • SELF_HEAL_MODE=guarded

Useful commands:

SELF_HEAL_MODE=suggest npm run test:smoke
SELF_HEAL_MODE=guarded npm run test:smoke
npm run self-heal:governance
npm run self-heal:repair

Artifacts are written to SELF_HEAL_ARTIFACTS_DIR when set, otherwise test-results/self-healing/*.json; governance summaries are written to test-results/self-healing-governance-summary.{json,md}. Promotion, cleanup, and repair CLIs default to Redis. Local promotion authorization is permissive with a warning; shared mode must provide CODEOWNERS plus protected-workflow evidence. Registry cleanup and repair are dry-run by default. Repair reports selector schema/index drift and requires --apply or SELF_HEAL_REGISTRY_REPAIR_APPLY=true before mutation. Use AURORAFLOW_SELF_HEALING_SCRIPT_STORE=memory only for deterministic local/process-boundary checks where non-durable state is expected.

When extending this area, keep these contracts intact:

  • DOM capture must stay bounded and redacted.
  • Guarded validation must remain policy-gated.
  • Auto-apply must not suppress a failed retry.
  • Promotion or registry writes need explicit tests, documentation, and review controls before being claimed as current behavior.

Observability development

Telemetry is disabled by default. Enable it explicitly:

npm run observability:up

AURORAFLOW_OBSERVABILITY_ENABLED=true \
AURORAFLOW_OBSERVABILITY_ENVIRONMENT=local \
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 \
npm run observability:smoke

npm run observability:snapshot
npm run observability:down

Keep observability changes aligned with:

  • docs/operations/observability-contract.md
  • docs/architecture/observability-stack.md
  • observability/README.md
  • tests/suites/contracts/observability/observabilityContract.spec.ts

The JSON/Markdown flakiness, SLO, and alert artifacts remain the deterministic merge-gate source. Local dashboards and alert rules are useful development assets, but production use should go through the dashboard review checklist and emitted-label validation.

Workflow and security checks

CI's Static Analysis (Node 22) lane installs native actionlint and shellcheck, then runs the same checks npm run verify bundles as individual steps; it does not invoke npm run verify itself. The manual release dry run is the only workflow that runs npm run verify verbatim. For the closest local match, install actionlint and shellcheck before running:

npm run workflows:lint
npm run shellcheck
npm run security:audit
npm run security:workflows
npm run security:all

npm run security:audit owns high-severity npm audit only. npm run security:workflows owns zizmor workflow analysis only. npm run security:all combines both local security checks; npm run security:check remains a compatibility alias.

Security workflows cover dependency review on pull requests, high-severity npm audit on non-PR events, CodeQL on non-PR events, gitleaks, a synthetic gitleaks failure proof, workflow security scanning, and a final security gate.

Release process

Releases are governed by a manual, dry-run-only workflow (.github/workflows/release.yml) that produces auditable evidence — pack report, SPDX/CycloneDX SBOMs, provenance-readiness check, and a changelog draft — without publishing. Publishing is intentionally disabled behind a protected-environment placeholder. See release-process.md for the changelog, rollback, provenance, and SBOM policy (AUR-DEC-012).

Contribution governance

Use ../CONTRIBUTING.md as the lightweight contributor entry point. It links the validation matrix, safety guardrails, advisory CODEOWNERS policy, and initial ADR set.

Ownership is advisory by default: .github/CODEOWNERS routes review to the current maintainer handle, but it becomes enforceable only if branch protection requires code-owner review. Confirm or replace owner handles with the maintainer before enabling that enforcement.

Architecture decision records live in adr/. Add or supersede an ADR when a change affects safety-first self-healing, API compatibility tiers, scoring/SLO policy, Redis ownership, observability support boundaries, release policy, adoption-gated backend/CI extensibility, or another durable architecture decision.

Documentation rules

Documentation should remain precise and source-backed:

  • Prefer "implemented", "available", or "enabled by" only for behavior present in source.
  • Prefer "planned", "reference", or "roadmap" for future services, deployments, or automation.
  • Lifecycle docs describe closeAuroraFlow(context?) and auroraflow/playwright as shipped in the lifecycle helper implementation; keep their idempotent-cleanup and consumer-owned-Playwright guarantees source-backed.
  • Link to the source-owning architecture or operations document when a README section would otherwise become too detailed.
  • Keep production observability wording clear: local assets and reference manifests are not environment ownership.

Troubleshooting

  • Redis integration skipped: confirm Docker is running. The integration suite should skip explicitly rather than hang when Testcontainers cannot start.
  • Workflow lint differs locally: install native actionlint; CI's Static Analysis lane installs it before running its workflow lint step.
  • No live telemetry: confirm AURORAFLOW_OBSERVABILITY_ENABLED=true and OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318.
  • Grafana has no metrics: check Prometheus targets at http://localhost:9090/targets and Collector health at http://localhost:13133.
  • Self-healing artifacts missing: confirm SELF_HEAL_MODE is suggest or guarded; off is the default.