diff --git a/.github/workflows/swift.yml b/.github/workflows/swift.yml index 6997d7f..22ec4a3 100644 --- a/.github/workflows/swift.yml +++ b/.github/workflows/swift.yml @@ -22,4 +22,7 @@ jobs: .build/debug/hypercode emit Examples/service.hc --hcs Examples/service.hcs --format json > /tmp/ir-dev.json .build/debug/hypercode emit Examples/service.hc --hcs Examples/service.hcs --ctx env=production --format json > /tmp/ir-prod.json - name: Validate IR v2 against schema - run: npm exec --yes --package=ajv-cli@5 -- ajv validate --spec=draft2020 -s Schema/hypercode-ir-v2.schema.json -d /tmp/ir-dev.json -d /tmp/ir-prod.json \ No newline at end of file + run: npm exec --yes --package=ajv-cli@5 -- ajv validate --spec=draft2020 -s Schema/hypercode-ir-v2.schema.json -d /tmp/ir-dev.json -d /tmp/ir-prod.json + + - name: Codegen demo — generated artifacts fresh & contract-conformant + run: python3 Examples/codegen-demo/check.py \ No newline at end of file diff --git a/.gitignore b/.gitignore index 1dfdc4e..97463ec 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,5 @@ .DS_Store *.xcodeproj site/ +__pycache__/ +*.pyc diff --git a/DOCS/Usage.md b/DOCS/Usage.md index 0f3e52f..a0676e8 100644 --- a/DOCS/Usage.md +++ b/DOCS/Usage.md @@ -198,8 +198,10 @@ specification, code is regenerated output. - Humans review the *specification diff*; machines expand it into code (review compression). -The missing piece for this loop is `hypercode diff ` — -HC-113 in the [work plan](../workplan.md). +This loop is runnable today: [`Examples/codegen-demo/`](../Examples/codegen-demo/) +generates a module per node, verifies freshness by node hash and contract +conformance of the generated values in CI. `hypercode diff` (HC-113 in the +[work plan](../workplan.md)) will productize the hash comparison. ## Scalar typing cheat-sheet diff --git a/Examples/codegen-demo/README.md b/Examples/codegen-demo/README.md new file mode 100644 index 0000000..5ada907 --- /dev/null +++ b/Examples/codegen-demo/README.md @@ -0,0 +1,114 @@ +# HC-124 — End-to-end AI codegen demo + +The runnable version of the pipeline from [RFC §9.7](../../RFC/Hypercode.md) +and [DOCS/Usage.md §5](../../DOCS/Usage.md): the spec is the durable artifact, +code is regenerated output, and the resolved graph sits between them. + +``` +Examples/service.{hc,hcs} ──▶ IR v2 ──▶ generated/*.py (one module per node) + │ │ │ + 1-line edit node hashes check.py validates artifacts + (review this!) (what changed?) against the same contracts +``` + +Every module in [`generated/`](generated/) was produced by Claude from the +production-context IR of [`Examples/service.{hc,hcs}`](../service.hc). Each +embeds the **hash of its source node** and every value carries its +**provenance** as a comment: + +```python +# node: /Service/APIServer +# hash: 7da0acd4b1617ed3… +CONFIG = { + "port": 8080, # APIServer > Listen @ Examples/service.hcs:26 +} +``` + +## 1. Verify: artifacts match the spec + +```console +$ python3 check.py + FRESH api_server.py /Service/APIServer + FRESH database.py /Service/Database#main-db + FRESH logger.py /Service/Logger.console + FRESH service.py /Service + +all modules fresh and contract-conformant +``` + +`check.py` re-emits the IR and does two things: compares each module's +embedded node hash against the current one (**freshness**), and validates the +values embedded in the generated code against the `contracts[]` echoed in the +IR (**conformance**): type and bounds for present keys, presence for required +contracted keys, and no CONFIG key outside the resolved spec. Node paths use +selector identity (`type[.class][#id]`) — the same addressing as +`hypercode diff`. CI runs this on every push. + +## 2. Scoped regeneration: a one-line spec edit + +```console +$ sed 's/port: 8080/port: 9090/' ../service.hcs > /tmp/edited.hcs +$ python3 check.py --hcs /tmp/edited.hcs + STALE api_server.py /Service/APIServer + FRESH database.py /Service/Database#main-db + FRESH logger.py /Service/Logger.console + STALE service.py /Service + +2 module(s) stale — regenerate with generate.sh +``` + +The port lives on the `Listen` node, so exactly `api_server.py` is stale — +plus the root wiring, because a node's hash is a Merkle hash over its subtree. +`logger.py` and `database.py` are untouched and **not** regenerated. This is +the review-compression loop: a human reviews the one-line spec diff; the +machine knows precisely which artifacts it invalidates. + +## 3. Guardrails on both sides of generation + +**Before** generation, the spec itself is gated — a bad edit never reaches +the generator: + +```console +$ hypercode validate bad.hc --hcs bad.hcs --ctx env=production +bad.hcs:1:1: error[HC2104]: contract violation for 'port': 99999 exceeds upper bound 65535.0 … +``` + +**After** generation, the artifacts are checked against the same contracts. +Hand-edit `generated/api_server.py` to `"port": 99999` and: + +```console +$ python3 check.py +contract violations in generated artifacts: + error[HC2104-gen] api_server.py: 'port' = 99999 exceeds 65535.0 (contract 'APIServer > Listen') +$ echo $? +2 +``` + +The contract written once in the `.hcs` governs the spec, the resolved graph, +and the generated code. + +## 4. Regenerate with Claude + +```console +$ ./generate.sh # needs the `claude` CLI +regenerating api_server.py from node /Service/APIServer … +``` + +`check.py --list-stale` scopes the work; `generate.sh` feeds Claude the IR +subtree of each stale node (the source of truth) plus the module conventions, +then re-runs the checks. No stale module — no LLM call. + +## 5. It actually runs + +```console +$ python3 generated/service.py +('0.0.0.0', 8080) +``` + +## Files + +| File | Role | +|---|---| +| `check.py` | freshness (node hashes) + contract conformance of artifacts | +| `generate.sh` | scoped regeneration of stale modules via `claude -p` | +| `generated/*.py` | the generated service — one module per `.hc` node | diff --git a/Examples/codegen-demo/check.py b/Examples/codegen-demo/check.py new file mode 100644 index 0000000..aa702d4 --- /dev/null +++ b/Examples/codegen-demo/check.py @@ -0,0 +1,204 @@ +#!/usr/bin/env python3 +"""HC-124 demo: verify generated modules against the Hypercode resolved graph. + +Two independent checks, stdlib only: + + 1. Freshness — each module embeds the hash of the IR node it was generated + from. Node hashes cover the stable resolved content (Merkle over the + subtree), so a hash mismatch means the spec changed underneath the module + and it must be regenerated. This is the invalidation signal that scopes + regeneration: only stale modules are rebuilt. + + 2. Contract conformance — values embedded in generated code are validated + against the contracts[] echoed in the IR: type and bounds for present + keys, presence for required contracted keys, and no CONFIG key may exist + outside the resolved spec. Even a hand-edited "generated" file that + smuggles in port: 99999 — or drops port entirely — is caught. + +Each module owns its node's subtree up to the boundary of the next generated +module (service.py owns /Service but not /Service/Logger.console, which +logger.py owns). Node paths use selector identity (type[.class][#id]) — the +same addressing as `hypercode diff`. + +Exit codes: 0 = fresh & conformant, 1 = stale modules, 2 = contract violation. +""" +import argparse +import ast +import json +import os +import re +import subprocess +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +REPO = os.path.abspath(os.path.join(HERE, "..", "..")) + + +def emit_ir(hc, hcs, ctx): + binary = os.environ.get( + "HYPERCODE_BIN", os.path.join(REPO, ".build", "debug", "hypercode")) + cmd = [binary, "emit", hc, "--hcs", hcs, "--format", "json"] + for pair in ctx: + cmd += ["--ctx", pair] + out = subprocess.run(cmd, capture_output=True, text=True, cwd=REPO) + if out.returncode != 0: + sys.exit(f"emit failed: {out.stderr.strip()}") + return json.loads(out.stdout) + + +def node_label(node): + """Selector identity, same addressing as `hypercode diff`.""" + label = node["type"] + if "class" in node: + label += f".{node['class']}" + if "id" in node: + label += f"#{node['id']}" + return label + + +def index_nodes(ir): + """path -> node, depth-first; paths are /-joined selector identities.""" + nodes = {} + + def walk(node, path): + path = f"{path}/{node_label(node)}" + if path in nodes: + sys.exit(f"ambiguous node path '{path}' — give same-type siblings" + " distinct classes or ids") + nodes[path] = node + for child in node["children"]: + walk(child, path) + + for root in ir["nodes"]: + walk(root, "") + return nodes + + +def owned_properties(node, path, claimed): + """Resolved properties of the subtree this module owns: its node's + subtree, stopping at children that are themselves generated modules.""" + props = {} + + def walk(n, p): + props.update(n["properties"]) + for child in n["children"]: + child_path = f"{p}/{node_label(child)}" + if child_path not in claimed: + walk(child, child_path) + + walk(node, path) + return props + + +def parse_module(path): + """Extract the embedded node path, hash and CONFIG dict.""" + text = open(path).read() + node = re.search(r"^# node: (.+)$", text, re.M) + digest = re.search(r"^# hash: ([0-9a-f]{64})$", text, re.M) + config = re.search(r"^CONFIG = (\{.*?\n\}|\{\})$", text, re.M | re.S) + if not (node and digest and config): + sys.exit(f"{os.path.basename(path)}: missing generated-module markers") + return node.group(1), digest.group(1), ast.literal_eval(config.group(1)) + + +TYPE_CHECK = { + "int": lambda v: isinstance(v, int) and not isinstance(v, bool), + "float": lambda v: isinstance(v, (int, float)) and not isinstance(v, bool), + "string": lambda v: isinstance(v, str), + "bool": lambda v: isinstance(v, bool), +} + + +def check_contracts(config, props, module): + """Validate embedded values against the contracts echoed in the IR.""" + violations = [] + # Drift: a CONFIG key the spec doesn't resolve is a hand-edit, not output. + for key in config: + if key not in props: + violations.append( + f"{module}: '{key}' is not a resolved property of this" + " module's nodes — not in the spec") + # Presence: a key under a required contract must be carried by CONFIG. + for key, prop in sorted(props.items()): + if key in config: + continue + for contract in prop.get("contracts", []): + if contract.get("required"): + violations.append( + f"{module}: required '{key}' missing from CONFIG" + f" (contract '{contract['selector']}')") + break + for key, value in config.items(): + for contract in props.get(key, {}).get("contracts", []): + sel = contract["selector"] + if not TYPE_CHECK[contract["type"]](value): + violations.append( + f"{module}: '{key}' = {value!r} is not {contract['type']}" + f" (contract '{sel}')") + continue + if isinstance(value, (int, float)) and not isinstance(value, bool): + if "min" in contract and value < contract["min"]: + violations.append( + f"{module}: '{key}' = {value} below {contract['min']}" + f" (contract '{sel}')") + if "max" in contract and value > contract["max"]: + violations.append( + f"{module}: '{key}' = {value} exceeds {contract['max']}" + f" (contract '{sel}')") + return violations + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--hc", default="Examples/service.hc") + parser.add_argument("--hcs", default="Examples/service.hcs") + parser.add_argument("--ctx", action="append", default=None, + help="key=value (default: env=production)") + parser.add_argument("--list-stale", action="store_true", + help="print stale module filenames only") + args = parser.parse_args() + + ir = emit_ir(args.hc, args.hcs, args.ctx or ["env=production"]) + nodes = index_nodes(ir) + + gen_dir = os.path.join(HERE, "generated") + modules = [ + (name, *parse_module(os.path.join(gen_dir, name))) + for name in sorted(os.listdir(gen_dir)) if name.endswith(".py") + ] + # Module boundaries: a node generated as its own module is not part of + # its parent module's owned subtree. + claimed = {node_path for _, node_path, _, _ in modules} + + stale, violations = [], [] + for name, node_path, embedded, config in modules: + node = nodes.get(node_path) + if node is None: + sys.exit(f"{name}: node '{node_path}' no longer exists in the IR") + fresh = node["hash"] == embedded + if not fresh: + stale.append(name) + owned = owned_properties(node, node_path, claimed - {node_path}) + violations += check_contracts(config, owned, name) + if not args.list_stale: + status = "FRESH" if fresh else "STALE" + print(f" {status} {name:16} {node_path}") + + if args.list_stale: + print("\n".join(stale)) + return 1 if stale else 0 + + if violations: + print("\ncontract violations in generated artifacts:") + for v in violations: + print(f" error[HC2104-gen] {v}") + return 2 + if stale: + print(f"\n{len(stale)} module(s) stale — regenerate with generate.sh") + return 1 + print("\nall modules fresh and contract-conformant") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/Examples/codegen-demo/generate.sh b/Examples/codegen-demo/generate.sh new file mode 100755 index 0000000..9ae0d4d --- /dev/null +++ b/Examples/codegen-demo/generate.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# HC-124 demo: regenerate stale modules from the resolved graph via Claude. +# +# check.py decides WHAT to regenerate (node-hash comparison scopes the work); +# this script asks Claude to regenerate ONLY those modules, feeding it the +# IR subtree of the module's node. Requires the `claude` CLI on PATH. +set -euo pipefail +cd "$(dirname "$0")" +REPO="$(cd ../.. && pwd)" +BIN="${HYPERCODE_BIN:-$REPO/.build/debug/hypercode}" +HC="${1:-$REPO/Examples/service.hc}" +HCS="${2:-$REPO/Examples/service.hcs}" +CTX="${CTX:-env=production}" + +IR="$(mktemp)" +trap 'rm -f "$IR"' EXIT +"$BIN" emit "$HC" --hcs "$HCS" --ctx "$CTX" --format json > "$IR" + +STALE="$(python3 check.py --hc "$HC" --hcs "$HCS" --ctx "$CTX" --list-stale || true)" +if [ -z "$STALE" ]; then + echo "all modules fresh — nothing to regenerate" + exit 0 +fi + +command -v claude >/dev/null || { + echo "stale modules:" "$STALE" + echo "claude CLI not found — install Claude Code to regenerate automatically" + exit 1 +} + +for MODULE in $STALE; do + NODE_PATH="$(grep -m1 '^# node: ' "generated/$MODULE" | cut -d' ' -f3)" + NODE_JSON="$(python3 - "$IR" "$NODE_PATH" <<'PY' +import json, sys +ir = json.load(open(sys.argv[1])) +def label(n): # selector identity, as in check.py + return n["type"] + ("." + n["class"] if "class" in n else "") \ + + ("#" + n["id"] if "id" in n else "") +def find(nodes, path): + for n in nodes: + p = path + "/" + label(n) + if p == sys.argv[2]: return n + if (r := find(n["children"], p)): return r +find_result = find(ir["nodes"], "") +print(json.dumps(find_result, indent=2)) +PY +)" + echo "regenerating $MODULE from node $NODE_PATH …" + claude -p "Regenerate the Python module below from this Hypercode IR v2 node. + +Conventions (must match the existing modules in this directory exactly): +- header comments: '# GENERATED…', '# node: $NODE_PATH', '# hash: ', '# context: $CTX' +- a CONFIG dict literal with one entry per resolved property used by the module, + each with a provenance comment '# @ :' +- keep the same class/function structure as the current file; only values, + hash and provenance comments change +- output ONLY the Python source, no markdown fences + +Current file: +$(cat "generated/$MODULE") + +IR node (the source of truth): +$NODE_JSON" > "generated/$MODULE.new" && mv "generated/$MODULE.new" "generated/$MODULE" +done + +python3 check.py --hc "$HC" --hcs "$HCS" --ctx "$CTX" diff --git a/Examples/codegen-demo/generated/api_server.py b/Examples/codegen-demo/generated/api_server.py new file mode 100644 index 0000000..3995b11 --- /dev/null +++ b/Examples/codegen-demo/generated/api_server.py @@ -0,0 +1,25 @@ +# GENERATED from the Hypercode resolved graph — do not edit by hand. +# Regenerate with ../generate.sh when check.py reports this module stale. +# node: /Service/APIServer +# hash: 7da0acd4b1617ed3a2e3ce7af9c3b5cbdd33e5d8ba7427bee2bfbb869da5101a +# context: env=production + +CONFIG = { + "host": "0.0.0.0", # APIServer > Listen @ Examples/service.hcs:26 + "port": 8080, # APIServer > Listen @ Examples/service.hcs:26 +} + + +class APIServer: + """API server with its Listen child node.""" + + def __init__(self, logger, database): + self.logger = logger + self.database = database + self.host = CONFIG["host"] + self.port = CONFIG["port"] + + def listen(self): + # child node: /Service/APIServer/Listen + self.logger.log("info", f"listening on {self.host}:{self.port}") + return (self.host, self.port) diff --git a/Examples/codegen-demo/generated/database.py b/Examples/codegen-demo/generated/database.py new file mode 100644 index 0000000..20248d7 --- /dev/null +++ b/Examples/codegen-demo/generated/database.py @@ -0,0 +1,25 @@ +# GENERATED from the Hypercode resolved graph — do not edit by hand. +# Regenerate with ../generate.sh when check.py reports this module stale. +# node: /Service/Database#main-db +# hash: baf7d11c1bf41439b03c0c82a62211465d47d43d84c769e0b702d1383e8c8243 +# context: env=production + +CONFIG = { + "driver": "postgres", # '#main-db' @ Examples/service.hcs:22 + "file": "dev.sqlite3", # Database @ Examples/service.hcs:7 (not overridden in production) + "pool_size": 50, # '#main-db' @ Examples/service.hcs:22 +} + + +class Database: + """Database #main-db with its Connect child node.""" + + def __init__(self): + self.driver = CONFIG["driver"] + self.pool_size = CONFIG["pool_size"] + self._pool = [] + + def connect(self): + # child node: /Service/Database#main-db/Connect + self._pool = [f"{self.driver}-conn-{i}" for i in range(self.pool_size)] + return len(self._pool) diff --git a/Examples/codegen-demo/generated/logger.py b/Examples/codegen-demo/generated/logger.py new file mode 100644 index 0000000..a4e1a48 --- /dev/null +++ b/Examples/codegen-demo/generated/logger.py @@ -0,0 +1,28 @@ +# GENERATED from the Hypercode resolved graph — do not edit by hand. +# Regenerate with ../generate.sh when check.py reports this module stale. +# node: /Service/Logger.console +# hash: 67f124e133a8b12970f02c154195f4ea9b176e137360ed6e37dbd88645fa4289 +# context: env=production + +CONFIG = { + "level": "info", # Logger @ Examples/service.hcs:16 + "format": "json", # .console @ Examples/service.hcs:19 +} + +LEVELS = ("debug", "info", "warning", "error") + + +class Logger: + """Console logger (class: console) for the Service node tree.""" + + def __init__(self): + self.level = CONFIG["level"] + self.format = CONFIG["format"] + + def log(self, level, message): + if LEVELS.index(level) < LEVELS.index(self.level): + return None + if self.format == "json": + import json + return json.dumps({"level": level, "message": message}) + return f"[{level}] {message}" diff --git a/Examples/codegen-demo/generated/service.py b/Examples/codegen-demo/generated/service.py new file mode 100644 index 0000000..7a298db --- /dev/null +++ b/Examples/codegen-demo/generated/service.py @@ -0,0 +1,29 @@ +# GENERATED from the Hypercode resolved graph — do not edit by hand. +# Regenerate with ../generate.sh when check.py reports this module stale. +# node: /Service +# hash: d476920df7d813dd3e422bc04ee8f2926624f2f21737bee37c3315e277fe96b1 +# context: env=production +# +# This is the wiring module: it instantiates children in .hc document order. +# Its node hash covers the whole subtree (Merkle), so any change anywhere in +# the resolved graph marks the wiring stale too. + +CONFIG = {} + +from logger import Logger +from database import Database +from api_server import APIServer + + +def build_service(): + """Wire the Service tree: Logger.console, Database#main-db, APIServer.""" + logger = Logger() + database = Database() + database.connect() + server = APIServer(logger, database) + return server + + +if __name__ == "__main__": + server = build_service() + print(server.listen()) diff --git a/workplan.md b/workplan.md index 8fe4e98..c7d2078 100644 --- a/workplan.md +++ b/workplan.md @@ -107,7 +107,7 @@ P1 — built on v2: - ⬜ HC-121 Dogfooding as the primary adoption path — Hyperprompt / Ontology consume the resolved IR (consumer-side dialects & backends per `DOCS/Dialects.md` / `DOCS/Backends.md`) - 🅿️ HC-122 SLSA-like generation attestation chain — signed `.hc`/`.hcs` → IR hash → generator identity/version → artifact hashes → validator report (RFC §8, §9.8) - 🅿️ HC-123 Agent Passport / 0AL integration — attestation chain plugs into 0AL's signed-agent model -- ⬜ HC-124 End-to-end AI codegen demo — one `.hc`/`.hcs` service spec → IR v2 → LLM generates code per node → generated artifacts validated against the same contracts; node hashes scope regeneration after a spec edit; the "review compression" story (RFC §9.7) made runnable +- [x] HC-124 End-to-end AI codegen demo — `Examples/codegen-demo/`: service spec → IR v2 → Claude-generated module per node (embedded node hash + provenance comments) → `check.py` validates artifacts against the same contracts (HC2104-gen) and scopes regeneration by node hash; `generate.sh` regenerates stale modules via `claude -p`; checked in CI on every push ## Cross-cutting - [x] HC-090 Swift CI workflow (build + test) — `.github/workflows/swift.yml`