Skip to content

Follow-up to #13: tailoring-schema and skill-emit gaps remain after v1.2.0 #18

@bburda

Description

@bburda

Summary

#13 was closed by the v1.2.0 release (PR #16 → merge → tag), but several sections of #13's roadmap did not land on main, and a v1.2 retest of the four user-facing slash skills (/pharaoh:pharaoh-mece, /pharaoh:pharaoh-req-from-code, /pharaoh:pharaoh-vplan-draft, /pharaoh:pharaoh-change) surfaced three new behaviours that were not in #13's original scope. Filing as a single follow-up rather than splitting into per-finding bugs because they are all the same shape: skills emit or wait in ways that diverge from the project's tailoring or from a non-interactive caller's expectations.

#13 sections that did not land

Re-checked against useblocks/pharaoh@v1.2.0:

Section 1 — canonical schemas at install path

examples/score/.pharaoh/project/schemas/ ships four canonical schemas. Only diagram-conventions.schema.json was promoted to skills/shared/. The four tailoring schemas (artefact-catalog, workflows, id-conventions, checklists-frontmatter) still sit only in the score example, are not referenced by shared/tailoring-access.md, and pharaoh-tailor-review does not load them at runtime.

Section 9a — pharaoh-arch-draft hardcoded type allow-list

pharaoh-arch-draft/SKILL.md still ships the hard FAIL: "Accepted values: module, component, interface. If the user supplies a different value, FAIL". Projects with swarch, sys-arch, or other architecture types remain unable to use the skill.

Section 9c — safety V drafting skills

No pharaoh-hazard-draft, pharaoh-safety_goal-draft, or pharaoh-fsr-draft skill exists in skills/. ISO 26262 safety-V corpora cannot be reverse-engineered or drafted by atomics; only reviewed via generic pharaoh-req-review.

New behaviours surfaced in v1.2 retest

Run via claude -p --model sonnet --permission-mode bypassPermissions against a sphinx-needs project whose .pharaoh/project/ tailoring conforms to the canonical schemas shipped in examples/score/.pharaoh/project/schemas/.

pharaoh-req-from-code emits Pharaoh-internal placeholders regardless of project declarations

Test prompt:

/pharaoh:pharaoh-req-from-code

src/automotive_adas.py — focused API contract for the LaneDetection
class as a child of REQ_001. Emit a single `req` directive. Use only
fields declared in ubproject.toml; do not emit :source_doc: or
:verification: unless the project's catalogue declares them.

Output (158 s, 11 turns):

.. req:: Lane Detection Camera Input Contract
   :id: REQ_100
   :status: draft
   :links: REQ_001
   :source_doc: src/automotive_adas.py
   :verification: tc__TBD

   The Lane Detection component shall accept a camera feed and return
   lane position data that identifies lateral deviation from the current
   lane boundary.

Neither source_doc nor verification is declared in the project's [needs.fields.X]. Pasting this RST under sphinx-build -W warns on undeclared fields and fails the build. Section 8 of #13 (which landed) reads [needs.fields.X] in pharaoh-setup to populate the catalog's optional_fields. Section 8 was not extended to the emit path of pharaoh-req-from-code, so the skill ignores the project's declarations on output.

pharaoh-change workflow gate stops at acknowledgement

Test prompt:

/pharaoh:pharaoh-change

REQ_001 is being revised: tighter lateral tolerance, ECU port.
Scope to the lane-detection domain.

Output (255 s, 28 turns, 184 chars):

Session state written with `acknowledged: false` for REQ_001.

Acknowledge this change analysis? Acknowledging allows proceeding to
the authoring skill for the affected needs.

The change document is not emitted until the user types an acknowledgement turn. Adding Emit the full change-impact report directly; do not stop at an acknowledgement prompt. to the prompt body bypasses the gate and produces the expected 10 K-character impact report.

Workflow-gate before output is a regression for non-interactive callers (claude -p, batch scripts, CI). Either the gate should be skill-internal (write session-state, return the report in the same turn) or it should be opt-in via a flag rather than the default.

pharaoh-mece returns brief summary instead of full report

Test prompt:

/pharaoh:pharaoh-mece

Output (665 s, 38 turns, 775 chars):

Session state updated. global.mece_checked = true...

Bottom line: The project's traceability graph is structurally sound...
The 102 ID format warnings all stem from a single root cause...

Compared to the same skill against the same corpus on pharaoh@a6d11ae (pre v1.2): an 11 K-character structured report with per-category tables, concrete IDs, and severity grouping. The v1.2 output is a one-paragraph executive summary. The structural findings still surface, but the loss of detail makes the skill less useful as a release-gate input or as a standalone audit.

Same workflow-gate root cause — the skill performs the analysis internally and returns only the gate confirmation. Adding Emit the full MECE report directly to the prompt body recovers the v1.0-style output.

Suggested approach

Two distinct fix bundles:

  1. Re-open the three sphinx-needs-demo feedback: setup, tailoring, and atomic-skill alignment gaps #13 sections that did not land (Section 1, 9a, 9c). The PR description claimed them as in-scope but the merged commits did not deliver. Either the PR body should have been amended before merge, or these belong in a new milestone.
  2. Treat the workflow-gate-before-output pattern as a regression in pharaoh-mece, pharaoh-change, and any other skill that adopted the same shape. Non-interactive callers cannot reach the output past the gate without an explicit emit directly nudge in the prompt. The default should be: write session state and emit the artefact in the same turn; the acknowledgement gate is opt-in, not opt-out.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions