Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 17 additions & 15 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,28 +40,30 @@ jobs:
- name: Validate schemas
run: |
jq empty schema/*.json internal/testfixtures/oss-adapter-contract/*.json
jq empty harness/contracts/orient-bundle.schema.json
jq empty harness/contracts/portolan-bundle.schema.json
jq empty harness/contracts/landscape-card.schema.json
jq empty harness/contracts/landscape-report.schema.json

- name: Harness orient smoke
run: scripts/harness-orient-smoke.sh
- name: Harness portolan smoke
run: scripts/harness-portolan-smoke.sh

- name: Orient wizard orchestration smoke
- name: Portolan scan orchestration smoke
run: |
scripts/orient-wizard.sh \
internal/testfixtures/orient-bundle/target \
/tmp/wizard-ci-orient \
scripts/portolan-scan.sh \
internal/testfixtures/portolan-bundle/target \
/tmp/wizard-ci-portolan \
--no-viewer --skip-install --yes
test -f /tmp/wizard-ci-orient/manifest.json
test -s /tmp/wizard-ci-orient/gaps.jsonl
jq -e 'select(.status == "not_assessed")' /tmp/wizard-ci-orient/gaps.jsonl >/dev/null
test -f /tmp/wizard-ci-portolan/manifest.json
test -s /tmp/wizard-ci-portolan/gaps.jsonl
jq -e 'select(.status == "not_assessed")' /tmp/wizard-ci-portolan/gaps.jsonl >/dev/null
jq -e 'select(.kind == "config" and .producer == "config-scan")' \
/tmp/wizard-ci-orient/hotspots.jsonl >/dev/null
if ! jq -e 'select(.kind == "debt-candidate")' /tmp/wizard-ci-orient/hotspots.jsonl >/dev/null 2>&1; then
/tmp/wizard-ci-portolan/hotspots.jsonl >/dev/null
if ! jq -e 'select(.kind == "debt-candidate")' /tmp/wizard-ci-portolan/hotspots.jsonl >/dev/null 2>&1; then
jq -e 'select(.id == "gap-ctags" or .surface == "symbols")' \
/tmp/wizard-ci-orient/gaps.jsonl >/dev/null
/tmp/wizard-ci-portolan/gaps.jsonl >/dev/null
fi
slug=$(printf '%s' "$(cd internal/testfixtures/orient-bundle/target && pwd)" | sha256sum | cut -c1-8)
test -n "$(find /tmp/wizard-ci-orient/producers/config -name "target-${slug}.jsonl" -print -quit)"
slug=$(printf '%s' "$(cd internal/testfixtures/portolan-bundle/target && pwd)" | sha256sum | cut -c1-8)
test -n "$(find /tmp/wizard-ci-portolan/producers/config -name "target-${slug}.jsonl" -print -quit)"

- name: Diff whitespace check
run: git diff --check
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
/bin/
/dist/
viewer/dist/
**/orient-smoke/
**/portolan-smoke/
.cursor/
/.portolan/
/coverage.out
Expand Down
10 changes: 5 additions & 5 deletions .specify/memory/constitution.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,18 +43,18 @@ be verified through link, schema, and placeholder checks instead of code tests.
## Product Constraints

- Primary delivery: harness artifacts (`harness/` skills, recipes, guardrails,
contracts) plus optional local orient viewer output.
contracts) plus optional local Portolan viewer output.
- Implementation language: Go remains for the legacy CLI and normalization
library; new product slices may use shell, TypeScript, or other local tooling
when a spec documents the boundary.
- Runtime default: harness-first workflow (OSS recipes → orient bundle → viewer);
- Runtime default: harness-first workflow (OSS recipes → Portolan bundle → viewer);
legacy Go CLI is frozen for new features until the Go decision gate resolves.
- Default output: ranked `orient/` hotspot bundle for navigation; legacy
- Default output: ranked Portolan bundle (`manifest.json`, `hotspots.jsonl`, …) for navigation; legacy
machine-readable evidence graph remains supported as an optional bridge.
- Local viewer: a read-only static orient viewer may be served locally for the
- Local viewer: a read-only static Portolan viewer may be served locally for the
duration of a user session; it must not mutate targets or require network by
default.
- Viewer truth boundary: graph nodes in the orient viewer must come from imported
- Viewer truth boundary: graph nodes in the Portolan viewer must come from imported
producer evidence or Portolan normalization; LLM-authored graphs are UX-only and
must never be labeled `source-visible`.
- Default privacy posture: no raw private source snippets, prompts, credentials,
Expand Down
22 changes: 15 additions & 7 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Portolan is not:
Portolan is:

- a read-only local discovery substrate an agent can run;
- a harness-first supplement (`harness/SKILL.md`, recipes, guardrails, orient
- a harness-first supplement (`harness/SKILL.md`, recipes, guardrails, Portolan
viewer) with an optional legacy Go CLI bridge;
- a normalizer for source, metadata, runtime, and claim evidence;
- a machine-readable evidence graph;
Expand All @@ -42,7 +42,7 @@ Portolan is:

## Engineering Rules

- Primary delivery: harness artifacts and orient bundle contract; see spec 087.
- Primary delivery: harness artifacts and Portolan bundle contract; see spec 087.
- Go CLI is frozen for new features per
[`docs/harness/GO-FREEZE-POLICY.md`](docs/harness/GO-FREEZE-POLICY.md).
- Keep `cmd/portolan` thin; put behavior in internal packages when Go changes are
Expand Down Expand Up @@ -193,9 +193,11 @@ Run:
go test ./...
go vet ./...
jq empty schema/*.json
jq empty harness/contracts/orient-bundle.schema.json
scripts/harness-orient-smoke.sh
scripts/orient-wizard.sh --help
jq empty harness/contracts/portolan-bundle.schema.json
scripts/harness-portolan-smoke.sh
scripts/portolan-scan.sh --help
jq empty harness/contracts/landscape-card.schema.json
jq empty harness/contracts/landscape-report.schema.json
git diff --check
```

Expand All @@ -214,11 +216,17 @@ shell commands, and other important information, read the current plan:
## Learned User Preferences

- Ask clarifying questions before drafting implementation plans when harness targets, Go role, or MVP surface are still open.
- Prioritize user-facing landscape navigation (hotspots, tech debt, duplication, where to look first) over evidence-discipline artifacts as the primary deliverable.
- Prioritize a report-first landscape viewer (Overview landing with ranked problems) over raw hotspot lists, map-only icicles, or evidence-discipline artifacts as the primary deliverable.
- Treat B2B evidence guardrails as a secondary layer on top of the map, not the main reason to use Portolan.
- Expand harness rules, guardrails, and OSS tool recipes rather than the Go codebase when adding product behavior.
- Use Portolan-prefixed names in user-facing CLI, scripts, and UI copy (e.g. `portolan-scan`, `scripts/portolan-scan.sh`).
- Explain in the viewer why findings rank as problems and what each navigation control does.
- Review delivery by observable facts and demo behavior, not code inspection alone.

## Learned Workspace Facts

- Understand-Anything is the reference UX for the local orient map; graph nodes must stay evidence-backed, not LLM-authored truth.
- Understand-Anything drill-down inside a landscape report is the reference UX; graph nodes must stay evidence-backed, not LLM-authored truth.
- Product-success questions center on concrete code pain (debt, duplication, risky zones), not abstract trust infrastructure alone.
- Landscape report UX references Portolan `map.md` sections and the standalone `landscape-card.json` contract; no sdp_lab runtime dependency.
- Demo-ready output must let an operator identify the target, repos, top problems, and gaps without external docs.
- Scan and viewer inventories must honor `.gitignore`.
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,22 +34,22 @@ partly black-box.
## Harness-First Quick Start (recommended)

Portolan is primarily a **harness supplement**: portable skill, OSS recipes,
guardrails, and a local orient map — not a Go module you must install first.
guardrails, and a local hotspot viewer — not a Go module you must install first.

**One command** (tool check → recipes → bundle → viewer):

```bash
scripts/orient-wizard.sh <target-root> <orient-dir> --yes
scripts/portolan-scan.sh <target-root> <bundle-dir> --yes
```

Add `--no-viewer` to build only. See `scripts/orient-wizard.sh --help`.
Add `--no-viewer` to build only. See `scripts/portolan-scan.sh --help`.

The orient viewer supports search, filters (including `config` and `debt-candidate`
kinds), a directory heat map, and click-to-source preview (local files only).
The viewer shows a ranked hotspot list, folder tree, search, filters (including
`config` and `debt-candidate` kinds), and click-to-source preview (local files only).
See [`docs/demo-runbook.md`](docs/demo-runbook.md).

Manual fallback: read [`harness/SKILL.md`](harness/SKILL.md), run recipes from
[`harness/recipes/`](harness/recipes/), then `scripts/build-orient-bundle.sh`.
[`harness/recipes/`](harness/recipes/), then `scripts/build-portolan-bundle.sh`.

See [`docs/harness/GO-FREEZE-POLICY.md`](docs/harness/GO-FREEZE-POLICY.md) for
legacy Go CLI status.
Expand Down Expand Up @@ -300,7 +300,7 @@ For repository development:
```bash
go test -count=1 ./...
jq empty schema/*.json
jq empty harness/contracts/orient-bundle.schema.json
scripts/harness-orient-smoke.sh
jq empty harness/contracts/portolan-bundle.schema.json
scripts/harness-portolan-smoke.sh
git diff --check
```
8 changes: 4 additions & 4 deletions docs/adr/001-go-cli-fate.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

**Status**: Provisional — thin maintenance layer (2026-06-10)

**Context**: Harness pivot (spec 087) ships recipes, orient bundle, and viewer
**Context**: Harness pivot (spec 087) ships recipes, Portolan bundle, and viewer
without requiring Go. Phase 5 smoke validates harness-only path on fixtures.

## Decision gate criteria

| Criterion | Result (2026-06-10 smoke) |
| --- | --- |
| Harness + scripts build orient bundle | Pass (`build-orient-bundle.sh`) |
| Viewer loads bundle without Go | Pass (`harness-orient-smoke.sh`) |
| Harness + scripts build Portolan bundle | Pass (`build-portolan-bundle.sh`) |
| Viewer loads bundle without Go | Pass (`harness-portolan-smoke.sh`) |
| Importer edge cases in production | Not re-tested; legacy `internal/importer` retained |
| Large JSONL / path safety | Legacy Go still has tests; harness uses jq/bash |

Expand All @@ -21,7 +21,7 @@ cases or a dedicated spec deprecates `cmd/portolan`.

Allowed:

- `map`, `import`, `query`, `orient-export-from-map.sh` bridge
- `map`, `import`, `query`, `portolan-export-from-map.sh` bridge
- Bugfixes and security fixes per GO-FREEZE-POLICY

Not allowed without new ADR:
Expand Down
14 changes: 7 additions & 7 deletions docs/agent/INSTALL-PROMPT.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,29 +14,29 @@ Replace variables with absolute local paths:
```text
PORTOLAN_PATH=<absolute path to Portolan checkout>
TARGET_PATH=<absolute path to local target>
ORIENT_PATH=<absolute path to empty orient output directory>
BUNDLE_DIR=<absolute path to empty bundle output directory>
```

Then send:

```text
Run the Portolan orient harness on TARGET_PATH. Write the orient bundle to
ORIENT_PATH. Execute now; do not ask unless a path is missing.
Run the Portolan harness on TARGET_PATH. Write the Portolan bundle to
BUNDLE_DIR. Execute now; do not ask unless a path is missing.

1. Read PORTOLAN_PATH/harness/SKILL.md
2. PORTOLAN_PATH/scripts/orient-wizard.sh TARGET_PATH ORIENT_PATH --no-viewer --yes
(or run recipes manually + build-orient-bundle.sh if operator prefers)
2. PORTOLAN_PATH/scripts/portolan-scan.sh TARGET_PATH BUNDLE_DIR --no-viewer --yes
(or run recipes manually + build-portolan-bundle.sh if operator prefers)
3. Read hotspots.jsonl and gaps.jsonl; cite hotspot.id and producer_ref per
PORTOLAN_PATH/harness/guardrails/citation-rules.md

If harness blocks external writes, use
PORTOLAN_PATH/.portolan/runs/<target-name> as ORIENT_PATH.
PORTOLAN_PATH/.portolan/runs/<target-name> as BUNDLE_DIR.

Answer with:
1. Recipes run (verified/failed/blocked)
2. Top 5 hotspots by rank with evidence
3. Gaps and not_assessed surfaces
4. Viewer command: cd PORTOLAN_PATH/viewer && npm run serve -- --bundle ORIENT_PATH
4. Viewer command: cd PORTOLAN_PATH/viewer && npm run serve -- --bundle BUNDLE_DIR
5. Unsupported claims avoided

Legacy Go CLI (optional): docs/harness/GO-FREEZE-POLICY.md
Expand Down
46 changes: 30 additions & 16 deletions docs/demo-runbook.md
Original file line number Diff line number Diff line change
@@ -1,52 +1,66 @@
# Portolan Demo Runbook

Live demo for a newcomer: **where code pain is** → **filter** → **directory tree** → **source**.
Live demo for a newcomer: **landscape report in 10 seconds** → **findings by section** → **gaps** → **drill-down to source**.

## Setup (two commands)
## Setup

**Single repo (portolan):**

```bash
scripts/orient-wizard.sh . /tmp/orient-portolan --yes
scripts/portolan-scan.sh . /tmp/portolan-portolan --yes
```

**Bounded multi-repo (bigtop quick sample):**

```bash
scripts/orient-wizard.sh ~/projects/bigtop-landscape/repos /tmp/orient-bigtop \
scripts/portolan-scan.sh ~/projects/bigtop-landscape/repos /tmp/portolan-bigtop \
--yes --limit-repos 3 --producers semgrep,syft
```

**Full landscape stress (18 repos, spec 091):**

```bash
scripts/orient-wizard.sh ~/projects/bigtop-landscape/repos /tmp/orient-bigtop-full \
scripts/portolan-scan.sh ~/projects/bigtop-landscape/repos /tmp/portolan-bigtop-full \
--no-viewer --yes --shard-timeout 600 --jscpd-memory-mb 2048
```

Expect 30–90+ minutes. Failed shards appear in gaps (not a wizard abort).
Use viewer on the result: `node viewer/scripts/serve.js --bundle /tmp/orient-bigtop-full`
Expect 30–90+ minutes. Failed shards appear in gaps (not a scan abort).

Wizard opens viewer automatically. For a fixed port:
`portolan-scan` opens the viewer automatically. For a fixed port:

```bash
cd viewer && node scripts/build-static.js
node scripts/serve.js --bundle /tmp/orient-portolan --port 4173
node scripts/serve.js --bundle /tmp/portolan-portolan --port 4173
```

Open http://127.0.0.1:4173/

## 5-step demo script
## 5-step demo script (report-first)

1. **Context** — Point at header: target path, hotspot count, evidence-backed (not LLM graph).
2. **Search** — Type `TODO` or `duplicate`; list and tree narrow instantly.
3. **Filter** — Toggle `static-finding`, `duplication`, or **`config`** (Docker/CI/env inventory); on bigtop, pick a **Repo** chip. Optional: `debt-candidate` for symbol-dense files when ctags ran.
4. **Directory heat map** — Expand a hot folder (severity bar + count); click a hotspot row.
5. **Source** — In Detail, show file paths and the **Source** snippet from local files (read-only).
1. **Overview tab (default)** — Project card: language, scale, maturity (README/CI/tests/Docker), repo matrix, findings shown vs scan total, top “where to look first”.
2. **Gaps tab** — What was **not** assessed (`not_assessed` / `cannot_verify`); not hidden findings.
3. **Findings tab** — Sections by kind (map.md parity), then folder tree + ranked list.
4. **Search & filter** — Type `TODO` or `duplicate`; use views (Tour top 15, Code pain, Config).
5. **Drill-down** — Click a finding → detail + **Source** snippet (read-only local files).

If bundle is truncated, use **Show all findings from scan** on the Findings tab.

## Demo bar (acceptance)

Operator answers without external docs:

| Question | Where in viewer |
| --- | --- |
| What is this target? | Overview → landscape card |
| How many repos? | Overview → repo table |
| Top problems? | Overview → next steps; Findings → sections |
| What was not checked? | Gaps tab |

Compare with `docs/test-corpora/apache-bigtop/examples/map-excerpt.md` for section parity.

## Talking points

- Gaps/truncation banner = honest limits (`not_assessed`, budget cap).
- Gaps/truncation = honest limits (`not_assessed`, budget cap).
- Every claim ties to `producer_ref` (jscpd, semgrep, syft, config-scan, ctags).
- No network; local-first.

Expand Down
6 changes: 3 additions & 3 deletions docs/harness/GO-FREEZE-POLICY.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ features** until the Go decision gate in
- Bug fixes with regression tests.
- Security fixes for path handling and untrusted artifact imports.
- Documentation corrections.
- Bridge commands/scripts that export legacy map output to `orient/` bundles.
- Bridge commands/scripts that export legacy map output to Portolan bundles.
- Dependency patches required for `go test ./...` on supported Go versions.

## Not allowed without a new spec and Go gate outcome
Expand All @@ -28,14 +28,14 @@ features** until the Go decision gate in
Use [`harness/SKILL.md`](../../harness/SKILL.md):

1. Run OSS recipes from `harness/recipes/`.
2. Build `orient/` with `scripts/build-orient-bundle.sh`.
2. Build a Portolan bundle with `scripts/build-portolan-bundle.sh`.
3. Open the local viewer from `viewer/`.

Optional legacy bridge:

```bash
go run ./cmd/portolan map --root <target> --out <map-dir>
scripts/orient-export-from-map.sh <map-dir> <orient-dir>
scripts/portolan-export-from-map.sh <map-dir> <bundle-dir>
```

## Review
Expand Down
2 changes: 1 addition & 1 deletion docs/harness/tool-profiles/jscpd.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
| User job | Code duplication / near-clone clusters |
| License | MIT |
| Review date | 2026-06-10 |
| Portolan action | Import JSON output into `orient/hotspots.jsonl` (`kind: duplication`) |
| Portolan action | Import JSON output into `<bundle-dir>/hotspots.jsonl` (`kind: duplication`) |

## Output surface

Expand Down
2 changes: 1 addition & 1 deletion docs/harness/tool-profiles/understand-anything.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
## Output surface

- Upstream: interactive graph + Q&A (not Portolan evidence)
- Portolan: `orient/` bundle consumed by `viewer/`
- Portolan: Portolan bundle consumed by `viewer/`

## Risks

Expand Down
4 changes: 2 additions & 2 deletions docs/onboarding.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ first.
| Intent | Start with | Then read | Boundary to preserve |
| --- | --- | --- | --- |
| Understand what Portolan is for | [README](../README.md) | [Product Claims](product-claims.md), [Product Boundary](product-boundary.md), [Product Quality Boundary](product-quality-boundary.md) | Portolan is a local evidence layer, not a coding harness, readiness gate, service catalog, observability platform, or Cursor/OpenCode replacement. |
| Ask an agent to inspect a local target | [Harness Skill](../harness/SKILL.md) | [Install Prompt](agent/INSTALL-PROMPT.md), [Agent Acceptance](agent/ACCEPTANCE.md) | Harness-first: recipes → orient bundle → viewer. Cite hotspot.id and preserve gaps. |
| Open the orient map (human) | [Viewer README](../viewer/README.md) | [Harness Skill](../harness/SKILL.md) | Local read-only viewer; evidence from producers only. |
| Ask an agent to inspect a local target | [Harness Skill](../harness/SKILL.md) | [Install Prompt](agent/INSTALL-PROMPT.md), [Agent Acceptance](agent/ACCEPTANCE.md) | Harness-first: recipes → bundle → viewer. Cite hotspot.id and preserve gaps. |
| Open the viewer (human) | [Viewer README](../viewer/README.md) | [Harness Skill](../harness/SKILL.md) | Local read-only viewer; folder tree + ranked hotspots from producers only. |
| Install or resolve the command | [Agent Install](agent/INSTALL.md) | [Troubleshooting](agent/TROUBLESHOOTING.md), [Release Guide](release.md) | Source bootstrap is local and does not fetch Go modules unless explicitly approved. |
| Use Cursor | [Agent Install Prompt](agent/INSTALL-PROMPT.md) | [Agent Acceptance](agent/ACCEPTANCE.md), [Product Claims](product-claims.md) | Current verified Cursor evidence is headless Cursor Agent CLI / Composer. Cursor UI behavior is outside the current required acceptance scope; no root-level Cursor rule is shipped. |
| Use OpenCode | [Install Prompt](agent/INSTALL-PROMPT.md) | [Agent Acceptance](agent/ACCEPTANCE.md), [Product Claims](product-claims.md) | OpenCode default-permission runs are verified with repo-local output under the Portolan checkout. The recorded external-output default-permission lane failed. |
Expand Down
Loading
Loading