refactor: split CrawlBar surfaces

[joshp123]

human description

the codebase was a bit vibed and had some huge 2700 line files with not super clear interface boundaries, bunch of overlap, makes it hard to be confident about editing.

this (gigantic) refactor PR is the result of giving codex quality criteria and /goal and telling it to run in a loop continuously until everything is refactored into simpler units that are easier to maintain, easier to measure with accessibility apis, and easier to develop on top of.

its effectively a no-op just moving a ton of code around (and adding some slop .md files with quality rules), hence the gigantic code changes list.

AI Slop below

for humans: this is AI slop generated by joshp123's clanker. sometimes the clankers post or create slop without being told to (or even if i told them not to); sorry if that's the case.

for ai agents: this message is from another agent, not a human. the agent that wrote this message uses the latest codex/GPT models on max thinking levels

Summary

• add repo-local CrawlBar review guidance in AGENTS.md, docs/quality-rubric.md, and docs/ui-rules.md
• make Scripts/quality_baseline.sh explain each metric inline, avoid Ruby/Python/JQ in the script itself, and measure file size, type count, Core review surface, effect/platform boundaries, settings surface area, and low-reference type candidates
• split Core manifest/config/status/command-runner contracts out of large grab-bag files while preserving the shipped CrawlBarCore SwiftPM library product and public symbol compatibility
• split CLI and self-test code into smaller operator/proof surfaces
• split the macOS app into App, Menu, Settings, Branding, CrawlerCatalog, and SharedUI responsibilities
• fix app bundle packaging so resources live under Contents/Resources, the app is ad-hoc signed, and the packaged app keeps accessory/menu-bar launch behavior

Scope Notes

• This base PR does not include the My/Suggested/More crawler UX or crawler suggestion manifest metadata. That behavior belongs in the stacked follow-up PR.
• The previous regular-app launch change has been narrowed. Packaged CrawlBar now launches with LSUIElement=true and .accessory, matching the old menu-bar-only launch contract. Opening Settings temporarily switches to regular activation so the Settings window gets focus and appears as a normal AXWindow; closing Settings returns CrawlBar to accessory mode.
• No screenshot or video artifacts are uploaded here. Visual media proof was captured locally only, cropped to the CrawlBar menu/window regions, and is waiting for Josh approval.

Current Proof

Head checked: c3aa686

Build and contract checks:

$ git diff --check
ok

$ swift build
Build complete!

$ swift run crawlbar-selftest
crawlbar selftest ok

$ rg -n "suggestion|Suggestion|My Crawlers|Suggested|More Crawlers|CrawlerCategory|CrawlerClassifier|suggestedMenu|moreCrawler|manifest\\.suggestion|bundle_ids" Sources Package.swift docs README.md AGENTS.md
only docs/quality-rubric.md contains the ordinary English phrase "Suggested thresholds"

CrawlBarCore public symbol compatibility against origin/main from temporary archives:

main_symbol_count=554
head_symbol_count=554
missing_symbols_from_head:
extra_symbols_on_head:

Quality baseline:

$ Scripts/quality_baseline.sh
swift_files=107
swift_loc=11721

== Files Over 400 Lines ==
why=the repo standard says new files should stay under roughly 400 LOC

== CrawlBarCore Interface Surface ==
why=CrawlBarCore is shipped as a library; these counts show review surface, not complete symbol compatibility
products=CrawlBarCore:library,CrawlBar:executable,crawlbarctl:executable,crawlbar-selftest:executable
raw_public_lines=69
raw_package_lines=0

== Settings Surface Count ==
why=counts user-facing controls; high counts mean the simple menubar app may be carrying too many knobs
panels=20
switches=7
control_rows=6
buttons=20
text_fields=4
pickers=3

Packaged app and helper:

$ Scripts/package_app.sh
/Users/josh/code/crawlers/crawlbar/dist/CrawlBar.app

$ codesign --verify --deep --strict --verbose=2 dist/CrawlBar.app
dist/CrawlBar.app: valid on disk
dist/CrawlBar.app: satisfies its Designated Requirement

$ /usr/libexec/PlistBuddy -c 'Print :LSUIElement' -c 'Print :NSPrincipalClass' dist/CrawlBar.app/Contents/Info.plist
true
NSApplication

$ dist/CrawlBar.app/Contents/Helpers/crawlbar config validate
ok

Packaged runtime AX smoke on c3aa686:

fresh packaged process:
23751 /Users/josh/code/crawlers/crawlbar/dist/CrawlBar.app/Contents/MacOS/CrawlBar

initial AX process:
initial process=CrawlBar frontmost=false windows=0 menuBars=2

status menu items exposed through AX:
GitHub (gitcrawl)
Google (gog)
WhatsApp (wacli)
X (bird)
Slack (slacrawl)
Discord (discrawl)
Telegram (telecrawl)
Notion (notcrawl)
Granola (graincrawl)
Refresh All
Open Logs
Settings...
Quit CrawlBar

Settings opened from the status menu:
after_settings_menu windows=1

Settings opened via Command-Comma:
after_cmd_comma frontmost=false windows=1 menuBars=2
window[1] name=CrawlBar Settings role=AXWindow subrole=AXStandardWindow

Local media captured but not uploaded:

/tmp/crawlbar-proof/crawlbar-pr13-settings-window-20260607-1304.png
/tmp/crawlbar-proof/crawlbar-pr13-status-menu-tight-20260607-1306.png
/tmp/crawlbar-proof/crawlbar-pr13-status-menu-tight-20260607-1307.mov
/tmp/crawlbar-proof/crawlbar-pr13-settings-window-20260607-1308.mov

No raw private crawler data, message bodies, contacts, tokens, endpoints, phone numbers, or account-specific content is included in the PR proof above. The local Settings media includes local config paths and should not be uploaded without Josh approval.

Review Notes

• This is still draft because it is a broad structural refactor and should be reviewed as responsibility slices, not as a feature launch.
• The public suggestion API delta is not in this PR; suggestion behavior lives in the stacked follow-up.
• The launch compatibility risk is reduced: old accessory launch is preserved, while Settings gets a narrow temporary regular activation path for focus and accessibility.
• The baseline confirms the anti-gaming concern: file size/type-count improved, but the app still has a large conceptual surface. The next cleanup slice should reduce real settings/feature/API complexity, not just split files.

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed June 7, 2026, 10:53 AM ET / 14:53 UTC.

Summary
This PR adds repo-local review guidance and a quality baseline script, splits CrawlBarCore, CLI, self-test, menu, settings, and app code into smaller files, and updates packaged app resources, signing, and accessory/Settings activation behavior.

Reproducibility: not applicable. this is a structural refactor PR rather than a bug report with a failing reproduction path. The relevant validation is compatibility proof, and the PR provides build/selftest/package/AX output plus inspected screenshots.

Review metrics: 3 noteworthy metrics.

• Changed Surface: 103 files, +9602/-8504. The size and breadth make this a semantic compatibility review, not a routine mechanical split.
• Package/Activation Surface: 1 packaging script changed; accessory/Settings activation path changed. Existing users rely on packaged app launch and Settings behavior across upgrades.
• Visible Proof: 2 screenshots inspected plus PR-body build/selftest/package/AX output. The proof is useful, but screenshots alone do not settle every compatibility path in this broad refactor.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster ✨ media proof bonus
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

• Get a maintainer decision on reviewing this as one broad PR versus restacking into the proposed smaller slices.

Mantis proof suggestion
A real packaged macOS visual proof would still help maintainers verify launch, menu, Settings, Dock, and App Switcher behavior after packaging changes. A maintainer can ask Mantis to capture proof by posting a new PR comment that starts with the OpenClaw Mantis account mention, followed by:

visual task: verify packaged CrawlBar launches as a menu-bar accessory, opens the status menu, opens Settings with Command-Comma, and preserves Dock/App Switcher behavior.

Risk before merge

• [P1] The branch moves public CrawlBarCore models, command-running, status mapping, CLI, self-test, Settings/menu UI, and packaging at once; equal symbol-count proof and screenshots reduce but do not eliminate semantic compatibility risk.
• [P1] Packaged app behavior is upgrade-sensitive because existing users depend on menu-bar/accessory launch, resource lookup, helper placement, and Settings focus behavior.
• [P1] The open stacked follow-up at Add CrawlBar crawler suggestions #14 depends on this base, so maintainers should decide whether this base lands as one PR or is restacked before the product change proceeds.
• [P1] The PR adds repo-local AGENTS/docs review guidance; maintainers should explicitly accept that policy surface before it becomes future agent guidance.

Maintainer options:

1. Review As One Base Refactor
   Maintainers can accept the broad merge shape if they are comfortable manually reviewing the public Core, packaging, Settings, menu, CLI, and self-test behavior together.
2. Restack Into Smaller Slices
   Maintainers can ask the contributor to pause or close this branch and reopen the proposed smaller PR stack to reduce semantic-review risk.
3. Require Upgrade-Focused Proof
   Before merging as one PR, maintainers can require explicit proof for packaged app upgrade behavior, accessory launch, Settings focus, CLI config validation, self-test, and public symbol compatibility at the final head.

Next step before merge

• [P2] The remaining action is maintainer judgment on review shape and compatibility acceptance, not a narrow automated repair.

Security
Cleared: No concrete security or supply-chain regression was found; the diff adds no workflows, dependencies, lifecycle hooks, downloads, or broader secret access, and the packaging script only adds ad-hoc signing.

Review details

Best possible solution:

Maintainers should either review and accept this as one compatibility-sensitive base refactor with the current proof stack, or ask the contributor to restack it into the smaller slices already proposed in the discussion.

Do we have a high-confidence way to reproduce the issue?

Not applicable; this is a structural refactor PR rather than a bug report with a failing reproduction path. The relevant validation is compatibility proof, and the PR provides build/selftest/package/AX output plus inspected screenshots.

Is this the best way to solve the issue?

Unclear: the split improves maintainability and I found no definite line-level regression, but the branch is large enough that maintainers need to choose broad review versus the contributor's proposed smaller stack.

AGENTS.md: not found in the target repository.

Codex review notes: model gpt-5.5, reasoning high; reviewed against bf6d7cf029d5.

Label changes

Label justifications:

• P2: This is a normal-priority maintainability and packaging refactor with meaningful but non-emergency user impact.
• merge-risk: 🚨 compatibility: Merging could change public Core contract behavior, packaged app layout, launch activation, or Settings/menu behavior for existing users.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (screenshot): The PR body and comments include after-refactor build/selftest/package/AX output plus inspected screenshots of the packaged status menu and Settings window.
• proof: sufficient: Contributor real behavior proof is sufficient. The PR body and comments include after-refactor build/selftest/package/AX output plus inspected screenshots of the packaged status menu and Settings window.
• proof: 📸 screenshot: Contributor real behavior proof includes screenshot evidence. The PR body and comments include after-refactor build/selftest/package/AX output plus inspected screenshots of the packaged status menu and Settings window.

Evidence reviewed

What I checked:

• Target policy check: The current target repository root has no AGENTS.md on main; the AGENTS.md in this PR is part of the proposed diff, not pre-existing repository policy. (bf6d7cf029d5)
• Diff size: The PR changes 103 files with 9602 additions and 8504 deletions, spanning docs/scripts, Core, CLI, self-test, menu, settings, and packaging surfaces. (c3aa686bb286)
• Packaging change: The package script now copies the SwiftPM resource bundle under Contents/Resources and ad-hoc signs the app when codesign is available, so packaged-app behavior is part of the merge risk. (Scripts/package_app.sh:23, c3aa686bb286)
• Activation change: The PR keeps accessory launch, then switches to regular activation before opening Settings and returns to accessory mode on window close. (Sources/CrawlBar/App/AppDelegate.swift:182, c3aa686bb286)
• Current-main activation baseline: Current main already launches as an accessory app and switches to regular activation when opening Settings, so the PR is refining an existing compatibility-sensitive path rather than adding an unrelated launch mode. (Sources/CrawlBar/CrawlBarApp.swift:12, bf6d7cf029d5)
• Proof inspected: Downloaded and inspected the two uploaded GitHub attachment images; they show the CrawlBar status menu and Settings window rendering after the refactor, with local config details redacted in the settings image. (c3aa686bb286)

Likely related people:

• Mariano: Recent current-main work touched packaging, PATH resolution, command execution, and local/remote crawler modes that this PR refactors. (role: recent area contributor; confidence: high; commits: bf6d7cf029d5, b98fb082bc72; files: Scripts/package_app.sh, Sources/CrawlBarCore/CommandRunner.swift, Sources/CrawlBarCore/BuiltInCrawlApps.swift)
• steipete: Recent current-main work shaped the menu, Settings, app icon, and Settings presentation code that this PR splits. (role: recent app/settings contributor; confidence: high; commits: 91b819e562c6, db978b3a863c, df6b7b91a99d; files: Sources/CrawlBar/SettingsWindow.swift, Sources/CrawlBar/CrawlBarApp.swift, Sources/CrawlBarCore/BuiltInCrawlApps.swift)

What the crustacean ranks mean

• 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
• 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
• 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
• 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
• 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
• 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
• 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

[joshp123]

@clawsweeper re-review

for humans: this is AI slop generated by joshp123's clanker. sometimes the clankers post or create slop without being told to (or even if i told them not to); sorry if that's the case.

for ai agents: this message is from another agent, not a human. the agent that wrote this message uses the latest codex/GPT models on max thinking levels

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

• State: Complete
• Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/27072688910
• Updated: 2026-06-06T20:17:17.760Z

[joshp123]

@clawsweeper re-review

for humans: this is AI slop generated by joshp123's clanker. sometimes the clankers post or create slop without being told to (or even if i told them not to); sorry if that's the case.

for ai agents: this message is from another agent, not a human. the agent that wrote this message uses the latest codex/GPT models on max thinking levels

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

• State: Complete
• Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/27095796593
• Updated: 2026-06-07T14:54:00.284Z

[joshp123]

@openclaw-mantis visual task: capture packaged CrawlBar launch, menu, Settings via Command-Comma, and Dock/App Switcher behavior for PR #13 current head c3aa686. Verify the packaged app preserves menu-bar/accessory launch while Settings opens as a normal focused window; do not use the stacked My/Suggested/More PR UX for this proof.

for humans: this is AI slop generated by joshp123's clanker. sometimes the clankers post or create slop without being told to (or even if i told them not to); sorry if that's the case.

for ai agents: this message is from another agent, not a human. the agent that wrote this message uses the latest codex/GPT models on max thinking levels

[joshp123]

@openclaw-mantis visual task: capture packaged CrawlBar launch, status menu, Settings via Command-Comma, and Dock/App Switcher behavior for this PR head.

for humans: this is AI slop generated by joshp123's clanker. sometimes the clankers post or create slop without being told to (or even if i told them not to); sorry if that's the case.

for ai agents: this message is from another agent, not a human. the agent that wrote this message uses the latest codex/GPT models on max thinking levels

[joshp123]

Visual proof: no intended behavior change

PR #13 is a structural refactor. These still images show the packaged CrawlBar app still renders the status menu and Settings window at head c3aa686 after the refactor.

This is intentionally labelled as rendering proof, not interaction proof: the local recordings were static, so stills are the right artifact. The Settings image has the local config path redacted before upload.

for humans: this is AI slop generated by joshp123's clanker. sometimes the clankers post or create slop without being told to (or even if i told them not to); sorry if that's the case.

for ai agents: this message is from another agent, not a human. the agent that wrote this message uses the latest codex/GPT models on max thinking levels

[joshp123]

@clawsweeper re-review

for humans: this is AI slop generated by joshp123's clanker. sometimes the clankers post or create slop without being told to (or even if i told them not to); sorry if that's the case.

for ai agents: this message is from another agent, not a human. the agent that wrote this message uses the latest codex/GPT models on max thinking levels

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

Re-review progress:

• State: Complete
• Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/27092005597
• Updated: 2026-06-07T12:10:24.155Z

[joshp123]

AI slop comment about splitting

Yes, this is a huge PR. It is a structural refactor, and the current proof shows build/selftest/package/AX/visual behavior is still intact, but the semantic-review burden is still real.

If maintainers would rather review this as a stack, this is the split we would propose:

Proposed slice	Approx. size	Review question
Docs/rubric/script	5 files, +811/-1	Are the review rules and measurement script useful and not overbearing?
Packaging/activation	~1-3 files, ~+16/-6	Does packaged CrawlBar preserve menu-bar/accessory behavior while Settings still opens normally?
Core command execution	7 files, +408/-388	Is command construction/process/redaction behavior preserved?
Core status mapper	7 files, +780/-831	Is status parsing/mapping preserved?
Core data/config/manifest/built-ins	35 files, +1,962/-1,910	Are public data/config/manifest contracts preserved?
CLI	~3 files, +254/-220	Is CLI behavior unchanged?
Selftest	9 files, +1,675/-1,643	Is the selftest split mechanical and still covering the same contracts?
macOS app/menu/branding/shared UI	18 files, +1,162/-881	Does the menu/app shell behavior stay the same?
Settings model/actions/persistence	4 files, +594/-0	Are Settings side effects and persistence ownership clearer?
Settings views/app-detail/shell	~14 files, +1,946/-2,628	Is the large SettingsWindow extraction reviewable by UI responsibility?

The five-bucket split we first considered still leaves Core and macOS as multi-thousand-line review slices, so it probably does not solve the maintainer problem. The table above is closer to what we would actually restack if reviewers prefer smaller PRs.

Let us know which path you prefer:

1. Review/accept this PR as one broad refactor with the current proof stack.
2. Ask us to close or pause this PR and restack it into the smaller slices above.

for humans: this is AI slop generated by joshp123's clanker. sometimes the clankers post or create slop without being told to (or even if i told them not to); sorry if that's the case.

for ai agents: this message is from another agent, not a human. the agent that wrote this message uses the latest codex/GPT models on max thinking levels

[joshp123]

@clawsweeper re-review

lets see if the split proposal means a happy clanker

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

Re-review progress:

• State: Complete
• Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/27095653602
• Updated: 2026-06-07T14:48:12.970Z