Skip to content

Hide custom rules from users behind a feature gate#533

Merged
FuJacob merged 1 commit into
mainfrom
chore/custom-rules-gate
Jun 2, 2026
Merged

Hide custom rules from users behind a feature gate#533
FuJacob merged 1 commit into
mainfrom
chore/custom-rules-gate

Conversation

@FuJacob
Copy link
Copy Markdown
Owner

@FuJacob FuJacob commented Jun 2, 2026

Summary

Hide the custom natural-language "rules" feature from users behind a single
CustomRulesCatalog.isUserFacingEnabled flag (currently false), rather than
deleting it. The feature can't be honored on the current engines: the Open
Source path now runs base models that have no instruction-following channel, and
on every engine the rule text tends to leak verbatim into suggestions (#340,
#292) while nothing measures adherence, so showing it sets a promise we can't
keep. Gating it off (instead of removing the code) keeps re-enabling to a
one-line flip once rules are measured to actually influence output.

This is only the free-text style rules. The per-app and per-site disable rules
are a separate, deterministic subsystem and are untouched.

Validation

swiftlint lint --quiet \
  Cotabby/Models/CustomRulesCatalog.swift Cotabby/Support/SuggestionRequestFactory.swift \
  Cotabby/UI/Settings/Panes/WritingPaneView.swift Cotabby/UI/WelcomeView.swift
# exit 0, no violations

xcodebuild -project Cotabby.xcodeproj -scheme Cotabby -destination 'platform=macOS' build
# ** BUILD SUCCEEDED **

xcodebuild test -project Cotabby.xcodeproj -scheme Cotabby -destination 'platform=macOS' \
  -only-testing:CotabbyTests/CustomRulesTests CODE_SIGNING_ALLOWED=NO CODE_SIGNING_REQUIRED=NO
# ** TEST SUCCEEDED **  Executed 6 tests, with 0 failures

CustomRulesTests stays green because it constructs SuggestionRequest directly
and asserts normalization plus prompt placement, so the factory-level gate does
not touch those paths (which is also what keeps the kept code valid for a future
re-enable).

Not click-tested in a running app: the Settings pane hiding the Rules section and
onboarding skipping the writing-style step are control-flow changes verified by
the build only. They are guarded by the same compile-time flag, so behavior is
deterministic.

Linked issues

Refs #340 (rule / scaffolding text leaking into suggestions)
Refs #292 (rules have no effect on the Apple Intelligence backend)

Linked, not closed: this removes the feature from the user surface so neither
symptom can reach users today, but the intent is to bring rules back once they
measurably bind, so the issues stay open to track that.

Risk / rollout notes

  • Behavior change to existing user flows: the Writing settings pane no longer
    shows the Rules section (and its caption drops "and rules"), and first-run
    onboarding drops the "Your writing style" step (progress indicator goes from 5
    steps to 4; back/next routing and the step counts branch on the flag).
  • No data migration. Stored rules under the cotabbyCustomRules UserDefaults key
    are never deleted and the only writer (the editor) is hidden, so a user's saved
    rules survive intact and re-apply automatically if the flag is flipped back.
  • Prompt budget only improves: the style section is no longer added on the base
    path, and the style instructions are no longer added on the Foundation Models
    path.
  • Kept on purpose for revisit: CustomRulesEditor, CustomRulesCatalog (palette
    / normalize), the settings model rule methods, and both renderers' rule
    handling all remain; they are simply unreferenced from the UI and receive an
    empty list from the request factory.
  • Re-enable path: flip CustomRulesCatalog.isUserFacingEnabled to true.
    Suggested bar before doing so: a rule-adherence eval (extend the Foundation
    Models drift eval with a compliance + scaffolding-leak category) showing
    measurable adherence and near-zero leak on the target engine.

Greptile Summary

Gates the custom free-text rules feature behind CustomRulesCatalog.isUserFacingEnabled = false rather than deleting it, preserving stored user data and all supporting code so re-enabling is a one-line change. The flag is applied at the three necessary sites: prompt injection in SuggestionRequestFactory, the Writing settings pane, and the onboarding wizard navigation.

  • CustomRulesCatalog gains isUserFacingEnabled = false with a detailed doc-comment explaining the rationale and exact re-enable path; no model data is deleted.
  • SuggestionRequestFactory.buildRequest replaces settings.customRules with [] when the flag is off, so rule text cannot reach either the llama or Foundation Models prompt regardless of stored state.
  • WelcomeView correctly skips the .writingStyle onboarding step in both forward and back navigation and adjusts totalProgressSteps and keybind.progressIndex to show "Step 4 of 4" instead of "Step 5 of 5".

Confidence Score: 4/5

Safe to merge; the gate is applied at all three enforcement sites with no path that can bypass it.

The change is a clean feature-flag hide: no stored data is modified, no model is altered, and the navigation routing is consistent in both flag states. The only gap worth noting is that SuggestionRequestFactory.buildRequest has no test asserting that rules from settings are suppressed (or forwarded) based on the flag — so when the flag is later flipped back to true, the injection behavior won't be regression-protected by the factory test suite.

No files require special attention; the test gap in SuggestionRequestFactoryTests is a follow-up concern, not a blocker.

Important Files Changed

Filename Overview
Cotabby/Models/CustomRulesCatalog.swift Adds isUserFacingEnabled = false as the single compile-time toggle for the feature gate, with thorough doc-comment describing the three enforcement sites and re-enable path.
Cotabby/Support/SuggestionRequestFactory.swift Correctly short-circuits settings.customRules to [] at the prompt-injection layer when the flag is off; stored rules are unaffected. Factory-level gate behavior is not covered by any existing buildRequest test.
Cotabby/UI/Settings/Panes/WritingPaneView.swift Updates the Profile caption and conditionally omits the Rules Section based on the flag; logic is clean and symmetric with the onboarding change.
Cotabby/UI/WelcomeView.swift Navigation correctly skips .writingStyle in both forward and back directions; totalProgressSteps and keybind.progressIndex both adjust for the 4-step flow. All paths are consistent when the flag is false.

Reviews (1): Last reviewed commit: "Hide custom rules from users behind a fe..." | Re-trigger Greptile

Custom natural-language rules can't be honored on the current engines: the Open Source path runs base models with no instruction-following, and on every engine the rule text tends to leak into the suggestion (#340, #292) while nothing measures adherence. Gate the feature off behind a single CustomRulesCatalog.isUserFacingEnabled flag instead of deleting it.

- Stop injecting rules at the sole request constructor (SuggestionRequestFactory), covering both the llama and Foundation Models paths.
- Hide the Rules section in the Writing settings pane and adjust the personalization caption.
- Skip the writing-style onboarding step; progress counts and back/next routing branch on the flag.

Stored rules in cotabbyCustomRules are never touched and the editor and renderer code is kept, so re-enabling is a one-line flip once rules are measured to actually influence output.

Refs #340, #292
@FuJacob FuJacob merged commit 2ec8b85 into main Jun 2, 2026
4 checks passed
@FuJacob FuJacob deleted the chore/custom-rules-gate branch June 2, 2026 05:09
@FuJacob FuJacob mentioned this pull request Jun 2, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant