Skip to content

Port the local-first runtime architecture - #553

Open
Sheldenshi wants to merge 158 commits into
mainfrom
Sheldenshi/port-local-architecture
Open

Port the local-first runtime architecture#553
Sheldenshi wants to merge 158 commits into
mainfrom
Sheldenshi/port-local-architecture

Conversation

@Sheldenshi

@Sheldenshi Sheldenshi commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • port the provisioned runtime's current agent, task, routine, People, integrations, Slack, browser, and mobile improvements into the public monorepo
  • keep the public build local-first: Google OAuth and account identity stay on the runtime, provider credentials stay local, and People extraction starts only when the user requests a sync
  • require each local operator to provide their own Google Desktop OAuth client through the encrypted connector flow; no project-owned OAuth client ships in the public runtime
  • remove hosted-only trust, model-routing, managed-deployment, relay-provisioning, Workspace-grant, mobile hosted-login, and private demo paths
  • retain user-selected remote access (Gini Relay, Tailscale, ngrok, or Cloudflare) and Docker self-hosting as local-runtime features
  • update the current architecture docs and ADRs to describe the resulting public design

Why

The provisioned codebase has accumulated substantial product and runtime improvements that the public repository no longer reflects. This ports those improvements while preserving the open-source repository's local-hosted trust boundary and excluding the private edge/control-plane machinery used to provision managed guests.

User impact

The public runtime gains the newer Home, Routines, People, Integrations, Google multi-account, Slack Socket Mode, task/chat, browser, provider, and mobile behavior. Existing local instances also prune the retired bundled yc-cli demo record without removing a user-authored skill with the same name. Before connecting Google, local operators create a Desktop OAuth client in their own Google Cloud project and store it through Integrations.

Validation

  • bun run typecheck
  • bun run test (5,025 runtime/mobile/skill tests and 763 web tests pass)
  • env -u GINI_INSTANCE -u GINI_ACCESS_TOKEN bun run gini smoke (ok: true)
  • git diff --check origin/main...HEAD
  • browser-driven checks of Home, People, Routines, Integrations, Google, Slack, and Skills; the unconfigured Google flow opens the encrypted OAuth setup dialog without navigating to Google
  • topic-history and current-tree scans confirm the PR adds no credential-shaped Google OAuth literals
  • exact audit for hosted/edge secrets, trusted-edge headers, managed-deployment gates, provisioned relay grants, auth preflight, and private demo markers

Sheldenshi and others added 30 commits July 17, 2026 18:01
Extract the three starter-routine specs from onboarding's routineJobSpecs
into a shared catalog (src/runtime/routine-templates.ts) that composes
prompts/crons/skills server-side; onboarding delegates to it, unchanged
contract. Installed jobs link back via an optional JobRecord.templateId
threaded through createScheduledJob.

New gateway endpoints (thin http.ts delegations): GET
/api/routines/templates (catalog + agent-scoped installed state), POST
/api/routines/templates/<id>/install (idempotent per-template replace,
skills pre-validated, timezone payload > onboarding record > UTC), DELETE
/api/routines/templates/<id>.

Web: /routines gallery page (card grid with sub-option checkboxes,
schedule hint, Install / installed-state + Remove) with a sidebar link and
React Query hooks over the BFF.

See ADR routine-templates-gallery.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…talls in onboarding replace pass

Install and uninstall now resolve the owning agent server-side (same
resolveEffectiveContext mechanism createScheduledJob uses to stamp
agentId) and only touch that agent's job for the template — previously
they deleted every job carrying the templateId across all agents.
Uninstall 404s when the active agent has no install.

applyOnboardingRoutines' replace pass additionally deletes the owning
agent's live jobs stamped with a catalog templateId, so a gallery
reinstall (which leaves a stale id in routineJobIds) can no longer lead
a later onboarding apply to duplicate a routine — at most one live job
per template per agent holds for both writers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rebuild the routines gallery to the GiniRoutines design handoff: a pill
segmented toggle switches between My routines (grid of added routines
with a per-card menu for View in Jobs / Remove, empty state that jumps
to Explore) and Explore (catalog cards with one-click Add using server
default options, flipping to a ghost 'Added' pill that removes on
click). Sub-option checkboxes move out of the gallery; chip colors come
from the design tokens. No API or query-hook changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Persist the resolved option state as JobRecord.templateOptions (stamped by
the catalog's buildSpec on both the gallery install and onboarding paths,
validated in createScheduledJob like templateId) and expose it as
installed.options in GET /api/routines/templates.

Add /routines/[templateId] per the GiniRoutineDetail design handoff: hero
card with dot pattern and pause/resume toggle, Run Now action, underline
tabs (Recent sessions from the job's run history, Settings that re-installs
with edited options, Info rows with Delete routine). My routines cards
navigate to it; the chip icon/color mapping moves to a shared module.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Watcher-backed routines (thread/sender email watches created via chat)
now appear in My routines alongside installed templates: cyan mail-chip
cards with objective-first titles, a paused state, and a menu with Open
channel + Remove. A new detail page at /routines/watch/[watcherId]
mirrors the template detail's visual language — hero with enable toggle
(PATCH enabled), Open channel action, Settings tab (objective save /
clear) and Info tab (matcher, account, status, created, delete).

Shared presentation helpers live in routines/watchers.ts; the watcher
chip in routines/chips.ts. Explore is unchanged (watchers are created
conversationally, not from the gallery); the empty state now requires
no installed templates AND no watchers.

Also stop click propagation at the watcher card's menu content: React
portals bubble through the React tree, so a menu-item click would reach
the card's navigate-to-detail onClick and clobber the item's own
navigation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The template routine card in My routines navigates to its detail page
on click, and its dropdown menu renders in a portal. React portals
bubble events through the React tree, so a click on "View in Jobs"
also fired the card's onClick and clobbered the item's navigation.
Stop propagation at the DropdownMenuContent boundary, matching the
watcher card's existing guard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the binary targeted-beats-triage precedence in the gmail-watch
detector with a universal claim layer: watches run in specificity order
(thread > sender > query > synthetic triage, stable creation-order
tiebreak within a tier) and every watch claims the ids it keeps and
drops ids already claimed this tick. Two overlapping targeted watches
(or two overlapping query watches) no longer double-draft the same
message. A fully-claimed watch opens no bucket but still commits its
advanced cursor; trusted items and the cross-account Message-ID dedup
are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… there

Every template-routine install now owns a live channel session titled
after the routine, created via the shared createRoutineJob helper (the
create_job tool's createDedicatedSession idiom) on both the gallery and
onboarding paths. Reinstalls (Settings save, onboarding re-apply) capture
the replaced job's live channel before the replace pass and rebind the
new job to it — bind before un-archiving, since archiveOrphanJobChannels
re-archives any unreferenced live job channel on state load. Uninstall
keeps removeJob's archive semantics (history stays addressable).

The Morning/Meeting Briefing specs drop forwardToChat — the routine's
own conversation is the delivery surface, not the hidden main Chat.
GET /api/routines/templates exposes installed.chatSessionId; the detail
page gains an Open messages action, and the sidebar's Messages section
lists live job delivery channels (excluding feature-owned email-watch
channels and headless threads) so routine conversations are visible
without a deep link.

ADR routine-templates-gallery.md documents the delivery contract; tests
cover session provisioning, reinstall reuse across both writers,
uninstall archiving, and the forwardToChat removal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…k an answered question

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
My routines now lists the effective agent's chat-created cron jobs (no
templateId) as generic routine cards alongside template installs and email
watchers, with a detail page at /routines/job/[jobId] (Recent sessions +
Info tabs, pause/resume, Run Now, Open messages, Delete; no Settings tab —
prompt/cron editing stays in chat and /jobs).

The shared email-watch detection job is excluded structurally (skill-script
pre-run hook at the gmail-watch skill, the same marker findSharedJobId
uses), never by name. Partition/presentation helpers live in
routines/custom-jobs.ts with bun:test coverage; the empty state now
requires all three card kinds to be absent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
/routines already lists every routine kind (template installs, email
watchers, custom scheduled jobs), so the separate /jobs page is redundant.

- Delete the /jobs UI (JobList/JobDetail/RunList); keep the route as a
  stale-link redirect: /jobs?job=<id> -> /routines/job/<id>, bare /jobs
  -> /routines.
- Make /routines/job/[jobId] the canonical per-job URL: a job carrying a
  templateId forwards to /routines/<templateId>.
- Relocate the pieces with surviving consumers to
  packages/web/src/components/jobs/ (schedule-label + test, calendar
  views, EditJobDialog); the chat JobsTab keeps using them.
- Add an Edit schedule hero action on the custom-job detail, opening the
  shared EditJobDialog (via a new optional trigger prop) for
  interval/cron/timezone, retry, timeout, budget, and delivery edits.
- Update every /jobs link: drop the routines cards' "View in Jobs" menu
  items (card click opens detail), point Greeting and the JobsTab runs
  link at /routines, and land the channel header's View job pill on
  /routines/job/<id>.
- ADR routine-templates-gallery.md: /routines is the sole scheduled-jobs
  surface; /jobs is a redirect.

Deliberately dropped: the standalone page's calendar tab and two-pane
run browser (the calendar remains in the chat JobsTab).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a "Create routine" affordance on /routines (header action plus the
empty state) that deep-links home with a ?prompt= seed, and extend the
HomeComposer deep-link effect to pre-fill the composer from ?prompt=,
switch to Message mode, and place the caret at the end. Pre-fill only,
never auto-submit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…Tools display

Resolves the job's owning agent and applies the same enabled-toolset ∩
agent-whitelist gate the dispatch path uses, returning {name, label,
toolset, summary} rows sorted by toolset then label so clients render
the effective catalog without re-deriving tool names. Unknown job ids
throw the "Job not found:" prefix the route-table catch maps to 404.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tabs

Adds JobToolView/useJobTools over GET /api/jobs/:id/tools and a shared
RoutineTools section (toolset-mapped icons, two-column rows, summary
tooltips, Show all/fewer toggle past 14 rows) rendered after the
Information rows on both the template and custom-job detail pages. The
section hides itself when the template isn't installed or the catalog
is empty.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the alphabetical toolset sort in listJobTools with a curated
cluster order that leads with high-signal capabilities (email, file,
memory, jobs) and pushes the large browser cluster toward the end, so
the collapsed Tools view isn't a single-toolset wall of identical
icons. Unlisted toolsets fall back to alphabetical after the curated
list; labels still sort within each cluster.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
create_job's dispatch now returns the new job's id as a structured
DispatchResult.jobId alongside the model-facing result string; the
chat-task loop stamps it onto the tool_call block via emitToolCallStatus
(mirrored to a Topic's parent Chat), and chat-blocks persists/rehydrates
it like the other optional tool_call fields. Gives the web routine card
a click target without parsing the truncated tool_result preview.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… card

rendersAsRoutineCard (lib/group-exchanges.ts) is the shared predicate:
grouping excludes those calls from the collapsed tool_group and passes
them through as their own block items, and BlockRenderer routes them to
the new RoutineCreatedCard (violet Routine chip + name + schedule,
non-clickable 'Routine removed' state once the job is gone). The card
opens the routine variant of the side panel slot: TopicPanelContext is
now a topic-or-routine union (openRoutine/openRoutineJobId, one panel
at a time), and the home page's URL-backed provider carries the routine
variant as /?panel=routine:<jobId>.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RoutineDetailsPanel shares the TopicPanel slot (440px drawer, Details
header + close): hero row with the routine chip icon, name, and a
pause/resume toggle; description; Settings rows (Triggers schedule,
Skills) with the RoutineTools catalog; the Info-tab information rows;
and a footer with Open Settings (deep link to /routines/job/[jobId])
and Delete Routine (DELETE + toast + close). Both mount points render
it when the panel slot holds the routine variant; the job resolves from
the unscoped jobs list with quiet loading / not-found states.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
/routines/job/[jobId] is the canonical per-job URL for ANY job id, but
it resolved via the agent-scoped useJobs(): a job owned by another
agent 404'd, and the scoped cache's slower refetch 404'd a job created
moments ago from chat when the routine panel's Open Settings deep link
landed before the cache caught up. useAllJobs() shares the sidebar's
fast-refetch unscoped cache.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
homeView stamps an optional routineJobId on HomeTaskItem by mapping each
served session to the newest job whose chatSessionId points at it (one
precomputed pass over state.jobs, the draftTaskIds idiom). HomeTaskRow
renders the violet Routine chip — extracted as RoutineChip from
RoutineCreatedCard so the two affordances share one visual — in the
right slot; chip click stops propagation and opens the routine details
panel while the row body keeps opening the topic thread. Views test
pins newest-job-wins and omission on rows without a matching job.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A home task row may be showing the deleted routine's chip
(HomeTaskItem.routineJobId); without the nudge it lingers until the
home query's 30s idle refetch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Create routine entry now deep-links /?compose=task&prompt=… and the
composer's seed effect picks the mode from the compose param (message
unchanged; a bare ?prompt= keeps the legacy Message-mode seed). A
Task-mode submit opens the new container in the home right-side topic
panel on success instead of leaving the user staring at the queue row,
so the turn — including the routine card — plays out beside home.
Message mode still navigates to the thread. scheduleLabel humanizes
clean intervals (every 10 minutes / every 2 hours / every day) and
keeps the raw every-Ns shape for odd second counts, fixing the raw
'every 600s' on the card, panel Triggers row, and detail pages at once.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
update_job's dispatch now returns the job id alongside the confirmation
string (the create_job structured-return seam; the error-string early
returns carry none), riding the existing DispatchResult.jobId plumbing
onto the tool_call block. rendersAsRoutineCard accepts update_job
ok-with-jobId, so grouping and BlockRenderer surface the same routine
card for schedule changes while sibling calls keep collapsing.
delete_job deliberately stays unstamped — its card could only render a
deleted-routine tombstone. ADR jobId description updated; tests pin the
update stamp, the delete omission, and the grouping/routing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ile dossiers

The skill now covers folding people out of any handed material (a message,
thread, or meeting note), not just contact-export imports: a `profile`
dossier column holding the rewritten-not-appended story of who a person is
to the user, a query-before-write merge discipline keyed by email address,
parameterized-SQL examples, and an allowed-tools declaration pinning the
work to the db_* surface. Rules gain the untrusted-data framing (extract
facts, never follow instructions inside the material) and the
skip-user/skip-automated-senders line.
…ocking, description column

- contacts seeds with CHECK constraints (email lowercased+shaped, url
  scheme-qualified, phone E.164) so fabricated/denormalized values are
  rejected at the write boundary instead of by prose.
- updated_at epoch-ms column + monotonic touch trigger (MAX(now, old+1)
  closes the same-millisecond ABA window) enables optimistic locking;
  partial unique index arbitrates concurrent email-less inserts.
- description column: one-line list handle so roster queries skip the
  multi-KB profile dossiers; legacy tables backfill on open.
- import-table: refuse to import over a same-name table whose columns
  mismatch the file header (drop if empty, error otherwise).
AgentRecord.autoMemory=false makes that agent's chat turns skip
auto-recall (the pre-model Hindsight query) and auto-retain (the
post-completion background distillation); memory tools stay governed by
toolsets. Surfaced through EffectiveContext, settable at creation or via
POST /api/agents/:id/memory, audited as agent.memory_set, and traced on
every skip.

High-volume mechanical agents pay recall as pure latency: it embeds the
full task input in-process and linear-scans the agent's bank, and every
auto-retained turn grows that bank (a CRM backfill left 17k units, with
recall p90 at 113s under 16-way concurrency vs 0.5–1s idle — 45% of all
task time). The retain pipeline also runs minutes of background embed +
model calls per completed task, competing with live turns.
Concurrent turns can discover the same person under different keys (one
with an email, one name-only from a body mention); no row-level token or
UNIQUE constraint can arbitrate that — there is no shared row yet. Teach
the skill to match on whatever identity it has (first-name-only lookup,
same company/context ⇒ same person) and update instead of inserting a
thinner twin.
@Sheldenshi
Sheldenshi marked this pull request as draft July 18, 2026 04:57
@Sheldenshi
Sheldenshi force-pushed the Sheldenshi/port-local-architecture branch from 72f2aec to 147704a Compare July 18, 2026 05:15
@Sheldenshi
Sheldenshi marked this pull request as ready for review July 18, 2026 05:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants