Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,17 +28,20 @@ Packages join GemStack by **graduating**, one at a time, when they prove framewo
- A `vike-*` package moves here only if a genuinely agnostic *core* falls out of it that is useful beyond its framework. In that case the core graduates (e.g. `@gemstack/<core>`) while the framework binding stays `vike-*`.
- Because both repos are co-governed and the `vike-*` set sits in the Vike orbit, any such move is decided with the Vike team, when there is brand traction, not unilaterally.

### Graduation candidates already in `vike-data`
### The data layer (graduated — Phase 0, pending brand confirmation)

An audit of `vike-data` shows the agnostic engines are not in the `vike-*` packages (those are bindings) but in the `universal-*` packages, which already carry **zero Vike imports**. These are the real candidates, in priority order after `@gemstack/ai-sdk`:
An audit of `vike-data` showed the agnostic engines are not in the `vike-*` packages (those are bindings) but in the `universal-*` packages, which carry **zero Vike imports**. The data layer is the second graduation after `@gemstack/ai-sdk`:

| Candidate (today) | Would become | Notes |
| Source (in `vike-data`) | Graduated to | Notes |
|---|---|---|
| `@universal-orm/core` (+ `@universal-orm/drizzle` / `/memory` / `/rudder`) | `@gemstack/orm` (+ adapters) | The ORM analog of `@gemstack/ai-sdk`. Mature, clearly agnostic. The strongest next candidate; move the core + its adapter family together. |
| `@vike-data/universal-schema` | `@gemstack/schema` | "Usable standalone by any framework or ORM." Agnostic, but currently mis-scoped under `@vike-data`. |
| `@vike-data/kit` | (stays) | Agnostic primitives (`createPort`), but it is the kit for *authoring bindings*, so it belongs with the binding ecosystem, not the engine umbrella, unless GemStack later wants a shared-primitives package. |
| `@universal-orm/core` (+ `/drizzle` / `/memory`) | `@gemstack/orm` (+ `orm-drizzle` / `orm-memory`) | The ORM analog of `@gemstack/ai-sdk`. Mature, clearly agnostic. Core + its adapter family graduated together. |
| `@vike-data/universal-schema` | `@gemstack/schema` | Agnostic, was mis-scoped under `@vike-data`. **Graduated with the ORM** (not later): the ORM test suite builds fixtures with `defineSchema`/`mergeSchemas`, so they cannot split. Still an early-experiment API. |
| `@universal-orm/rudder` | (decision pending) | The one **framework-coupled** adapter (peer `@rudderjs/database`). Excluded from the agnostic move; it is the "living binding" question — graduate as `@gemstack/orm-rudder` or keep it in the Rudder orbit. |
| `@vike-data/kit` | (stays) | Agnostic primitives (`createPort`), but it is the kit for *authoring bindings*, so it belongs with the binding ecosystem, not the engine umbrella. |

Realized fully, GemStack is the unified home for agnostic engines: `@gemstack/ai-sdk` (AI), `@gemstack/orm` (data), `@gemstack/schema` (schema).
**Status:** copied + renamed and green in the gemstack workspace (Phase 0 spike, #65, on `main` as `private` packages). The packages are **not yet published**, and the move is **gated on the brand-consolidation decision** below (#66): the `@gemstack/*` names are provisional until the Vike team confirms consolidation vs keeping `@universal-orm` as a parallel brand. Publishing, the re-export shims at the old names, and repointing dependents all wait on that confirmation.

Now realized, GemStack is the unified home for agnostic engines: `@gemstack/ai-sdk` (AI), `@gemstack/orm` (data), `@gemstack/schema` (schema).

### The open brand-consolidation question (for the Vike team)

Expand Down
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ Full documentation lives in [`docs/`](./docs/guide/index.md) (a hosted site is o
| [`ai‑autopilot`](./packages/ai-autopilot) | Orchestration: a Supervisor that plans, dispatches subagents (bounded concurrency + budget guardrails), and synthesizes the result. | [Guide](./docs/packages/ai-autopilot.md) | [![npm](https://img.shields.io/npm/v/@gemstack/ai-autopilot)](https://www.npmjs.com/package/@gemstack/ai-autopilot) |
| [`ai‑mcp`](./packages/ai-mcp) | The agent/MCP bridge: consume a remote MCP server's tools as agent tools, and expose an agent as an MCP server. | [Guide](./docs/packages/ai-mcp.md) | [![npm](https://img.shields.io/npm/v/@gemstack/ai-mcp)](https://www.npmjs.com/package/@gemstack/ai-mcp) |
| [`mcp`](./packages/mcp) | A standalone framework for *authoring* MCP servers: tools, resources, prompts, decorators, OAuth 2.1, a framework-neutral HTTP handler, and a test client. Agent-agnostic. | [Guide](./docs/packages/mcp.md) | [![npm](https://img.shields.io/npm/v/@gemstack/mcp)](https://www.npmjs.com/package/@gemstack/mcp) |
| [`orm`](./packages/orm) | The data engine: a narrow, ORM-free repository (`db.users.upsert(...)`) over a composed schema, plus the adapter contract and a one-adapter registry. The data-layer twin of `ai-sdk`. | [Guide](./docs/packages/orm.md) | [![npm](https://img.shields.io/npm/v/@gemstack/orm)](https://www.npmjs.com/package/@gemstack/orm) |
| [`schema`](./packages/schema) | The shape engine: declare tables once as plain data, merge contributions, derive migrations, compile to Prisma / Drizzle / Rudder. **Preview.** | [Guide](./docs/packages/schema.md) | [![npm](https://img.shields.io/npm/v/@gemstack/schema)](https://www.npmjs.com/package/@gemstack/schema) |
| [`orm‑memory`](./packages/orm-memory) | In-process `Map` adapter for `orm` — tests, demos, zero-config dev. | [Guide](./docs/packages/orm.md#adapters) | [![npm](https://img.shields.io/npm/v/@gemstack/orm-memory)](https://www.npmjs.com/package/@gemstack/orm-memory) |
| [`orm‑drizzle`](./packages/orm-drizzle) | Drizzle adapter for `orm` — real databases. | [Guide](./docs/packages/orm.md#adapters) | [![npm](https://img.shields.io/npm/v/@gemstack/orm-drizzle)](https://www.npmjs.com/package/@gemstack/orm-drizzle) |

### How they fit together

Expand All @@ -29,10 +33,17 @@ Full documentation lives in [`docs/`](./docs/guide/index.md) (a hosted site is o
@gemstack/ai-mcp agent <-> MCP bridge (the "adapter") -> ai-sdk
-----------------------------------------------------------------------------------
@gemstack/mcp standalone MCP server framework agent-agnostic, not ai-*
-----------------------------------------------------------------------------------
@gemstack/schema data shape: define tables, merge, derive migrations (preview)
@gemstack/orm runtime data access over a composed schema
@gemstack/orm-memory in-process Map adapter (tests/demos) -> orm
@gemstack/orm-drizzle Drizzle adapter (real databases) -> orm
```

The `ai-` prefix means **"depends on the agent runtime."** `skills`, `autopilot`, and `ai-mcp` all depend on `ai-sdk`; `ai-sdk` depends on none of them, and nothing depends "up." A package about AI that is agent-agnostic (like `@gemstack/mcp`) is a peer of the family, not a member of it.

The **data family** (`orm` + `schema` + adapters) is a second engine family — framework-agnostic data access, parallel to the AI family and independent of it. `orm` is the runtime (read/write without importing an ORM), `schema` is the shape (declare tables once, compile to any ORM), and `orm-memory` / `orm-drizzle` are adapters that bind the repository to a real backend.

See [`Architecture.md`](./Architecture.md) for the full layering, naming rule, and graduation policy.

### Which MCP package do I use?
Expand Down
11 changes: 10 additions & 1 deletion docs/.vitepress/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ export default defineConfig({
{ text: 'ai-autopilot', link: '/packages/ai-autopilot' },
{ text: 'ai-mcp', link: '/packages/ai-mcp' },
{ text: 'mcp', link: '/packages/mcp' },
{ text: 'orm', link: '/packages/orm' },
{ text: 'schema', link: '/packages/schema' },
],
},
{
Expand Down Expand Up @@ -83,14 +85,21 @@ export default defineConfig({
],
},
{
text: 'The family',
text: 'The AI family',
items: [
{ text: 'ai-skills', link: '/packages/ai-skills' },
{ text: 'ai-autopilot', link: '/packages/ai-autopilot' },
{ text: 'ai-mcp', link: '/packages/ai-mcp' },
{ text: 'mcp', link: '/packages/mcp' },
],
},
{
text: 'The data family',
items: [
{ text: 'orm — the data engine', link: '/packages/orm' },
{ text: 'schema — the shape engine', link: '/packages/schema' },
],
},
],
},

Expand Down
10 changes: 9 additions & 1 deletion docs/guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ All packages publish under the **`@gemstack/`** scope.
| [`ai-autopilot`](/packages/ai-autopilot) | Orchestration: a Supervisor that plans, dispatches subagents (bounded concurrency + budget guardrails), and synthesizes the result. |
| [`ai-mcp`](/packages/ai-mcp) | The agent/MCP bridge: consume a remote MCP server's tools as agent tools, and expose an agent as an MCP server. |
| [`mcp`](/packages/mcp) | A standalone framework for *authoring* MCP servers: tools, resources, prompts, decorators, OAuth 2.1, a framework-neutral HTTP handler, and a test client. Agent-agnostic. |
| [`orm`](/packages/orm) | The data engine: a narrow, ORM-free repository (`db.users.upsert(...)`) over a composed schema, plus the adapter contract and a one-adapter registry. The data-layer twin of `ai-sdk`. |
| [`schema`](/packages/schema) | The shape engine: declare tables once as plain data, merge contributions, derive migrations, compile to Prisma / Drizzle / Rudder. **Preview.** |
| [`orm-memory`](/packages/orm#adapters) / [`orm-drizzle`](/packages/orm#adapters) | Adapters that bind the `orm` repository to a backend: in-process `Map`s (tests/demos) or Drizzle (real databases). |

## How they fit together

Expand All @@ -25,9 +28,14 @@ ai-autopilot orchestration / autonomy (the "director") -> ai-sdk (+ skills)
ai-mcp agent <-> MCP bridge (the "adapter") -> ai-sdk
-----------------------------------------------------------------------------------
mcp standalone MCP server framework agent-agnostic, not ai-*
-----------------------------------------------------------------------------------
schema data shape: define tables, merge, derive migrations (preview)
orm runtime data access over a composed schema
orm-memory in-process Map adapter (tests/demos) -> orm
orm-drizzle Drizzle adapter (real databases) -> orm
```

`ai-sdk` is the foundation: it owns the single-agent loop, tools, and streaming. `ai-skills` and `ai-autopilot` build on top of it. `ai-mcp` bridges an agent to the Model Context Protocol. `mcp` stands apart - it is for *authoring* MCP servers and knows nothing about agents.
`ai-sdk` is the foundation of the **AI family**: it owns the single-agent loop, tools, and streaming; `ai-skills` and `ai-autopilot` build on top of it, and `ai-mcp` bridges an agent to the Model Context Protocol. `mcp` stands apart - it is for *authoring* MCP servers and knows nothing about agents. The **data family** is a second, independent set of engines: `orm` reads and writes without importing an ORM, `schema` declares tables once and compiles to any ORM, and the adapters bind the repository to a real backend.

## Design principles

Expand Down
16 changes: 16 additions & 0 deletions docs/packages/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ All packages publish under the **`@gemstack/`** scope (e.g. `npm install @gemsta
| [`ai-autopilot`](/packages/ai-autopilot) | Orchestration: a Supervisor that plans, dispatches subagents (bounded concurrency + budget guardrails), and synthesizes the result. |
| [`ai-mcp`](/packages/ai-mcp) | The agent/MCP bridge: consume a remote MCP server's tools as agent tools, and expose an agent as an MCP server. |
| [`mcp`](/packages/mcp) | A standalone framework for *authoring* MCP servers: tools, resources, prompts, decorators, OAuth 2.1, a framework-neutral HTTP handler, and a test client. Agent-agnostic. |
| [`orm`](/packages/orm) | The data engine: a narrow, ORM-free repository (`db.users.upsert(...)`) over a composed schema, plus the adapter contract and a one-adapter registry. The data-layer twin of `ai-sdk`. |
| [`schema`](/packages/schema) | The shape engine: declare tables once as plain data, merge contributions, derive migrations, compile to Prisma / Drizzle / Rudder. **Preview.** |
| [`orm-memory`](/packages/orm#adapters) | In-process `Map` adapter for `orm` — tests, demos, zero-config dev. |
| [`orm-drizzle`](/packages/orm#adapters) | Drizzle adapter for `orm` — real databases. |

## How they fit together

Expand All @@ -19,8 +23,20 @@ ai-autopilot orchestration / autonomy (the "director") -> ai-sdk (+ skills)
ai-mcp agent <-> MCP bridge (the "adapter") -> ai-sdk
-----------------------------------------------------------------------------------
mcp standalone MCP server framework agent-agnostic, not ai-*
-----------------------------------------------------------------------------------
schema data shape: define tables, merge, derive migrations (preview)
orm runtime data access over a composed schema
orm-memory in-process Map adapter (tests/demos) -> orm
orm-drizzle Drizzle adapter (real databases) -> orm
```

### Two engine families

GemStack has two independent engine families plus the standalone MCP framework:

- The **AI family** (`ai-sdk` + `ai-skills` / `ai-autopilot` / `ai-mcp`) — everything for running agents.
- The **data family** (`orm` + `schema` + adapters) — framework-agnostic data access: `orm` reads and writes without importing an ORM, `schema` declares tables once and compiles to any ORM. Parallel to the AI family, and independent of it — adopt either on its own.

### Two MCP packages, two jobs

`ai-mcp` and `mcp` both touch the Model Context Protocol, but from opposite ends:
Expand Down
108 changes: 108 additions & 0 deletions docs/packages/orm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# orm

`@gemstack/orm` is the **runtime** half of the data layer: how code reads and writes its data **without importing an ORM**. It is the runtime twin of [`schema`](/packages/schema) (the shape half) — you declare your tables once with the schema DSL, then talk to a neutral repository, and a per-ORM **adapter** executes the calls against the real database.

```bash
npm install @gemstack/orm
```

Zero framework imports, zero ORM imports. Usable standalone by any framework or ORM.

## The surface is narrow on purpose

```js
db.users.insert(row) // -> inserted row
db.users.find(filter) // -> matching rows (array)
db.users.findOne(filter) // -> first match | null
db.users.upsert(row, { onConflict }) // -> upserted row
db.users.update(filter, patch) // -> updated rows (array)
db.users.delete(filter) // -> number of rows deleted
```

Filters are simple **equality** or **`in`** conditions — nothing more:

```js
db.users.find({ active: true }) // equality
db.users.find({ id: { in: ['u1', 'u2'] } }) // membership
db.users.find() // all rows
```

Joins, aggregates, ranges, raw SQL — deliberately **out of scope**. Drop to the underlying ORM for those, the same escape hatch as DB-specific column types. This is not a query language (that is Kysely's job); it is the 90%-case repository.

## Using it

```js
import { createRepository } from '@gemstack/orm'
import { defineSchema, mergeSchemas } from '@gemstack/schema'

const { tables } = mergeSchemas([
defineSchema('users', (t) => {
t.uuid('id').primary()
t.string('email').unique()
t.boolean('active')
}),
])

const db = createRepository({ tables }, adapter) // adapter: see below
await db.users.upsert({ id: 'u1', email: 'a@b.com', active: true }, { onConflict: 'email' })
```

Tables and their columns come from the **merged schema** (the output of `mergeSchemas`), the same single source the ORM artifacts are generated from. A typo'd column or an unknown table is a clear error, not a silent no-op.

## The adapter contract

The app installs **one** adapter and hands it the connection; consumers never import an ORM. An adapter implements six operations, each taking the table **name** first:

```js
const adapter = {
insert(table, row), // -> inserted row
find(table, filter, opts), // -> rows[] (opts: { limit, offset, orderBy })
count(table, filter), // -> number of matching rows
upsert(table, row, { onConflict }), // -> upserted row (onConflict: column names)
update(table, filter, patch), // -> updated rows[]
delete(table, filter), // -> number deleted
}
```

`findOne` is not an adapter op; the repository derives it from `find` (with `limit: 1`).

In-process adapters can reuse the shared filter matcher so every adapter agrees on what a filter means; SQL adapters translate the same shape into a `WHERE` clause:

```js
import { matchesFilter } from '@gemstack/orm'
matchesFilter(row, { active: true })
```

> No transactions yet: the common operation is a single (atomic) upsert.

## The adapter registry: one adapter, every consumer

So the **app picks the backend once** and everything routes through it, the core holds a tiny runtime registry. The app calls `setAdapter(...)` at server start; each consumer reads the same adapter via `getAdapter()` and builds its repository over its own schema. Nothing hardcodes a backend, and nothing imports an ORM.

```js
// the app, once at server start:
import { setAdapter } from '@gemstack/orm'
import { createDrizzleAdapter } from '@gemstack/orm-drizzle'
setAdapter(createDrizzleAdapter(drizzle(pool), schema)) // or createMemoryAdapter()

// a consumer, anywhere:
import { getAdapter, createRepository } from '@gemstack/orm'
import { createMemoryAdapter } from '@gemstack/orm-memory'
const adapter = getAdapter() ?? createMemoryAdapter() // app's choice, else zero-config memory
const db = createRepository({ tables }, adapter)
```

`getAdapter()` returns `null` until the app sets one, so a consumer falls back to the memory adapter for zero-config dev/demo. `setAdapter` validates the six-op contract up front, and the registry is cached on `globalThis` so pointer-import / HMR double-eval can't fork it. `clearAdapter()` resets it (tests).

## Adapters

The repository is backend-agnostic; an adapter binds it to a real store. Two ship today:

| Package | Backend | Use it for |
|---|---|---|
| [`orm-memory`](/packages/orm) (`@gemstack/orm-memory`) | plain in-process `Map`s | tests, demos, zero-config dev — no database |
| [`orm-drizzle`](/packages/orm) (`@gemstack/orm-drizzle`) | [Drizzle](https://orm.drizzle.team) over your connection | real databases (Postgres, etc.) |

`orm-memory` reuses the core filter matcher, so it agrees with SQL adapters on what a filter means — the proof that the contract, not the backend, defines behavior. `orm-drizzle` declares `drizzle-orm` as a peer dependency.

Authoring your own adapter is just implementing the six-op contract above.
Loading
Loading