Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/skills/pg-durable-sql/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Generate correct, idiomatic pg_durable durable function SQL using the `df.*` sch
4. **Operators are SQL-level custom operators.** They work on `TEXT` operands. Parentheses control grouping.
5. **`df.setvar()` must be called BEFORE `df.start()`.** Variables are captured at start time and are immutable during execution.
6. **Two variable syntaxes:** `{varname}` for durable function variables (from `df.setvar`), `$name` for result captures (from `|=>`). Do NOT mix them up.
7. **Operators live in the `df` schema.** They are resolved in the calling session, so `df` must be on the `search_path` (e.g. `SET search_path TO "$user", public, df;`) for the unqualified operator syntax to work. Functions (`df.seq`, `df.join`, …) are always schema-qualified and work regardless.

## Operators — Complete Reference

Expand Down
2 changes: 2 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ pub const NAME: &str = "pg_durable::activity::execute-sql";
### DSL Creates Graph Nodes
DSL functions like `df.sql()` insert rows into `df.nodes`. The `Durofut` struct represents a node reference passed between operators.

The DSL operators (`~>`, `|=>`, `&`, `|`, `?>`, `!>`, `@>`) are created in the `df` schema (see `extension_sql!` in [src/lib.rs](../src/lib.rs)). They are resolved in the caller's session before `df.start()`/`df.explain()` run, so `df` must be on the session `search_path` for unqualified operator syntax. The E2E runners set this at the database level after `CREATE EXTENSION`.

### E2E Test Structure
Tests in `tests/e2e/sql/` follow this pattern:
1. Create temp state table, call `df.start()`
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@

Pre-1.0 note: while `pg_durable` is in major version `0`, minor releases may include breaking changes.

## [0.2.3] - unreleased

### Breaking Changes

- **DSL operators moved from `public` to the `df` schema (#202):** The seven DSL operators (`~>`, `|=>`, `&`, `|`, `?>`, `!>`, `@>`) are now created in the `df` schema instead of `public`, so they no longer pollute the public namespace (resolving a pgspot PS017 finding). Because operators are resolved in the calling session before `df.start()`/`df.explain()` run, the unqualified operator syntax now requires `df` on the session `search_path` (e.g. `SET search_path TO "$user", public, df;`, or `ALTER ROLE`/`ALTER DATABASE ... SET search_path`). The `df.*` function forms (`df.seq`, `df.join`, …) are unaffected. Existing installs move the operators when they run `ALTER EXTENSION pg_durable UPDATE`; a non-upgraded `.so` keeps the operators in `public` and continues to work unchanged.

### Changed

- **`df.grant_usage()` now manages `search_path` (#202):** `df.grant_usage('role')` adds `df` to the target role's `search_path` (via `ALTER ROLE`) by default, so the unqualified DSL operators resolve without manual setup. This adds a fourth optional parameter, `set_search_path boolean DEFAULT true` (signature is now `df.grant_usage(text, boolean, boolean, boolean)`); pass `set_search_path => false` to opt out. `df.revoke_usage('role')` removes that `df` entry again (idempotent; other `search_path` entries are preserved). If the caller lacks privilege to alter the role, a `NOTICE` is raised and the grant/revoke otherwise succeeds.

## [0.2.2] - 2026-05-28

First open-source release of `pg_durable` on GitHub under the PostgreSQL License.
Expand Down
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,13 +94,18 @@ The model is intentionally SQL-shaped. If a step needs arbitrary code, a non-HTT
## Quick Example

```sql
-- Add the df schema to your search_path so the DSL operators resolve
SET search_path TO "$user", public, df;

-- A durable function that processes data in steps
SELECT df.start(
'SELECT id FROM documents WHERE processed = false LIMIT 100' |=> 'batch'
~> 'UPDATE documents SET processed = true WHERE id = ANY($batch)'
);
```

> The DSL operators (`~>`, `|=>`, `&`, `|`, `?>`, `!>`, `@>`) live in the `df` schema and resolve in your session, so `df` must be on your `search_path`. `df.grant_usage('role')` adds it for a role automatically (the `SET` above is for an ad-hoc session). See the [User Guide](USER_GUIDE.md#enable-the-extension) for details.

## Packages

Tagged releases publish Debian packages for PostgreSQL 17 and 18 on amd64 from the GitHub release assets. Packages are named `pg-durable-postgresql-<PG major>_<pg_durable version>-1_<arch>.deb` and install the extension library, control file, and SQL upgrade files into the matching PostgreSQL installation directories.
Expand Down Expand Up @@ -137,7 +142,7 @@ After installing a package, add `pg_durable` to `shared_preload_libraries`, rest
CREATE EXTENSION pg_durable;
```

The default pg_durable database is `postgres`; see [User Guide](USER_GUIDE.md) for background worker configuration and privilege setup.
The default pg_durable database is `postgres`; see [User Guide](USER_GUIDE.md) for background worker configuration and privilege setup. The unqualified DSL operator syntax needs the `df` schema on your `search_path` — `df.grant_usage('role')` adds it per role automatically, or set it yourself (e.g. `ALTER DATABASE postgres SET search_path = "$user", public, df;`).

Each release also publishes source archives for building from source and a `SHA256SUMS` file for verifying downloaded assets.

Expand Down
26 changes: 25 additions & 1 deletion USER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,21 @@ SELECT df.grant_usage('app_role');

After `CREATE EXTENSION`, the background worker initializes the engine schema asynchronously (normally within a few seconds). Until initialization completes, `df.*` functions will return: `"pg_durable background worker not yet initialized — try again in a moment"`. Simply retry after a short delay.

> ℹ️ **Using the DSL operators?** The DSL operators (`~>`, `|=>`, `&`, `|`, `?>`, `!>`, `@>`) live in the `df` schema, and they are resolved in your session *before* `df.start()`/`df.explain()` run, so `df` must be on your `search_path`. **`df.grant_usage('app_role')` handles this for you by default** — it adds `df` to the role's `search_path` (via `ALTER ROLE`) during onboarding, so the operators resolve the next time that role connects. Pass `set_search_path => false` to opt out and manage `search_path` yourself.
>
> To set it manually (or for the current session, before reconnecting):
>
> ```sql
> -- Per session
> SET search_path TO "$user", public, df;
>
> -- Or persist it for a role / database
> ALTER ROLE app_role SET search_path = "$user", public, df;
> ALTER DATABASE mydb SET search_path = "$user", public, df;
> ```
>
> Alternatively, schema-qualify each operator with `OPERATOR(df.~>)` syntax. Other `df.*` functions are always called schema-qualified and work without this.

> ⚠️ **Important**: If you include `pg_durable` in `shared_preload_libraries` but don't create the extension, the worker will remain idle and durable functions cannot execute.

### Your First Durable Function
Expand Down Expand Up @@ -197,6 +212,8 @@ df.sql('SELECT 1') ~> df.sql('SELECT 2')

### Operators

> **Note:** Operators live in the `df` schema. Using the unqualified syntax below requires `df` on your `search_path` (see [Enable the Extension](#enable-the-extension)). Without it, use the explicit `OPERATOR(df.~>)` form.

| Operator | Name | Description | Example |
|----------|------|-------------|---------|
| `~>` | Sequence | Run left, then right | `'SELECT 1' ~> 'SELECT 2'` |
Expand Down Expand Up @@ -1635,13 +1652,16 @@ SELECT df.grant_usage('admin_role', include_http => true, with_grant => true);

This function is purely additive — it never issues REVOKE. To downgrade a role's privileges (e.g., remove HTTP access), call `df.revoke_usage()` first, then `df.grant_usage()` with the desired options.

By default it also adds `df` to the role's `search_path` (see `set_search_path` below) so the unqualified DSL operators resolve — the only change it makes outside of privilege grants.

**Parameters:**

| Parameter | Default | Description |
|-----------|---------|-------------|
| `p_role` | *(required)* | Target role name |
| `include_http` | `false` | Grant EXECUTE on `df.http()` (opt-in — makes outbound network requests) |
| `with_grant` | `false` | Grant all privileges WITH GRANT OPTION and allow the role to call `df.grant_usage()` / `df.revoke_usage()` to manage other roles' access. The caller must hold each underlying privilege WITH GRANT OPTION (automatically true for superusers and delegated admins). |
| `set_search_path` | `true` | Add `df` to the role's `search_path` (via `ALTER ROLE`) so the unqualified DSL operators resolve without manual setup. Takes effect the next time the role connects; append-only and idempotent. Pass `false` to manage `search_path` yourself. If the caller lacks privilege to alter the role, a `NOTICE` is raised and the grant otherwise succeeds. |

<details>
<summary>Equivalent manual grants (for reference)</summary>
Expand Down Expand Up @@ -1695,6 +1715,10 @@ GRANT SELECT ON df.nodes TO app_role;
GRANT INSERT (id, label, root_node, submitted_by, database) ON df.instances TO app_role;
GRANT INSERT (id, instance_id, node_type, query, result_name, left_node, right_node, submitted_by, database) ON df.nodes TO app_role;
GRANT SELECT, INSERT, UPDATE, DELETE ON df.vars TO app_role;

-- Add df to the role's search_path so the unqualified DSL operators resolve
-- (df.grant_usage does this too, unless set_search_path => false)
ALTER ROLE app_role SET search_path = "$user", public, df;
```

</details>
Expand Down Expand Up @@ -1744,7 +1768,7 @@ To remove a role's access to pg_durable:
SELECT df.revoke_usage('app_role');
```

This revokes all privileges previously granted by `df.grant_usage()`. As a safety measure, `df.revoke_usage()` prevents revoking privileges from a role the caller is a member of (including the caller's own role).
This revokes all privileges previously granted by `df.grant_usage()`, and also removes the `df` entry that `df.grant_usage()` added to the role's `search_path` (idempotent — a no-op if `df` was never added or was already removed; any other entries are preserved). As a safety measure, `df.revoke_usage()` prevents revoking privileges from a role the caller is a member of (including the caller's own role).

For non-superusers, `df.revoke_usage()` is still subject to PostgreSQL's normal grantor rules because it is a SECURITY INVOKER helper. In practice, that means a delegated admin can only revoke the privileges that delegated admin granted; removing grants made by another role requires the original grantor or a superuser.

Expand Down
17 changes: 10 additions & 7 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -303,38 +303,41 @@ This allows users to write `'SELECT 1' ~> 'SELECT 2'` instead of `df.sql('SELECT

### SQL Operators

Operators are syntactic sugar that call DSL functions:
Operators are syntactic sugar that call DSL functions. They are created in the
`df` schema (alongside the functions they wrap), so callers need `df` on their
`search_path` to use the unqualified operator syntax — operators are resolved in
the calling session before `df.start()`/`df.explain()` run:

```sql
-- src/lib.rs (extension_sql!)

-- Sequence: a ~> b calls df.seq(a, b)
CREATE OPERATOR ~> (
CREATE OPERATOR df.~> (
FUNCTION = df.seq,
LEFTARG = text,
RIGHTARG = text
);

-- Name result: a |=> 'name' calls df.as_op(a, name)
CREATE OPERATOR |=> (
CREATE OPERATOR df.|=> (
FUNCTION = df.as_op,
LEFTARG = text,
RIGHTARG = text
);

-- Parallel join: a & b calls df.join(a, b)
CREATE OPERATOR & (
CREATE OPERATOR df.& (
FUNCTION = df.join,
LEFTARG = text,
RIGHTARG = text
);

-- Conditional: cond ?> then !> else
CREATE OPERATOR ?> (FUNCTION = df.if_then_op, ...);
CREATE OPERATOR !> (FUNCTION = df.if_else_op, ...);
CREATE OPERATOR df.?> (FUNCTION = df.if_then_op, ...);
CREATE OPERATOR df.!> (FUNCTION = df.if_else_op, ...);

-- Loop prefix: @> body calls df.loop(body)
CREATE OPERATOR @> (FUNCTION = df.loop_prefix_op, RIGHTARG = text);
CREATE OPERATOR df.@> (FUNCTION = df.loop_prefix_op, RIGHTARG = text);
```

### Node Insertion
Expand Down
10 changes: 7 additions & 3 deletions docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Parameters marked with ✅ **Auto-wrap** accept either:

Parameters marked with ❌ **Literal** expect a literal value (not auto-wrapped).

> **Operators and `search_path`:** The operator forms documented below (`~>`, `|=>`, `&`, `|`, `?>`, `!>`, `@>`) are defined in the `df` schema and are resolved in the calling session, so `df` must be on your `search_path`. `df.grant_usage('role')` adds it per role automatically; you can also set it yourself (e.g. `SET search_path TO "$user", public, df;`) or call the equivalent `df.*` function form (always schema-qualified).

---

## Node Functions
Expand Down Expand Up @@ -418,9 +420,9 @@ SELECT df.clearvars();

## Administration Functions

### df.grant_usage(role_name [, include_http] [, with_grant])
### df.grant_usage(role_name [, include_http] [, with_grant] [, set_search_path])

Grants the privileges a role needs to use pg_durable. By default this grants general `df` usage but does not grant `EXECUTE` on `df.http()`. Pass `include_http => true` to opt a role into HTTP access. Pass `with_grant => true` to allow the role to delegate access to others.
Grants the privileges a role needs to use pg_durable. By default this grants general `df` usage but does not grant `EXECUTE` on `df.http()`. Pass `include_http => true` to opt a role into HTTP access. Pass `with_grant => true` to allow the role to delegate access to others. By default it also adds `df` to the role's `search_path` so the unqualified DSL operators resolve; pass `set_search_path => false` to opt out.

Authorization is enforced by PostgreSQL’s native mechanisms: EXECUTE on this function is revoked from PUBLIC (so only roles explicitly granted access can call it), and the inner GRANT statements run as the caller via SECURITY INVOKER, so the caller must hold the underlying privileges WITH GRANT OPTION.

Expand All @@ -429,16 +431,18 @@ Authorization is enforced by PostgreSQL’s native mechanisms: EXECUTE on this f
| `role_name` | TEXT | The role to grant privileges to |
| `include_http` | BOOLEAN | Optional, defaults to `false`; when `true`, also grants `EXECUTE` on `df.http(text, text, text, jsonb, integer)` |
| `with_grant` | BOOLEAN | Optional, defaults to `false`; when `true`, grants all privileges WITH GRANT OPTION and retains EXECUTE on `df.grant_usage` / `df.revoke_usage` |
| `set_search_path` | BOOLEAN | Optional, defaults to `true`; when `true`, adds `df` to the role's `search_path` (via `ALTER ROLE`) so the unqualified DSL operators resolve. Takes effect on the role's next connection; append-only and idempotent. A `NOTICE` is raised (and the grant otherwise succeeds) if the caller cannot alter the role |

```sql
SELECT df.grant_usage('app_role');
SELECT df.grant_usage('app_role', include_http => true);
SELECT df.grant_usage('admin_role', with_grant => true);
SELECT df.grant_usage('app_role', set_search_path => false); -- manage search_path yourself
```

### df.revoke_usage(role_name)

Revokes all privileges previously granted by `df.grant_usage()`, including any `df.http()` access. Authorization is enforced the same way as `df.grant_usage()` — EXECUTE is revoked from PUBLIC, and the inner REVOKE statements run as the caller. On upgraded installs, revoking `df.http()` from `PUBLIC` is still a separate manual step.
Revokes all privileges previously granted by `df.grant_usage()`, including any `df.http()` access, and removes the `df` entry `df.grant_usage()` added to the role's `search_path` (idempotent; other entries are preserved). Authorization is enforced the same way as `df.grant_usage()` — EXECUTE is revoked from PUBLIC, and the inner REVOKE statements run as the caller. On upgraded installs, revoking `df.http()` from `PUBLIC` is still a separate manual step.

| Parameter | Type | Description |
|-----------|------|-------------|
Expand Down
2 changes: 2 additions & 0 deletions docs/grammar.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ From highest to lowest binding:
| 5 | `?>` `!>` | right | Conditional (if-then-else) |
| 6 (lowest) | `@>` | prefix | Loop (forever) |

> **Schema:** These operators are defined in the `df` schema. They are resolved in the calling session, so `df` must be on your `search_path` (e.g. `SET search_path TO "$user", public, df;`) to use the unqualified syntax shown below. The `df.*` function forms are always schema-qualified and need no `search_path` change.

## Examples

### Sequence
Expand Down
Loading
Loading