feat(sqlite): wire function registry and array operator support#50
Merged
Conversation
…QLite compat
Add four capabilities needed for the full 615-test postgrest-compat suite
to pass against dbrest+SQLite:
- rpc.ParseRegistry: parse a JSON function declaration array into a
StaticRegistry so non-NativeRPC backends can expose /rpc/<fn> endpoints
without a stored-procedure catalog.
- cmd/dbrest/main.go: wire DBREST_FUNCTION_REGISTRY (or PGRST_FUNCTION_REGISTRY)
config field — read the JSON, parse it, and Register it on backends that
implement the Register(rpc.Registry) interface.
- sqlgen.Dialect.ArrayLiteral: new method converts a PostgreSQL {a,b} array
literal to the engine's native format before it is bound as a parameter.
Postgres/MySQL/SQL Server pass through unchanged; SQLite converts to a
JSON ["a","b"] array so json_each() can iterate over it.
- sqlite.ArrayOp: replace the no-op stub with json_each()-based @>, <@, &&
implementations. sqlite.result.go: BOOLEAN columns coerce int64 0/1 to
Go bool; JSON columns return json.RawMessage so the value is embedded
verbatim rather than double-encoded as a string.
…n alias - Open() enables PRAGMA foreign_keys = ON via SetMaxOpenConns(1) so FK constraint violations surface as 409 rather than silent 201 - drain() now calls ColumnTypes() and applies the same BOOLEAN→bool and JSON→json.RawMessage coercions that rowStream.Values() already does; write-path responses (UPDATE/INSERT with return=representation) were returning raw int64 for boolean columns - writeSelect alias condition now also fires when a cast is present (name == lastPath but expr != bare column); SQLite returns the full expression string as the column name when no AS alias is given, so done::text was keyed "CAST(done AS TEXT)" instead of "done"
Some tests PUT rows with PostgreSQL array literal syntax ({a,b}) into
JSON columns. Wrapping a non-JSON string as json.RawMessage causes the
encoder to return a 500 when marshaling the response. Guard both drain()
and rowStream.Values() with json.Valid so only well-formed JSON values
are embedded verbatim; the rest fall through as plain strings.
Lint: - Fix gofmt alignment in rpc/registry.go fnDecl struct tag columns - Fix gofmt alignment in backend/sqlgen/compile_test.go stub methods Conformance (sqlite) + Test race: - Extend Dialect.ArrayOp to accept colType so the dialect can decide whether the column supports array semantics - Enrich ir.Compare.ColumnType in plan/plan.go for array ops (same pattern as FullText index attachment for FTS) - SQLite ArrayOp: return ok=false for non-JSON column types so the compiler raises PGRST127; json_each only works on JSON-typed columns; TEXT/INTEGER/etc. now correctly return 400 instead of 500 - Update all dialect implementations (postgres/mysql/sqlserver/compile_test) to match new 4-arg ArrayOp signature
ArrayOp now implements @>, <@, and && for JSON columns using MySQL 8.0.17+
functions. For non-JSON columns it returns false so the compiler raises
PGRST127, matching the SQLite behaviour and the conformance allowlist.
ArrayLiteral converts the PostgreSQL {a,b} format to ["a","b"] so the JSON
functions receive a valid JSON array argument.
When NativeRPC=true the plan carries no portable registry function (plan.Func==nil). CompileCall panics on nil fn; introduce compileNativeCall that emits EXEC [dbo].[name] @arg=@pn instead. Guard count=exact path against nil fn as well: T-SQL cannot wrap EXEC in SELECT count(*), so count is skipped for native procs.
IsBool: add Dialect.IsBool hook so SQL Server generates col = 1/0
instead of the invalid col IS 1/0 (IS only accepts NULL in T-SQL).
LimitOffset: return OFFSET 0 ROWS when hasOrder=true and no
paging was requested, making ORDER BY valid inside derived tables.
JSONAgg: replace JSON_ARRAYAGG (SQL Server 2025 only) with
'['+STRING_AGG(CAST(elem AS NVARCHAR(MAX)),',')+']' which works
on SQL Server 2022.
ArrayOp/ArrayLiteral: implement @>, <@, && via OPENJSON; convert
{a,b} PostgreSQL array literals to JSON arrays for OPENJSON input.
embed.go LIMIT 1: replace the hardcoded LIMIT 1 clause with a
dialect-aware LimitOffset call so SQL Server emits OFFSET 0 ROWS
FETCH NEXT 1 ROWS ONLY instead of the invalid T-SQL LIMIT.
executeUpsert: add multi-statement UPDATE … ; IF @@rowcount=0
INSERT … batch inside the request transaction, routing ir.Upsert
queries away from the single-statement compiler path that returns
errUpsertMultiStatement. Rows are read back via a post-batch SELECT
on the conflict key when returning columns are requested.
ArrayLiteral now passes through elements that are already JSON-quoted
(start and end with ") rather than wrapping them again. The postgrest-go
client sends {"go"} with the element double-quoted per PG array literal
syntax, and the old code produced ["\"go\""] instead of ["go"].
executeUpsert is rewritten as a single MERGE statement. The previous
UPDATE; IF @@rowcount=0 INSERT pattern uses a semicolon-separated batch
that go-mssqldb rejects inside sp_executesql. MERGE is a single
statement, takes all rows in the source VALUES table, and includes an
OUTPUT clause when returning columns are requested.
…trospection asMSSQLError used *mssql.Error as target for errors.As but mssql.Error implements error via a value receiver; switch to value target so constraint violations (547=FK, 2627/2601=unique, 2812=proc not found) are correctly mapped to 409/422/404 instead of falling through to 500. Add schema.Column.Identity to track auto-generated identity columns. SQL Server introspection now fetches COLUMNPROPERTY IsIdentity alongside the existing column query. executeUpsert enables SET IDENTITY_INSERT ON before the MERGE when any conflict column is an identity column, allowing explicit id values to be upserted without error 8101.
Three fixes for the MySQL compat job: 1. IsBool now returns "col = 1/0" instead of falling back to "col IS 1". MySQL 8 only accepts IS NULL/UNKNOWN/TRUE/FALSE, not integer literals. 2. normalizeArgs converts ISO 8601 strings (e.g. "2024-01-01T00:00:00Z") to time.Time before binding. MySQL's string-to-DATETIME cast rejects the T separator and Z timezone suffix; passing time.Time bypasses the cast. Applied to all CompileRead, CompileCount, compileWrite, and CompileCall statement args. 3. Open() sets ClientFoundRows=true so MySQL counts matched rows rather than changed rows. Without this, an UPDATE that sets the same values reports RowsAffected=0, which gates the re-select and returns empty results instead of the matched rows.
The re-select after UPDATE must use primary keys, not the original
filter. The original filter may reference a column being updated
(e.g. PATCH /todos?task=eq.old with body {task:new}), so after the
UPDATE the filter matches nothing and the re-select returns empty rows.
Fix: capture matching PKs before the UPDATE executes, then re-select
by those PKs after the UPDATE to get the post-mutation representation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four capabilities needed for the full 615-test postgrest-compat suite to pass against dbrest+SQLite:
rpc.ParseRegistry: decode a JSON function-declaration array into aStaticRegistry, so non-NativeRPC backends (SQLite, MySQL) can expose/rpc/<fn>without a stored-procedure catalogcmd/dbrest/main.go: wireDBREST_FUNCTION_REGISTRY/PGRST_FUNCTION_REGISTRYconfig field — parse the JSON andRegisterit on backends that accept a registrysqlgen.Dialect.ArrayLiteral: new method converts a PostgreSQL{a,b}array literal to the engine's native format before binding; SQLite converts to["a","b"]JSON sojson_each()can iterate; Postgres/MySQL/SQL Server pass through unchangedsqlite.ArrayOp: implement@>(contains),<@(contained-by),&&(overlaps) viajson_each()subqueries;sqlite.result.go: coerceBOOLEANcolumns fromint640/1 tobool, andJSONcolumns fromstringtojson.RawMessageTest plan
go test ./...passes (all existing unit tests green)go build ./...succeedstest-dbrest-sqlitejob passes all 615 client tests against dbrest+SQLite