m-iris: IRIS engine driver (M0+M1) on the shared m-driver-sdk#1
Merged
Conversation
Rename the irissync binary/module to m-iris and adopt the m engine-driver
contract (driver-contract.md v1.0), test-first throughout.
M0 (scaffold + SDK seam + meta):
- module github.com/vista-cloud-dev/m-iris, binary m-iris, env M_IRIS_*
- internal/driver: honest CapsDoc() (golden), ContractVersion, verb-level
Transport seam (Exec/Load/ReadGlobal/SetGlobal/Health) + FakeTransport
- clikit: contract exit ladder 0/2/3/4/5/6/7 + engineError envelope field (§7)
- axis command tree `m-iris <axis> <verb>`: meta (caps/info/version/schema)
+ sync (the regrouped source verbs)
Remote spike (risk B2 — the whole remote substrate):
- m_iris.Runner.cls: role-gated, parameterized SqlProc runner; faults captured
to ^mIrisRun(rid,"error") in §7 shape
- atelier.Query: POST {ns}/action/query (parameter-bound SQL)
- internal/remote.Transport (implements driver.Transport): lazy PUT+compile of
the runner, Exec run/eval with fault→EngineError, data set/get, Health probing
the action/query privilege
- unit tier green every commit (fake AtelierAPI); TestRemoteSpike_RealEngine
gated on M_IRIS_IT=1 + a provisioned IRIS CE container
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Atelier root probe foundation, the lifecycle axis, and the doctor preflight —
all on the remote (attach) transport, test-first against an httptest Atelier.
- atelier.ServerInfo: GET /api/atelier/v1/ → version/api/namespaces; typed
*HTTPError with IsUnauthorized/IsForbidden so 401 (bad credential) and 403
(no privilege) are distinct (risks C3, C7)
- --transport local|docker|remote flag (default remote; only remote wired)
- lifecycle axis (remote attach): status + --probe (CI gate, exit 0/6),
wait --timeout (poll → exit 6), up/down/restart; provision/destroy report
unsupported (exit 7) over Atelier — you cannot create/destroy a namespace
there, so conformance runs in attached mode (risk B4)
- meta doctor: typed matrix {name,ok,detail,fix}, exit 0/5/6 — reachable, auth,
version (>= 2022.1), namespace, query-privilege (action/query SELECT 1 proves
the runner privilege, risk C7), license (honestly not-probed on remote)
- caps grown to advertise lifecycle + meta.doctor (still honest)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adopt the extracted shared SDK; the frozen verb-level Transport now lives in
m-driver-sdk (package mdriver) instead of internal/driver.
- Delete internal/driver/{transport,fake,transport_test}.go (moved to the
SDK, incl. FakeTransport).
- caps.go: Caps map→mdriver struct (honest set unchanged); regenerate golden;
add UPDATE_GOLDEN support. meta.go/meta_test.go use Axes.Wired() + the
struct Features.
- internal/remote (Atelier-SQL runner substrate) retargeted to mdriver types;
readEngineError now returns *mdriver.EngineError (behavior unchanged).
- go.mod: require + local replace ../m-driver-sdk.
Reconciliation that landed in the SDK: SetGlobal kept, Compile dropped,
field-based Exec, GlobalNode tree, EngineError in the SDK. All race/vet/gofmt
clean; the gated TestRemoteSpike_RealEngine remains gated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switch from the local replace ../m-driver-sdk to the published module github.com/vista-cloud-dev/m-driver-sdk (pseudo-version, proxy+sumdb verified) now that the SDK is on GitHub, so CI resolves it standalone. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
doctor/lifecycle payloads now alias mdriver.{Check,DoctorResult,Status,
StateResult} so m-ydb and m-iris emit identical JSON. Aliases keep existing
literals/renderers/goldens unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First real-engine validation of m-iris (M0 remote spike + M1 status/doctor) against a disposable IRIS Community 2026.1 container. Four latent bugs that the fake-Atelier unit tier never caught, each now fixed + re-tested green: 1. Runner class name `m_iris.Runner` is INVALID — IRIS class/package names forbid underscores (#16006). Renamed to `m.iris.Runner`; package "m.iris" projects to SQL schema "m_iris", so all m_iris.* SQL call sites are unchanged. 2. Atelier error `code` is a JSON number on IRIS 2026.1 (client typed it string) → compile-response decode failed. Added errCode (accepts string or number). 3. Runner ObjectScript used spaces after commas → #1043 "QUIT argument not allowed" in `quit ..fault(rid, ex)`. Removed comma-whitespace; catch blocks now `do ..fault(...)` + fall through to the single return. 4. ServerInfo hit the version-prefixed root `/api/atelier/v1/` (404 on modern IRIS) → status/doctor unreachable. Switched to the unversioned `/api/atelier/` version-discovery root (present in every Atelier release). Validated GREEN vs IRIS 2026.1: remote spike (deploy runner, set/get/eval, structured EngineError), lifecycle status (running/healthy/version/namespaces), meta doctor (all checks). Added `make test-it` (gated real-engine tier). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-filter
Brings the sync axis to 8-verb parity with m-ydb (plan §5 task 6):
- sync diff <name> [--from DIR]: unified diff of the instance copy (GET,
Stat-gated so an absent routine diffs as pure add/del) vs the mirror or a
--from dir. {unified}. New internal/udiff (LCS, 3-line context) ported
byte-identical from m-ydb.
- sync rm <name>: DeleteDoc on the instance + remove mirror file + manifest
entry; honors --dry-run. {removed}. Already-absent is reported, not an error.
- push --from DIR: push routines from an arbitrary directory (incl. fresh
creates), staged into the mirror so the conflict-check / single-writer lock /
compile path runs unchanged; the up-to-date check reads the --from copy so
--dry-run is accurate.
- bare-name --filter: the glob matches the extension-stripped routine name
(DG*/DGREG select DGREG.mac; *.mac never matches), parity with m-ydb
source.Match and driver-contract §5.2.
- caps advertises all 8 sync verbs (honest gate; golden regenerated); meta
schema picks up diff/rm automatically.
Tests: unit tier (fake/rw Atelier) for diff (incl. missing-instance), rm
(dry-run + delete), push --from (dry-run + fresh create), and bare-name match.
Adds gated TestSyncAxis_RealEngine (M_IRIS_IT, make test-it) round-tripping an
ephemeral zzMIRISIT scratch routine; the verbs ride GET/PUT/DELETE doc +
docnames already validated against IRIS CE 2026.1 in M1 — the gated round-trip
was not run this session (container down, docker/curl sandbox-blocked).
go test -race / vet / gofmt clean. SDK unchanged (diff/rm/push results are
driver-local shapes already in contract §5.2); both drivers stay pinned to
m-driver-sdk v0.2.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… PutDoc rejection)
Running the new gated TestSyncAxis_RealEngine against live IRIS CE 2026.1 surfaced
two latent bugs in the Atelier client that the fake-transport unit tests could not:
1. Missing-doc detection — IRIS 2026.1 answers GET /doc/{name} for an absent
document with a bare HTTP 404 (older servers embedded "does not exist"/#5002 in
status.errors). isNotFound now also recognizes a 404 HTTPError, so Stat/DeleteDoc
treat an absent routine as not-found (exists=false) instead of a hard error.
Without this, push's conflict-check, sync diff, and sync rm all errored on any
routine not yet on the instance.
2. Silent PUT rejection — a save-time rejection (e.g. #16021 Illegal Header Line on
a modern .mac lacking a `ROUTINE name [Type=MAC]` header) comes back HTTP 200
with an EMPTY status.errors[] and the reason in the *per-document* result.status;
result.content is "" (string) not [] (array), so the old Doc unmarshal silently
failed and PutDoc reported success while the routine was never stored. PutDoc now
decodes result.status and returns an error when it is non-empty.
Regression guards added in internal/atelier/write_test.go (TestPutDocRejectedByStatus,
TestStatMissing404). The gated TestSyncAxis_RealEngine fixture now writes a valid
.mac with the ROUTINE header; the full push --from → diff → rm round-trip is GREEN
against IRIS 2026.1 (and m-ydb's real tier stays green against YottaDB r2.07).
go test -race / vet / gofmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…carve-outs) Driver-spike rules for m-iris (lane = push m-iris only; SDK pinned, never edited here; honest caps). Memory moves in-repo to ./docs/memory/ (recall symlinked here); step-2 tracker is ./docs/m-iris-tracker.md — both EXCEPTIONS to the global/org rules to keep parallel iris/ydb spikes from clashing on the docs repo. See docs/m-engine-drivers/coordination-model.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add irisdriver.New(Config) (mdriver.Transport, error), composing atelier.New +
remote.New so an external module (m-cli's VistaEngine) holds an IRIS Transport
without importing internal/. type Config = atelier.Config; construction is lazy
(runner deploys on first verb). Peer of m-ydb's ydbdriver facade — both drivers
now present a public constructor returning the neutral contract.
Live-validated vs m-test-iris (IRIS CE 2026.1, :52774) via a gated M_IRIS_IT=1
facade test: New -> Health -> Exec returns the real $ZV banner.
Documents the cross-engine capture rule: IRIS Exec captures the result-global
^mIrisRun(rid,"out"), not device W output (the runner xecutes with no IO
redirection), so the unified readiness/version probe is Health()+Version, not
Exec("W $ZV"). YottaDB captures session stdout directly.
Gates: go test -race ./..., go vet, gofmt green; facade IT green vs real IRIS.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…t,build})
m-driver-conformance flagged that `meta version` used the shared
clikit.VersionCmd, emitting {version,commit,date,go} — which lacks the
engine/contract that contract §5.7 requires (version = {driver, engine,
contract, build}). m-ydb already had a driver-specific version; m-iris was the
drift.
Replace the field with a driver-specific versionCmd emitting
{driver:"m-iris", engine:"iris", contract, build{version,commit,date,go}}.
clikit stays engine-agnostic and byte-identical across drivers (engine/contract
are driver facts, not clikit's). Conformance: 16/16 live vs m-test-iris (remote).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…w/ m-ydb) Mirror the shared clikit fix from m-ydb (kept byte-identical): add Context.ResultExit(data, exit, text) and have Run return cc.ExitCode(). meta doctor now uses ResultExit, so its unreachable path emits ok=false/exit=6 with process exit 6 — fixing the same latent cc.Result-then-Fail mismatch m-ydb had (driver-contract §2: envelope.exit == process exit). Conformance: 16/16 live vs m-test-iris (remote). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
doctor now emits its data envelope via cc.ResultExit and returns nil (the exit is recorded on the Context, not a returned *clikit.Error). Update the three doctor exit tests to assert cc.ExitCode() instead of the returned error. Fixes the red gate the previous commit introduced. go test -race ./... green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…5 IRIS driver-path Adds the neutral `exec load/run/eval` axis to m-iris (the gap that made `v pkg install --engine iris` silently no-op — the SDK reference Client shells `m-iris exec …`, which previously returned USAGE/exit 2). All additive in m-iris; SDK stays pinned v0.2.0. - exec.go + execCmd mounted as `Exec` in CLI; caps advertises exec (golden regen); load → remote.Transport.Load, run/eval → remote.Transport.Exec; IRIS fault → §7. - Load: map neutral `.m` → `.int` and prepend the UDL `ROUTINE <name> [Type=INT]` header Atelier requires (else #16021 Illegal Header Line) — caught only live. - Device-`W` capture: runner RunRef/Eval bracket execution with start^mIrisIO/ stop^mIrisIO (%Device.ReDirectIO + a companion mIrisIO.int whose wstr/wchr/wnl labels append to ^mIrisRun(rid,"out") — a class method can't host mnemonic-space labels). stop() never throws and restores the original mnemonic. - KIDS-over-Atelier recovery: EN^XPDIJ reconfigures the SQL-gateway device, losing the action/query response body (HTTP 200 + empty) though the run completes. So RunRef/Eval record status/out/error in ^mIrisRun(rid,*) and set "done" last; Exec recovers the outcome from those globals — Base64 via GetOut (control bytes survive), retrying on fresh connections (CloseIdleConnections) until a clean gateway process serves the read. Tests: exec_test.go (command tier), TestLoad_MapsDotMToIntDocname + #16021 fake guard, TestRemoteExecAxis_RealEngine (load→run→read-stdout). Gates green: race/gofmt/vet/lint, make test-it vs foia (sync/spike/exec RealEngine), SDK conformance 16/0. T0a.5 driver-path PROVEN on foia: v pkg install/verify/uninstall --engine iris, all 3 M0a invariants, deterministic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…test-iris) Closes the last exec verb. Abort over the synchronous Atelier path needs a live target, so the runner now records its own $job in ^mIrisRun(rid,"pid") (set right after status; "done", set last, marks completion). New m_iris.Abort(rid) SqlProc terminates the recorded pid via $system.Process.Terminate(pid,2), guarded by a ^$JOB(pid) liveness check, a self-check (never $job), and the "done" flag; it returns the pid, "" when nothing is live (parity with m-ydb "no jobs matched"), or "DENIED" on a role failure. remote.Transport.Abort -> exec abort --prefix (driver-local, not an SDK Transport verb — same shape as m-ydb's Session.Abort). caps Exec is now [load,run,eval,abort]. IRIS facts caught live: $ZCHILD is a YottaDB-ism (<SYNTAX> in IRIS) so we capture the run's own $job, not a child pid; ^$JOB(pid) is the M-native liveness check. Gates: go test -race ./... + gofmt + vet green; make test-it green vs m-test-iris (new TestRemoteAbort_RealEngine aborts a live `hang 30`, reports the pid, second abort finds nothing); SDK conformance 16/16 remote. M3 stays in progress: local/docker `iris session` transports remain before it flips to done. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…onformance 16/16 remote+docker) Adds internal/session implementing mdriver.Transport + Abort over `iris session <instance> -U <ns>` (docker wraps it in `docker exec -i <container>`; local runs it on the host). Unlike the remote/Atelier transport, a session writes to the principal device, so device `W` output is captured directly — none of the mIrisIO redirect / result-global recovery machinery is needed (that is a remote-only problem). A transport.go selector (newExecTransport / execTransport interface = the SDK Transport plus the driver-local Abort) picks remote vs session, so the exec axis is transport-agnostic. lifecycle (status/up/down/restart/wait) and meta doctor now dispatch to a session probe too: docker `up` = `docker start` + wait-healthy, `down` = `docker stop`; remote/local `down` detaches. Capture protocol (live-proven): an `iris session` runs each stdin line independently at the prompt, so a $ZTRAP set on a prior line does NOT fire — fault capture uses a single-line TRY/CATCH bracketed by @@MIRIS-BEGIN@@/@@MIRIS-RESULT@@<status>|<§7 frame>. Load pipes source into the container (or a host temp file) then $SYSTEM.OBJ.Load(path,"ck"); ReadGlobal base64-encodes the value so control bytes survive the terminal capture. New config: --container / M_IRIS_CONTAINER, --iris-instance / M_IRIS_IRIS_INSTANCE (default IRIS). Also de-flakes TestRemoteAbort_RealEngine: pre-deploy the runner on both transports (a ReadGlobal calls ensureRunner) before the timing-sensitive pid poll, removing the concurrent PUT+compile race. Gates: go test -race ./... + gofmt + vet green; conformance 16/16 on BOTH remote and docker; make test-it green vs m-test-iris — new TestSessionAxis_RealEngine exercises health/version, load(.m->.int+compile)->run with capture, eval clean+fault(§7), data set/get through a control byte, and abort of a live `hang`. M3 -> done. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mance 16/16 remote+docker) Adds the data axis (driver-contract §5.4): get/set/kill/query globals for fixtures + namespace inspection. get=ReadGlobal, set=SetGlobal (existed); kill and query are new on both transports: - kill: runner m_iris.KillGlobal SqlProc (remote) / `kill @ref` (session). - query: a $query subtree walk. Containment is one expression — $name(@Cur,$qlength(ref))=ref (collation makes the subtree contiguous, so the walk quits as soon as it leaves it). The @Cur indirection is essential: $name operates on the variable unless its value is indirected into a reference first (unlike $qsubscript, which takes the string directly). --order forward/reverse, --depth (0 = whole subtree). Both transports return the SAME node-list wire format — Base64(ref)<TAB>Base64(value) per line (base64 each field so control bytes survive) — so one parseNodes decodes either. The runner QueryGlobal SqlProc returns the string; the session walk writes each node to the principal device, captured between the markers. engineTransport (transport.go) now carries Abort + KillGlobal + QueryGlobal, so the data axis is transport-agnostic. caps Data:[get,set,kill,query] (export/import deferred to a follow-up — not advertised, honest-by-construction). Gates: go test -race ./... + gofmt + vet green; conformance 16/16 on BOTH remote and docker; make test-it green vs m-test-iris — new TestRemoteData_RealEngine + the session-axis query/kill block (seeded subtree queries back its contained nodes excluding a sibling; kill removes the subtree). M4 -> in progress (export/import remain). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… -> in progress) The remote and docker transports already pass the full SDK conformance suite (16/16) via m-driver-conformance. M8 stays in progress pending local-transport validation (needs a host IRIS install) and the remaining cover/admin/native axes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The runner's GetOut Base64-encoded ^mIrisRun(rid,"out") directly, but $system.Encryption.Base64Encode requires an 8-bit byte string and faults <ILLEGAL VALUE>GetOut+2^m.iris.Runner.1 on a captured value holding a char >255. The capture path makes this easy to hit (wchr(c) do app($char(c))), so a script's W $C(8212) (em-dash) turns the out global into a 16-bit string. This blocked the non-ASCII m-stdlib suites (STDURL/STDREGEX/STDJSON/STDXML) on the VSL T0b.2 IRIS remote leg — they errored with no result frame. Fix: GetOut now $zconvert(...,"O","UTF8") before Base64. UTF-8 leaves ASCII/<=127 bytes identical (the v-pkg KIDS marker path is byte-unchanged; the exec-axis IT stays green) and emits multi-byte sequences for the rest, so Base64 is always byte-safe. The Go getOut is unchanged: string(raw) of the Base64-decoded UTF-8 bytes is already the correct Go (UTF-8) string. TDD: TestRemoteWideChar_RealEngine (RED reproduced the exact <ILLEGAL VALUE> fault) — `W "<<W>>",$C(233),$C(8212),"end"` now round-trips to Stdout containing "<<W>>é—end", proving the trailing result marker survives a wide char (the suites' failure mode). make test-it 6/6 RealEngine green; go test -race/vet/gofmt clean. Remote (Atelier) transport only — the docker/session transport captures via iris session stdout markers, a separate path. Downstream confirmation owed: re-run kids-test-in-place.sh iris on foia so the 4 suites frame. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add repo.meta.json with "layer": "m" so `m arch check` reads m-iris's side of the m/v waterline. m-iris is an engine-neutral driver; G1 is clean (no vista-cloud-dev/v-* dep, no VSL* refs). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the arch job calling vista-cloud-dev/.github arch-waterline.yml@main, so `m arch check` (the G1 dependency-direction gate) runs in CI on every push/PR — the ADR §3.3 'no exception' enforcement, not just the local make gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mechanical lint hygiene so the go-ci gate passes (tests already green, conformance 16/16): errcheck on httptest w.Write/io.WriteString in test handlers (_, _ = …), two unused-param `cc` in the remote-unsupported lifecycle command stubs (→ _), an unused test param, and a comment typo (modelling→modeling). No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Turns the renamed
irissyncrepo into m-iris, the InterSystems IRIS engine driver (component D1 of m-engine-drivers), built to the neutral driver-contract v1.0 via the sharedm-driver-sdk.What's here
M_IRIS_*);meta caps/info/version/schemawith an honest caps golden; clikit aligned to the contract exit ladder (0/2/3/4/5/6/7) +engineError.m_iris.Runnerclass invoked over Atelieraction/query; faults surface in^mIrisRun(rid,"error")(§7). Fake-API unit tests run every commit;TestRemoteSpike_RealEngineis gated (M_IRIS_IT=1).lifecycle(status/--probe/wait/up/down/restart; provision/destroy report unsupported exit 7 over Atelier — attach mode, risk B4) +meta doctortyped matrix (reachable/auth/version/namespace/query-privilege).m-driver-sdk(verb-level Transport frozen against both engines), now pinned at v0.1.0.Status
All packages green,
-race/go vet/gofmtclean. The real-engine integration tier stays gated pending a CI IRIS Community container.🤖 Generated with Claude Code