CodeMirror: upgrade to latest 6.x, full schema autocomplete, embeddable <datasette-sql-editor> by asg017 · Pull Request #2833 · simonw/datasette

asg017 · 2026-07-10T18:58:17Z

Note

This PR is the integration view of the whole branch. For review, the work is split into a stack of four PRs: #2834 (upgrade + build) -> #2835 (schema autocomplete + endpoint) -> #2836 (ESM primitives) -> #2837 (element + core-page dogfood). Review those; this one tracks the combined branch and interactive QA.

Modernizes Datasette's CodeMirror SQL editor across three workstreams. Draft while interactive QA and the docs ticket finish; may be split into a PR stack (see checklist at bottom).

1. Upgrade to current CodeMirror 6.x

codemirror 6.0.1 → 6.0.2, @codemirror/lang-sql 6.3.3 → 6.10.0 (autocomplete 6.20.3, view 6.43.6, state 6.7.1) — all semver-compatible, no CM7 exists
Fixes a long-standing silent bug: sql() was called with defaultTableName/defaultSchemaName, which were never valid lang-sql options (real names: defaultTable/defaultSchema)
caseInsensitiveIdentifiers: true on the curated SQLite dialect (stops auto-quoting mixed-case identifiers)
Version-agnostic filenames (cm-editor.bundle.js), rollup.config.mjs + npm run build:codemirror replacing the documented one-liner; cache busting already handled by static()'s ?_hash=

2. Full schema autocomplete

Completions now carry column types (detail), rank columns above keywords (boost), distinguish views and give them their real columns via PRAGMA table_xinfo (previously empty)
Table pages' "View and edit SQL" link passes a validated ?_table= so that table's columns complete unprefixed (defaultTable)
New GET /{database}/-/editor-schema.json endpoint: neutral {database, tables: [{name, view, columns: [{name, type}]}]} shape, gated on view-database + execute-sql, hidden tables excluded, 403 leaks no table names. (Named editor-schema.json because /-/schema.json is the existing DDL endpoint.)
window.editor.updateSchema({schema, defaultTable}) reconfigures autocomplete live via a Compartment

3. Embeddable editor for plugins

datasette/static/datasette-sql-editor.js — ESM primitives module (also built to an importable datasette-sql-editor.bundle.js): createSqlEditor() with delegable history (host undo/redo for ProseMirror/collab hosts like datasette-paper), hostChange annotation for echo suppression, submit/escape callbacks, fixed-tooltip mode; datasetteSchema() fetches the new endpoint and maps it identically to the server-inlined shape; CM modules re-exported so plugins share one CodeMirror instance per page
<datasette-sql-editor> — form-associated, light-DOM custom element: name/database/schema-url/default-table/readonly attributes, value/schema/view/readOnly properties, format(), cancelable submit event driving form.requestSubmit(), CSS-custom-property theming, never blocks editing on schema-fetch failure
Core pages dogfood the element: the five SQL templates now render it wrapping a plain <textarea name=sql> fallback that keeps working with JS disabled (adopted + removed on mount). Includes a parser-timing guard (definition loading from <head> connects the element before its children parse). window.editor back-compat preserved; cm global left in place (deprecation is a separate decision)
Manual-QA page: demos/sql-editor-element.html (uv run datasette fixtures.db --static demos:demos/)

Testing

Full suite: 2304 passed (new tests for the schema payload, endpoint permissions, _table= validation, migrated markup)
Live-verified: five page types, bundle serving with ?_hash=, endpoint payloads; element behavior (window.editor, form value, fallback adoption, parser-timing) verified in headless Chrome
Still wants a human browser pass: keyboard nuances, dark mode, resize handle, canned-query parameter refresh

Remaining before ready-for-review

Plugin-author documentation (endpoint / primitives / element)
Interactive browser QA
Decide: keep as one PR or split into a stack (upgrade+build → schema → primitives → element+dogfood)

🤖 Generated with Claude Code

codemirror 6.0.1 -> 6.0.2, @codemirror/lang-sql 6.3.3 -> 6.10.0 (autocomplete 6.20.3, view 6.43.6, state 6.7.1). defaultTableName/defaultSchemaName were never valid SQLConfig options; the real names are defaultTable/defaultSchema. Also enables caseInsensitiveIdentifiers on the SQLite dialect. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

cm-editor-6.0.1.{js,bundle.js} -> cm-editor.{js,bundle.js}; new npm run build:codemirror replaces the documented rollup one-liner. Cache busting already handled by the static() template helper's ?_hash= content hash. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

_editor_schema() emits lang-sql Completion objects (column type as detail, boost above keywords) and gives views their real columns via PRAGMA table_xinfo in a self/children container labelled 'view'. _table_columns() is unchanged for the write-template path. Note the table_columns field in the database JSON context now carries this richer shape. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Per-view Compartment wraps the sql() extension; window.editor.updateSchema( {schema, defaultTable, defaultSchema}) reconfigures autocomplete live. Declares @codemirror/state as a direct dependency since it is now imported directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Neutral {database, tables: [{name, view, columns: [{name, type}]}]} shape, gated on view-database + execute-sql with no table-name leak on 403, hidden tables excluded. /-/schema.json was already taken by the DDL endpoint, hence editor-schema.json. _editor_schema() now maps from the shared _schema_tables() introspection helper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The table page's 'View and edit SQL' link now carries ?_table=<name>; QueryView validates it against the actor-visible tables/views for the database before exposing it as default_table, so the editor completes that table's columns unprefixed. Stored/canned queries are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Follow-up to d12f0d2: the test regexes the inlined schema= JS and still asserted the old flat list-of-strings shape. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Single source of truth for the CodeMirror setup: SQLiteDialect, createSqlEditor() (delegable history with host undo/redo forwarding, hostChange annotation for echo suppression, submit/escape callbacks, fixed-tooltip mode, per-editor Compartment updateSchema), and datasetteSchema() which fetches /-/editor-schema.json and maps it to a lang-sql SQLNamespace identical to the server-inlined shape. cm-editor.js is now a thin consumer; rollup emits both the IIFE and an importable ESM bundle. Submit key is Mod-Enter (Cmd on mac as before, now also Ctrl elsewhere) plus Shift-Enter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Light-DOM element built on createSqlEditor(): form participation via ElementInternals (name= field, reset support), schema fetched from {base-url}/{database}/-/editor-schema.json or schema-url= without ever blocking editing, cancelable submit event on Mod/Shift-Enter driving form.requestSubmit(), readOnly/value/schema/view properties, format() via the sql-formatter global, theming through CSS custom properties with appearance-preserving fallbacks. Auto-registers the tag on import (guarded). Manual-QA page at demos/sql-editor-element.html. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The five SQL pages now render the element wrapping a plain <textarea name=sql> fallback that keeps working without JavaScript (the element adopts and removes it on mount). The element also gained a parser-timing guard: when its definition loads from <head> the browser connects it at the start tag before its children are parsed, so mounting defers to DOMContentLoaded in that case. window.editor back-compat preserved; cm global deprecation deferred. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

codecov · 2026-07-10T19:06:54Z

Codecov Report

❌ Patch coverage is 0% with 55 lines in your changes missing coverage. Please review.
✅ Project coverage is 0.00%. Comparing base (db82123) to head (4ec0f94).

Files with missing lines	Patch %	Lines
datasette/views/query_helpers.py	0.00%	30 Missing ⚠️
datasette/views/special.py	0.00%	19 Missing ⚠️
datasette/views/database.py	0.00%	4 Missing ⚠️
datasette/app.py	0.00%	1 Missing ⚠️
datasette/views/execute_write.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #2833   +/-   ##
=====================================
  Coverage   0.00%   0.00%           
=====================================
  Files         73      73           
  Lines      12208   12262   +54     
=====================================
- Misses     12208   12262   +54

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

asg017 and others added 11 commits July 10, 2026 10:39

Document DatabaseEditorSchemaView label for docs coverage test

49f1660

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Update test_execute_sql schema assertions for rich completion shape

174099b

Follow-up to d12f0d2: the test regexes the inlined schema= JS and still asserted the old flat list-of-strings shape. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

CodeMirror: upgrade to latest 6.x, full schema autocomplete, embeddable <datasette-sql-editor>#2833

CodeMirror: upgrade to latest 6.x, full schema autocomplete, embeddable <datasette-sql-editor>#2833
asg017 wants to merge 11 commits into
mainfrom
asg017/codemirror

asg017 commented Jul 10, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jul 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

asg017 commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. Upgrade to current CodeMirror 6.x

2. Full schema autocomplete

3. Embeddable editor for plugins

Testing

Remaining before ready-for-review

Uh oh!

codecov Bot commented Jul 10, 2026

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

asg017 commented Jul 10, 2026 •

edited

Loading