Skip to content

Support JSON columns (read and insert) - #6

Merged
tycooon merged 4 commits into
masterfrom
claude/json-column-support
Jun 23, 2026
Merged

Support JSON columns (read and insert)#6
tycooon merged 4 commits into
masterfrom
claude/json-column-support

Conversation

@tycooon

@tycooon tycooon commented Jun 22, 2026

Copy link
Copy Markdown
Member

What

Adds support for ClickHouse JSON columns — both decoding (read) and inserting.

client.execute(<<~SQL, settings: { enable_json_type: 1 })
  CREATE TABLE events (id UInt32, payload JSON) ENGINE = Memory
SQL

client.insert("events", [{ id: 1, payload: { "user" => "alice", "tags" => %w[a b] } }])

client.query("SELECT payload FROM events").first[:payload]
# => {"user" => "alice", "tags" => ["a", "b"]}

How

  • Bump vendored clickhouse-cpp v2.6.1 → v2.6.2, which adds a string-backed ColumnJSON. The in-tree declared-type patch still applies cleanly (the changed files don't touch the patched regions).
  • Decode (value_at): a JSON column is parsed into a Ruby Hash, consistent with how Map decodes to Hash and Decimal/Date/Time decode to rich types.
  • Encode (append_value): a Hash/Array is serialized with #to_json; a String is sent verbatim as JSON text. nil becomes {} (ClickHouse rejects empty strings) on a plain column, or SQL NULL on Nullable(JSON) via the existing Nullable path + a new append_default arm.
  • Reads auto-enable the string format: output_format_native_write_json_as_string=1 is injected on every read path (query/query_value/query_each). It's sent with the non-important flag, so servers that don't know the setting silently ignore it instead of erroring; user-supplied settings: still override it.
  • Pin CH_MAP_BOOL_TO_UINT8=ON in extconf.rb: v2.6.2 introduces a distinct Bool column, and the gem relies on Bool normalising to UInt8 (see the declared-type handling in client.cpp). The upstream default is already ON; pinning it guards against a future flip.

Drive-by fixes

  • extconf.rb self-heals git installs. Bundler doesn't fetch submodules for git: dependencies unless the Gemfile sets submodules: true, which left vendor/clickhouse-cpp empty and aborted the build. extconf.rb now runs git submodule update --init --recursive itself when the vendored tree is missing and it's inside a git checkout (no-op for released gems and submodules: recursive CI checkouts). The README also documents the submodules: true flag. Verified end-to-end with a fresh no-submodule clone built via rake compile.
  • README: corrected a stale line claiming Map inserts aren't supported — they are (there's a Map arm in append_value and round-trip specs).

Behaviour notes

  • Requires ClickHouse 25.x+. The string wire format ColumnJSON relies on doesn't exist on 24.x, and 24.x's experimental JSON uses an incompatible on-wire serialization. Specs are gated with a min_ch_major: 25 tag and skip cleanly on older servers (verified against a real 24.8 — 3 pending, 0 failures).
  • ClickHouse renders dynamically-typed JSON leaves as strings — the number 1 round-trips as "1", while booleans/nulls stay native and explicitly typed paths (e.g. JSON(a UInt32)) keep their type. This mirrors toJSONString(col) server-side; the specs assert the real round-trip.

Testing

  • rspec green on 25.3 (JSON specs run) and 24.8 (JSON specs skip) — 72 examples, 0 failures on both. CI passes the full matrix (ruby 3.3/3.4/4.0 × ch 24.8/25.3) plus macOS compile.
  • rubocop clean.
  • Plain JSON, parametrized JSON(max_dynamic_paths=N, a UInt32), and Nullable(JSON) all verified round-tripping.
  • Git-install path verified: fresh clone without submodules → rake compile auto-fetches clickhouse-cpp and builds through json.cpp; the resulting extension loads and round-trips JSON.

🤖 Generated with Claude Code

Bump vendored clickhouse-cpp v2.6.1 -> v2.6.2, which adds a string-backed
ColumnJSON, and wire it through the type codec:

- decode a JSON column to a parsed Ruby Hash (like Map -> Hash)
- encode a Hash/Array via #to_json, or a String verbatim; nil becomes
  "{}" (CH rejects empty strings) / SQL NULL on Nullable(JSON)
- auto-inject output_format_native_write_json_as_string on the read
  paths so JSON reads just work; the setting is sent non-important so
  servers that don't know it ignore it rather than erroring
- pin CH_MAP_BOOL_TO_UINT8=ON: v2.6.2 adds a distinct Bool column, and
  the gem relies on Bool normalising to UInt8

JSON's string wire format requires ClickHouse 25.x+; specs are gated via
a min_ch_major tag and skip cleanly on older servers.
tycooon added 3 commits June 22, 2026 14:20
Bundler git: dependencies don't fetch submodules unless the Gemfile sets
submodules: true, leaving vendor/clickhouse-cpp empty and aborting the
build. When running inside a git checkout, fetch it ourselves before
giving up. No-op for released gems and submodules:recursive checkouts.
Also document the submodules: true Gemfile flag.
@tycooon
tycooon merged commit 219755e into master Jun 23, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants