Skip to content

THRIFT-6041: Add js:bigint compiler flag for native BigInt int64 codegen#3529

Open
jimexist wants to merge 1 commit into
apache:masterfrom
jimexist:claude/goofy-meninsky-ea46b4
Open

THRIFT-6041: Add js:bigint compiler flag for native BigInt int64 codegen#3529
jimexist wants to merge 1 commit into
apache:masterfrom
jimexist:claude/goofy-meninsky-ea46b4

Conversation

@jimexist
Copy link
Copy Markdown
Member

@jimexist jimexist commented May 25, 2026

Summary

The js generator now emits native BigInt for i64 by default. A new compiler flag js:bigint (default true) controls the behavior, and a pair of JS-level helpers (thrift.toBigInt, thrift.fromBigInt) bridge between the wire format and bigint at the codegen site. TBinaryProtocol itself is unchanged.

Node 10.4+ has supported BigInt for years and the published package already targets Node ≥ 10.18 via engines, so every supported runtime can use it.

Compiler — js:bigint (default true)

When node is in the option list and bigint=true (the default), the generator:

  • Emits 42n / 9223372036854775807n BigInt literals for I64 consts instead of new Int64(42).
  • Drops the node-int64 import from js_includes, ts_includes, and ts_service_includes.
  • Declares I64 TypeScript fields as bigint instead of Int64.
  • Wraps deserialize sites: this.field = thrift.toBigInt(input.readI64()).
  • Wraps serialize sites: output.writeI64(thrift.fromBigInt(this.field)).
thrift --gen js:node MyService.thrift                # bigint=true is the default
thrift --gen js:node,bigint=false MyService.thrift   # legacy node-int64 output

The flag is silently forced false for plain --gen js (browser JS), so callers that don't enable node are unaffected.

Runtime — two helpers, no protocol changes

lib/nodejs/lib/thrift/index.js (and browser.js) export:

  • thrift.toBigInt(i64) → bigint — reads via Buffer#readBigInt64BE, honoring the Int64's offset.
  • thrift.fromBigInt(value) → Int64 — packs via Buffer#writeBigInt64BE, wrapping out-of-range values with BigInt.asIntN(64, ...).

TBinaryProtocol is untouched. readI64 still returns a Thrift.Int64, writeI64 still expects one. Existing code that uses Thrift.Int64 directly keeps working.

const thrift = require("thrift");
const big = thrift.toBigInt(prot.readI64());    // Int64  → bigint
prot.writeI64(thrift.fromBigInt(big));          // bigint → Int64

Tests

  • lib/nodejs/test/bigint_helpers.test.js (new, 32 assertions, passes locally) — round-trips the helpers across the full signed 64-bit range, verifies protocol round-trip via the helpers, checks non-zero Int64.offset handling, and checks BigInt.asIntN(64, ...) wrapping on overflow.
  • lib/nodejs/test/int64_bigint.test.js (new) — generates Int64Test.thrift with the default bigint=true and asserts the output uses BigInt literals, has no require('node-int64'), and has no new Int64(...). Runs against a separate gen-nodejs-bigint/ fixture in CI.
  • Existing fixtures in lib/nodejs/test/testAll.sh and lib/nodets/test/testAll.sh are pinned to bigint=false, so int64.test.js, int64.test.ts, test_driver.mjs, and test-cases.mjs keep working with their existing node-int64 semantics — no rewrites.

Also fixes the stale "node version 6 or later" claim in lib/nodejs/README.md that contradicted the engines field.

JIRA

No JIRA ticket filed yet — a committer should create one (suggested summary: "Add BigInt support to the js:node generator via a default-true bigint flag") and rename the PR title / branch to the THRIFT-NNNN: convention per AGENTS.md.

AI authorship

Per AGENTS.md § "AI-Generated Contributions", drafted with Claude Opus 4.7; the human author reviewed and ran the runtime tests. The compiler-side test relies on CI to rebuild the generator binary.

Test plan

  • node lib/nodejs/test/bigint_helpers.test.js — 32/32 pass locally
  • lib/nodejs/test/testAll.sh green in CI (integration suite + int64.test.js + int64_bigint.test.js)
  • lib/nodets/test/testAll.sh green in CI
  • Cross-language CI green (no wire-format change; only generator output + additive runtime exports)

@jimexist jimexist requested review from Jens-G and emmenlau as code owners May 25, 2026 01:31
@mergeable mergeable Bot added javascript Pull requests that update Javascript code nodejs typescript compiler labels May 25, 2026
@jimexist jimexist force-pushed the claude/goofy-meninsky-ea46b4 branch from 8afbcad to a359296 Compare May 25, 2026 01:38
@jimexist jimexist changed the title nodejs+compiler: Add BigInt support for int64 (runtime opt-in + compiler default) nodejs+compiler: Add BigInt support for int64 via js:bigint flag May 25, 2026
@jimexist jimexist changed the title nodejs+compiler: Add BigInt support for int64 via js:bigint flag Add js:bigint compiler flag for native BigInt int64 codegen May 25, 2026
@jimexist jimexist force-pushed the claude/goofy-meninsky-ea46b4 branch from a359296 to b201658 Compare May 25, 2026 01:44
@jimexist jimexist changed the title Add js:bigint compiler flag for native BigInt int64 codegen THRIFT-6041: Add js:bigint compiler flag for native BigInt int64 codegen May 25, 2026
@jimexist jimexist requested a review from Copilot May 25, 2026 01:47
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Thrift JS (node) generator to emit native bigint for i64 by default via a new js:bigint flag (default true), and adds runtime helper APIs (thrift.toBigInt, thrift.fromBigInt) to bridge between node-int64 Int64 and bigint at generated read/write sites.

Changes:

  • Add js:bigint generator option (default true for js:node, forced false for plain js) and adjust JS/TS codegen to use bigint + helper wrappers.
  • Export toBigInt / fromBigInt helpers from the Node and browser entrypoints, and add new unit tests covering helper behavior and bigint-mode codegen.
  • Pin existing node/nodets integration fixtures to bigint=false to preserve legacy node-int64-based assertions.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
lib/nodets/test/testAll.sh Pins js:node,ts test fixtures to bigint=false for legacy semantics.
lib/nodejs/test/testAll.sh Pins existing fixtures to bigint=false, adds a separate bigint-mode generated fixture and runs new bigint tests.
lib/nodejs/test/int64_bigint.test.js New test asserting bigint-mode codegen output (constants, no node-int64 import).
lib/nodejs/test/bigint_helpers.test.js New test suite validating helper conversions + protocol round-trips.
lib/nodejs/README.md Updates stated Node compatibility and documents js:bigint behavior and helpers.
lib/nodejs/lib/thrift/index.js Adds thrift.toBigInt / thrift.fromBigInt exports (needs compatibility fallback).
lib/nodejs/lib/thrift/browser.js Adds thrift.toBigInt / thrift.fromBigInt exports for browser bundle (needs compatibility fallback).
lib/nodejs/lib/thrift/binary_protocol.js Minor formatting change (trailing comma) in an exception constructor call.
compiler/cpp/src/thrift/generate/t_js_generator.cc Implements js:bigint flag, adjusts includes, const rendering, (de)serialization wrapping, and TS typing for i64.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/nodejs/lib/thrift/index.js
Comment thread lib/nodejs/lib/thrift/browser.js
@jimexist jimexist force-pushed the claude/goofy-meninsky-ea46b4 branch from b201658 to 1a04bda Compare May 25, 2026 01:52
@mergeable mergeable Bot added the build and general CI cmake, automake and build system changes label May 25, 2026
Client: nodejs,compiler

Adds BigInt support to the js:node binding entirely through the
compiler and JS-level helpers; the binary protocol itself is not
touched.

### Compiler — `js:bigint` flag (default `true`)

t_js_generator.cc gains a `bigint` flag that applies when `node` is in
the option list. When enabled (the default):

- I64 constants are emitted as native BigInt literals (e.g. `42n`,
  `9223372036854775807n`) instead of `new Int64(42)`.
- The node-int64 import is dropped from `js_includes`, `ts_includes`,
  and `ts_service_includes` (node + esm + ts variants).
- TypeScript I64 fields are declared as `bigint` instead of `Int64`.
- Read sites are wrapped: `this.field = thrift.toBigInt(input.readI64())`.
- Write sites are wrapped: `output.writeI64(thrift.fromBigInt(this.field))`.

Pass `bigint=false` to retain the legacy node-int64 codegen. The flag
is silently forced to false for plain `--gen js` (browser JS).

### Runtime — two JS-level helpers, no protocol changes

`lib/nodejs/lib/thrift/index.js` (and browser.js) export:

- `thrift.toBigInt(i64) -> bigint` — reads the underlying buffer with
  `readBigInt64BE`, honoring the Int64's offset.
- `thrift.fromBigInt(value) -> Int64` — packs a bigint into an 8-byte
  buffer with `writeBigInt64BE`, wrapping out-of-range values via
  `BigInt.asIntN(64, ...)`.

`TBinaryProtocol` is unchanged. The conversion boundary lives in
generated code (and any user code that wants to opt in), keeping the
protocol code paths simple.

### Test coverage

- `lib/nodejs/test/bigint_helpers.test.js` (new) — 32 assertions:
  round-trip the helpers across the full signed 64-bit range, verify
  the protocol round-trip via the helpers, check offset handling, and
  check `BigInt.asIntN(64, ...)` wrapping on overflow. Passes locally.
- `lib/nodejs/test/int64_bigint.test.js` (new) — validates the
  compiler emits BigInt literals, drops `require('node-int64')`, and
  drops `new Int64(...)` from generated output. Runs against a
  separate `gen-nodejs-bigint/` fixture in CI.
- All existing test generation invocations in
  `lib/nodejs/test/testAll.sh` and `lib/nodets/test/testAll.sh` now
  pass `bigint=false` explicitly, so `int64.test.js`, `int64.test.ts`,
  `test_driver.mjs`, and `test-cases.mjs` keep working with their
  existing node-int64 semantics without rewrites.

Also fixes the stale "node version 6 or later" claim in the nodejs
README that contradicted the `engines` field.

No JIRA ticket has been filed for this work yet; a committer should
create one (suggested summary: "Add BigInt support to the js:node
generator via a default-true bigint flag") and the PR title / branch
name can then be updated to match the THRIFT-NNNN convention.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Generated-by: Claude Opus 4.7
@jimexist jimexist force-pushed the claude/goofy-meninsky-ea46b4 branch from 1a04bda to 92e9f3e Compare May 25, 2026 01:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build and general CI cmake, automake and build system changes compiler javascript Pull requests that update Javascript code nodejs typescript

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants