Add tokenization regression tests#24

Closed

Sainan wants to merge 4 commits intomasterfrom

codex/add-regression-testing-for-tokenisation

Member

Sainan commented Jun 23, 2025

Summary

add node_modules/ to .gitignore
document running regression tests
add generate-baseline and test scripts in package.json
add Node-based utilities for tokenization regression testing
store baseline tokenization of visual-*.pluto
include package-lock.json

Testing

npm install
npm test

https://chatgpt.com/codex/tasks/task_e_6858ae5fd5ec83259c130d7fb44bff3b


          Add grammar regression tests

e1b90ec

Sainan added the codex label

— with

ChatGPT Codex Connector

Sainan added 3 commits

June 23, 2025 03:43


          Add node_modules to .vscodeignore

964796c


          Add GitHub Actions workflow for tests

adb58af


          Remove this

2bf6b92

Sainan closed this

Sainan deleted the codex/add-regression-testing-for-tokenisation branch

June 23, 2025 02:27

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels