Skip to content

Add dynamic batch size support#2

Merged
ChinChangYang merged 8 commits intomasterfrom
feature/dynamic-batch-size
Jan 15, 2026
Merged

Add dynamic batch size support#2
ChinChangYang merged 8 commits intomasterfrom
feature/dynamic-batch-size

Conversation

@ChinChangYang
Copy link
Owner

Summary

  • Add min_batch_size, max_batch_size options to ConversionOptions
  • Implement setBatchDimension() in MILBuilder for dynamic batch support
  • Add ShapeRange support in CoreMLSerializer for flexible input shapes
  • Add --dynamic-batch <min,max> CLI option to katago2coreml
  • Add BatchSizeTests.cpp unit tests (5 tests)
  • Add verify_output_error.py for cross-validation

Bug Fixes

Fixed 4 reshape operations that used incompatible shapes for batch > 1:

  • Global bias reshape
  • Metadata encoder reshape
  • Global pooling block reshape
  • Policy head reshape

Test plan

  • Unit tests: 13/13 passed
  • Before/after regression test: 0 error (old code vs new code at batch=1)
  • Static vs Dynamic comparison: 0 error
  • Per-sample consistency: 1.59e-03 error (within 1e-2 threshold)

Usage

# Static batch (default, backward compatible)
./katago2coreml model.bin.gz output.mlpackage

# Fixed batch=4
./katago2coreml --dynamic-batch 4,4 model.bin.gz output.mlpackage

# Dynamic batch 1-8
./katago2coreml --dynamic-batch 1,8 model.bin.gz output.mlpackage

🤖 Generated with Claude Code

ChinChangYang and others added 3 commits January 12, 2026 08:13
Provides guidance for Claude Code including build commands, the three-stage
converter pipeline (Parser → MIL Builder → Serializer), key types, and
KataGo model version support details.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add min_batch_size, max_batch_size options to ConversionOptions
- Implement setBatchDimension() in MILBuilder for dynamic batch
- Add ShapeRange support in CoreMLSerializer for flexible inputs
- Add --dynamic-batch CLI option to katago2coreml
- Add BatchSizeTests.cpp unit tests (5 tests)
- Add verify_output_error.py for cross-validation
- Fix 4 reshape operations for batch > 1 compatibility

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add three KataGo test models to tests/models/ (22 MB total)
  - g170e-b10c128: Standard test model (11 MB)
  - g170-b6c96: Smaller/faster test model (3.7 MB)
  - b5c192nbt-distilled: Distilled model with metadata encoder (7 MB)

- Add tests/models/README.md with licensing and usage documentation
  - Clear attribution for g170 models (CC0/public domain)
  - License information for b5c192nbt-distilled model (BSD-3-Clause)
  - Reference info for other KataGo models (MIT-style)

- Update NOTICE with model license information

- Update CLAUDE.md and README.md with testing documentation
  - Document C++ unit tests (ctest)
  - Document Python integration tests (pytest)
  - Include model requirements and usage instructions

- Update tests/conftest.py for standalone repository structure
  - Fix paths to work with katagocoreml-cpp standalone repo
  - Update model directory to tests/models/

- Add Python-specific ignores to .gitignore
  - Ignore __pycache__/ directories
  - Remove checked-in .pyc files

- Remove TODO comment from ParserTests.cpp (now resolved)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@ChinChangYang ChinChangYang marked this pull request as ready for review January 14, 2026 13:56
ChinChangYang and others added 2 commits January 15, 2026 07:45
Add comprehensive CI workflow that builds, tests, and validates the
katagocoreml-cpp project on macOS. The workflow includes:
- Dependency installation (cmake, protobuf, abseil, zlib)
- C++ unit tests with CTest
- Real model conversion test using KataGo models
- Package structure validation
- Installation verification

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Remove --batch-size and --precision options that don't exist in
katago2coreml CLI. The tool uses --dynamic-batch for batching and
--float16 for precision control. Default settings are sufficient
for the CI test (batch size 1, FLOAT32 precision).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@ChinChangYang ChinChangYang marked this pull request as draft January 14, 2026 23:55
ChinChangYang and others added 3 commits January 15, 2026 08:02
Add comprehensive documentation for the --dynamic-batch CLI option and
min_batch_size/max_batch_size API fields. Include usage examples for
bounded and unbounded batch sizes, and performance considerations.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements comprehensive dynamic batch size testing for C++ converter:
- Add TestCppDynamicBatch class with parametrized tests for (1,4), (1,8), and unlimited batch configs
- Create deterministic batched inputs helper with per-sample seed consistency
- Extend convert_with_cpp helper to support min_batch_size/max_batch_size parameters
- Validate dynamic batch models produce numerically equivalent outputs to fixed baseline
- Add defensive shape validation and enhanced error messages
- Fix input generation to ensure consistent seeds between batched and individual samples
- Extract RANDOM_SEED_BASE constant and add proper type hints
- Add input validation and overflow protection

CI workflow improvements:
- Remove pre-installed dependencies (cmake, zlib) from brew install
- Document which dependencies are pre-installed on GitHub runners

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Increments converter version from 1.0.0 to 1.1.0 following semantic
versioning for the new backwards-compatible dynamic batch size feature.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@ChinChangYang ChinChangYang marked this pull request as ready for review January 15, 2026 06:43
@ChinChangYang ChinChangYang merged commit 2389746 into master Jan 15, 2026
1 check passed
@ChinChangYang ChinChangYang deleted the feature/dynamic-batch-size branch January 15, 2026 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant