Releases: Eamon2009/Quadtrix.cpp
Releases · Eamon2009/Quadtrix.cpp
v1.1.0
Quadtrix v1.1.0
Date: 2026-05-09 | Device: NVIDIA T4 (CUDA / bf16) | Framework: PyTorch 2.10.0+cu128
Model Configuration
| Parameter | Value |
|---|---|
| Layers | 6 |
| Heads | 6 |
| Embedding dim | 100 |
| Block size | 190 |
| Batch size | 64 |
| Dropout | 0.2 |
| Learning rate | 3e-4 |
| Total parameters | 10,837,257 |
Training Details
| Field | Value |
|---|---|
| Steps | 8,000 |
| Eval every | 200 steps |
| Optimizer seed | 1337 |
| Train tokens | 14,080,249 |
| Val tokens | 1,564,473 |
| Precision | bf16 |
| MFU | 60.0% |
Results
| Metric | Value |
|---|---|
| Best val loss | 2.3918 |
| Final train loss | 2.2825 |
| Total loss drop | 8.57 |
| Peak throughput | 19,602 tok/s |
| Mean throughput | 18,756 tok/s |
| Peak grad norm | 2.2504 |
| Mean grad norm | 1.6894 |
| Training time | 82m 43s |
| Checkpoint | best_model.pt |
Notes
- Throughput ramps from ~279 tok/s at step 0 to a steady ~19,600 tok/s after the first eval interval, reflecting CUDA kernel warm-up.
- Gradient norms remain stable throughout training (mean 1.69), with no anomalous spikes observed.
What's Changed
- build(deps): bump @tanstack/react-query from 5.100.6 to 5.100.9 in /frontend by @dependabot[bot] in #23
- Revert "build(deps): bump @tanstack/react-query from 5.100.6 to 5.100.9 in /frontend" by @Eamon2009 in #25
Full Changelog: v1.01...v1.1.0
What's Changed
- build(deps): bump @tanstack/react-query from 5.100.6 to 5.100.9 in /frontend by @dependabot[bot] in #23
- Revert "build(deps): bump @tanstack/react-query from 5.100.6 to 5.100.9 in /frontend" by @Eamon2009 in #25
New Contributors
- @dependabot[bot] made their first contribution in #23
Full Changelog: v1.01...v1.1.0
Contributors
dependabot and Eamon2009
Assets 4
Quadtrix v1.01
Quadtrix v1.0
Efficiency metrics
First release — token-level language model trained on CPU.
Training run
| Metric | Value |
|---|---|
| Loss reduction | 69.7% (10.82 → 3.25) |
| Best loss | 3.252 (step 2510) |
| Peak throughput | 435 tok/s |
| Wall time | ~61 min |
Loss curve
Model config
| Parameter | Value |
|---|---|
| Parameters | 6,684,497 |
| Architecture | 4 layers · 4 heads · 64d embedding |
| Batch · block size | 16 · 32 |
| Learning rate | 1e-3 |
| Dropout | 0.1 |
| Train tokens | 7,065,137 |
| Val tokens | 785,016 |
How to run
python engine/main.py
python engine/inference.pyNotes
- Training ran on CPU (PyTorch 2.4.1) with steady 60% bf16 MFU throughout
- Loss converged from 10.82 → 3.25 over 2,690 steps in ~61 minutes
- Gradient norms stable; no spikes or divergence observed
- Checkpoint saved at step 2510 (best validation loss)