diff --git a/site/src/content/docs/concepts/how-neural-cad-generation-works.md b/site/src/content/docs/concepts/how-neural-cad-generation-works.md index 94f132b..89ce14b 100644 --- a/site/src/content/docs/concepts/how-neural-cad-generation-works.md +++ b/site/src/content/docs/concepts/how-neural-cad-generation-works.md @@ -73,6 +73,16 @@ with newer holistic-latent approaches pushing past 80%. Common failure modes: non-watertight solids, self-intersections, dangling edges, broken face-edge-vertex connectivity. +:::note[Measured in this toolkit] +This shows up concretely in [ll_gen](/ll_toolkit/ll_gen/overview/): a diffusion that +denoises **independent face grids** and sews them reaches **0** valid solids on the +honest solid+volume gate — the sampled faces never mate. Re-targeting the same idea to +diffuse a **construction-program** latent and decode it autoregressively (so the kernel +*builds* the solid) reaches **0.934** sampled-z validity, and the autoregressive command +model reaches **0.914** — both measured through the real kernel. These are the toolkit's +own numbers, not the literature figures above. +::: + ## The role of reinforcement learning A recurring result: **RL alignment with solver/kernel feedback** unlocks large diff --git a/site/src/content/docs/concepts/index.md b/site/src/content/docs/concepts/index.md index 68e33df..a407b0c 100644 --- a/site/src/content/docs/concepts/index.md +++ b/site/src/content/docs/concepts/index.md @@ -12,9 +12,13 @@ project's research notes. A note on honesty: where these pages cite validity rates or benchmark numbers, those come from the **published research literature** (DeepCAD, BrepGen, -Text2CAD, and others), not from LatticeLabs' own models. The toolkit's neural -packages ship **untrained**; the numbers describe what the *field* has achieved -and frame what is realistic, not what this codebase outputs today. +Text2CAD, and others) and describe what the *field* has achieved. They are kept +distinct from **the toolkit's own measured results**, which are now real and +reproducible — e.g. [ll_brepnet](/ll_toolkit/ll_brepnet/overview/) segmentation at +test mIoU 0.828, and [ll_gen](/ll_toolkit/ll_gen/overview/)'s program-based generators +producing measured-valid CAD (0.914 / 0.934, gated on real non-degenerate solids). Where +a page shows a LatticeLabs number it is labeled as such; field numbers frame what is +realistic, package pages report what this codebase outputs. ## The pages diff --git a/site/src/content/docs/concepts/inside-cad-generation-models.md b/site/src/content/docs/concepts/inside-cad-generation-models.md index 166f15e..2fda23a 100644 --- a/site/src/content/docs/concepts/inside-cad-generation-models.md +++ b/site/src/content/docs/concepts/inside-cad-generation-models.md @@ -8,7 +8,9 @@ sidebar: This page looks inside the architectures, with the concrete shapes and choices that make them work. The numbers are drawn from the primary research papers and -describe the field — not LatticeLabs' (untrained) models. +describe the field; where the toolkit's *own* trained models are referenced, they are +labeled as such (see [ll_gen](/ll_toolkit/ll_gen/overview/) for the toolkit's measured +generation results). ## The four architectural families diff --git a/site/src/content/docs/concepts/the-reality-of-ai-cad-generation.md b/site/src/content/docs/concepts/the-reality-of-ai-cad-generation.md index 06a0062..acd2107 100644 --- a/site/src/content/docs/concepts/the-reality-of-ai-cad-generation.md +++ b/site/src/content/docs/concepts/the-reality-of-ai-cad-generation.md @@ -74,13 +74,28 @@ These are architecturally unrelated and routinely confused: ## What to expect from this toolkit -The toolkit's neural packages ship **untrained** — they are architectures and -training loops, not finished models. Treat generated output as a starting point -to be validated, never as a manufacturing-ready part. The honest near-term value -of neural CAD is prototyping, concept exploration, and education — with a human -doing the majority of the design effort. The toolkit is built to make the -*reliable* part (kernel execution + validation) load-bearing, and the -*unreliable* part (the neural proposal) improvable through training and RL. +The toolkit now ships **trained** generators — and they confirm the thesis above. The +[ll_gen](/ll_toolkit/ll_gen/overview/) generators that produce valid CAD are the ones +that generate the **construction program** and execute it: an autoregressive command +model (**0.914** valid) and a latent diffusion over a program autoencoder (**0.934** +valid), both measured through the real kernel and gated on a non-degenerate solid. The +route that *doesn't* work is the one that generates raw B-rep faces to be sewn — its +independently-sampled faces never mate, so honest validity is **0**. That is the +"code → kernel → validate" lesson made concrete in this codebase: validity comes from +the executable representation, not from a richer neural sampler. + +Still, treat generated output as a starting point to be validated, never as a +manufacturing-ready part. These models are trained on the DeepCAD distribution of +parametric sketch-and-extrude programs; quality drops off for high-complexity designs, +and a human does the majority of real design effort. The toolkit makes the *reliable* +part (kernel execution + validation) load-bearing, and the *neural* part improvable. + +:::note[Honest measurement] +Validity here is gated on a real closed solid with positive volume — `BRepCheck` alone +passes volume-less degenerates (an earlier metric scored a broken generator at 1.0 while +it produced 0 real solids). The harness also reports distinct-shape counts so a high rate +from one repeated shape is visible, not hidden. +::: ## See also diff --git a/site/src/content/docs/contributing/docs-site.md b/site/src/content/docs/contributing/docs-site.md index a46b31e..2da9fcb 100644 --- a/site/src/content/docs/contributing/docs-site.md +++ b/site/src/content/docs/contributing/docs-site.md @@ -32,7 +32,7 @@ site/src/content/docs/ guides/ # task-oriented how-tos / # one per package: overview, installation, usage reference/ # GENERATED API reference (do not hand-edit) - roadmap/ # planned, not-yet-built packages (e.g. ll_brepnet) + roadmap/ # roadmap + shipped milestones (e.g. ll_brepnet → done) contributing/ # this section ``` diff --git a/site/src/content/docs/get-started/installation.md b/site/src/content/docs/get-started/installation.md index 5802586..52f11b7 100644 --- a/site/src/content/docs/get-started/installation.md +++ b/site/src/content/docs/get-started/installation.md @@ -77,6 +77,19 @@ pip install -e ".[all]" # Everything available through conda, not PyPI. It is included in `environment.yml`. ::: +## Native MLX (Apple Silicon) + +The neural models also train and run natively in **[MLX](https://github.com/ml-explore/mlx)** +on Apple Silicon (M-series). Each package's `mlx/` trainers convert the real PyTorch +checkpoints and verify parity, or train from scratch — see each package's Usage page. + +```bash +pip install mlx mlx-lm # in the conda env; mlx-lm is used by ll_ocadr's LLM splice +``` + +The OCC kernel (`pythonocc-core`) is still required for the generation packages, since +validity is measured by executing generated programs through the real kernel. + ## Next steps Continue to the [Quickstart](/ll_toolkit/get-started/quickstart/) for a first diff --git a/site/src/content/docs/get-started/quickstart.md b/site/src/content/docs/get-started/quickstart.md index 0a53ce8..c26d110 100644 --- a/site/src/content/docs/get-started/quickstart.md +++ b/site/src/content/docs/get-started/quickstart.md @@ -65,13 +65,29 @@ topology = builder.build_complete_topology( embedding = encoder(token_ids, topology_data=topology) # [1, 1024] ``` -:::note[Models ship untrained] -The neural packages (`ll_stepnet`, `ll_ocadr`, `ll_gen`) ship with -architectures but **no trained checkpoints**. Until you train them, their -outputs are not meaningful predictions. Each package's pages explain how to -train or run a proof-of-life model. +:::note[Which models are trained] +Several models now ship **trained** with reproducible, honest metrics: +**ll_brepnet** (B-Rep segmentation, test mIoU 0.828), **ll_stepnet** (face-count +classifier, val acc 0.976), **ll_ocadr** (geometry-grounded, 0.919 vs 0.313 shuffled), +and **ll_gen**'s program-based generators (valid CAD: AR 0.914 / latent diffusion 0.934). +The neural models train and run natively in **MLX on Apple Silicon** as well as PyTorch. +Remaining task heads (e.g. ll_stepnet property prediction/QA) ship as architectures — +train them on your data before relying on their outputs. ::: +## Train / run natively on Apple Silicon (MLX) + +Each neural package has an `mlx/` trainer that runs on Apple Silicon. The ones with +existing PyTorch checkpoints convert the real weights and prove parity: + +```bash +python ll_stepnet/mlx/train_classification_mlx.py --mode parity # acc 0.976, argmax 1.0 vs PyTorch +python ll_brepnet/mlx/train_brepnet_mlx.py --mode parity # mIoU parity vs PyTorch +python ll_gen/mlx/ar_generator_mlx.py --mode train # valid CAD generation 0.914 +python ll_gen/mlx/latent_diffusion_mlx.py --mode train # latent-diffusion generation 0.934 +python ll_ocadr/mlx/train_ocadr_mlx.py --mode train # geometry-grounded multimodal +``` + ## Where to go next - **[cadling](/ll_toolkit/cadling/overview/)** — full CAD parsing, chunking, and SDG. diff --git a/site/src/content/docs/index.mdx b/site/src/content/docs/index.mdx index 7f37bce..240d1b7 100644 --- a/site/src/content/docs/index.mdx +++ b/site/src/content/docs/index.mdx @@ -18,10 +18,11 @@ import { Card, CardGrid, LinkCard, Badge } from '@astrojs/starlight/components'; export const base = import.meta.env.BASE_URL; -The **LatticeLabs Toolkit** is a monorepo of six Python packages that bring 3D +The **LatticeLabs Toolkit** is a monorepo of seven Python packages that bring 3D CAD geometry into the AI ecosystem — parsing CAD files, tokenizing geometry, -running neural models over STEP/B-Rep data, recognizing geometry for LLMs, -processing point clouds, and generating parametric CAD. +running neural models over STEP/B-Rep data, segmenting B-Rep faces, recognizing +geometry for LLMs, processing point clouds, and generating parametric CAD. The +neural models train and run **natively in MLX on Apple Silicon** as well as PyTorch. Each package is independently installable and documented here. The pages below describe **what actually exists in the code today** — maturity is called out per @@ -38,9 +39,14 @@ what is planned. /> + - **ll_brepnet** — a planned B-Rep face-graph network — is not yet implemented - (the directory is an empty scaffold). It is tracked honestly on the{' '} - Roadmap rather than documented as if - it worked. + + **ll_brepnet** — the B-Rep face-segmentation network — is built, trained, and + documented: **test mIoU 0.828** on the full Fusion 360 split, with a parity-verified + native-MLX port. See its docs and the{' '} + Roadmap. + + **ll_gen generation** now produces measured-valid CAD via the construction-program + route (autoregressive command generator + latent diffusion), gated on real + non-degenerate solids. + + Still experimental: **vLLM serving** for ll_ocadr (the HF-native path works today). + diff --git a/site/src/content/docs/ll_brepnet/overview.md b/site/src/content/docs/ll_brepnet/overview.md index 0f0696e..3216aac 100644 --- a/site/src/content/docs/ll_brepnet/overview.md +++ b/site/src/content/docs/ll_brepnet/overview.md @@ -77,6 +77,21 @@ convolution. (An earlier 4,800-solid subset run reached 0.709; training on the full set raised it to 0.828 and lifted the rare **RevolveEnd** class from 0.11 to 0.55.) +## Native MLX (Apple Silicon) + +A native-MLX port (`ll_brepnet/mlx/train_brepnet_mlx.py`) reproduces the exact +architecture — UV-Net encoders with BatchNorm + the coedge message-passing encoder +(input projection, residual coedge convolutions with LayerNorm, output projection) — and +**converts the real trained checkpoint** into MLX. Driving both models from the same +`BRepDataset`, it is verified at **100% per-face argmax agreement** with PyTorch and an +**identical mIoU (0.835 on the measured subset)**, so the MLX model *is* the trained GNN +running on Apple Silicon (the conversion handles Conv `OIHW→OHWI`/`OIW→OWI` permutes and +inference-mode BatchNorm running stats): + +```bash +python ll_brepnet/mlx/train_brepnet_mlx.py --mode parity # convert real weights + verify +``` + :::note[Scope] The repository ships the training/eval code and this reproducible recipe; the trained checkpoint (~4.8 MB) is produced by the run in **Usage** rather than diff --git a/site/src/content/docs/ll_gen/overview.md b/site/src/content/docs/ll_gen/overview.md index e5baa01..c913c31 100644 --- a/site/src/content/docs/ll_gen/overview.md +++ b/site/src/content/docs/ll_gen/overview.md @@ -1,61 +1,87 @@ --- title: ll_gen — Overview -description: Generation orchestration for CAD — neural propose, deterministic dispose in a CadQuery sandbox, with a REINFORCE training loop. Models ship untrained. +description: Generation orchestration for CAD — neural propose, deterministic dispose in a sandbox. Ships trained generators that produce measured-valid CAD via the construction-program route. sidebar: label: Overview order: 1 badge: - text: Untrained - variant: caution + text: Generates valid CAD + variant: success --- **ll_gen** orchestrates generative CAD modeling around a simple idea: **propose, -then dispose**. A generator proposes a candidate (neural latent sample, an LLM -code proposal, or a deterministic template), and a deterministic *dispose* stage -executes that proposal in a sandboxed [CadQuery](https://cadquery.readthedocs.io/) -subprocess to produce — and verify — real geometry. A REINFORCE training loop -closes the alignment loop using the verification result as reward. +then dispose**. A generator proposes a candidate (a neural sample or a deterministic +template), and a deterministic *dispose* stage executes that proposal in a sandboxed +geometry kernel ([OCC](https://dev.opencascade.org/) / +[CadQuery](https://cadquery.readthedocs.io/)) to produce — and verify — real +geometry. The verification result feeds an RL alignment loop. ## How it works ```text GenerationOrchestrator.generate(prompt) → propose - ├─ neural VAE / diffusion / VQ-VAE (stepnet models) - ├─ LLM code proposal + ├─ autoregressive command generator (the construction PROGRAM) + ├─ latent diffusion over a program autoencoder └─ deterministic template - → dispose (CadQuery subprocess sandbox) → real solid + validity + → dispose (OCC / CadQuery sandbox) → real solid + validity → verification + feedback (reward) - -Training: - RLAlignmentTrainer.train_step - → generate_for_training (log-probs on the live graph) - → reward (dispose success / validity) - → advantage = reward − baseline → loss.backward() → optimizer.step() ``` -The neural generators import their models from the **`stepnet`** package -(`STEPVAE`, `StructuredDiffusion`, `VQVAEModel`, `CADGenerationPipeline`). Code -proposals are always executed in the subprocess sandbox — never `exec`'d in -process. +## Generators that produce valid CAD + +The robust route to valid CAD — the one DeepCAD/Text2CAD take — is to generate the +**construction program** (sketch + extrude commands) and *execute* it, so the kernel +builds a watertight solid, rather than generating B-rep faces that must be sewn. Two +trained generators take this route and run natively in **MLX on Apple Silicon**: + +- **Autoregressive command generator** (`ll_gen/mlx/ar_generator_mlx.py`) — a causal + transformer over the CAD command vocabulary, trained on ~38k real DeepCAD programs, + sampled token-by-token → executed. **Measured validity 0.914** (234/256), **104 + distinct** shapes. +- **Latent diffusion** (`ll_gen/mlx/latent_diffusion_mlx.py`) — diffuses the latent of + a program autoencoder and decodes autoregressively. **Sampled-z validity 0.934** + (239/256), **138 distinct**. The validity comes from the execution-respecting + decoder; the diffusion contributes the diverse latent prior (138 distinct vs a + predict-the-mean baseline's 14). + +Two earlier routes are superseded because of their *representation*, not their +training: the command-VAE's parallel (non-autoregressive) decoder is primitive-limited +(~0–12% valid, posterior-collapses), and the raw-geometry diffusion produced **0** valid +solids — it denoised independent face grids that never mate when sewn. A faithful MLX +port of the command-VAE (`ll_gen/mlx/vae_mlx.py`) reproduces the PyTorch model exactly +(parity-verified) but inherits that limitation. -## Running a training loop +## Honest validity -ll_gen exposes a runnable training entry point: +Validity is **measured through the real kernel** and gated on a **non-degenerate solid** +(a closed solid with positive volume) — `BRepCheck` alone passes volume-less shells, so +gating on it would over-report. The harness also reports `num_distinct` (rounded +bounding boxes) as a mode-collapse guard, so a high rate from one repeated trivial shape +is visible, not hidden. (`GenerationMetrics.is_valid_solid`.) + +## Running it ```bash +# train + measure the autoregressive command generator (Apple Silicon / MLX) +python ll_gen/mlx/ar_generator_mlx.py --mode train + +# train the latent-diffusion generator and measure sampled-z validity +python ll_gen/mlx/latent_diffusion_mlx.py --mode train + +# the orchestration / RL training entry point python -m ll_gen.training.run --help ``` ## Status -:::caution[Maturity: pipeline real, models untrained] -The orchestration, dispose sandbox, verification, and REINFORCE loop are real and -run end-to-end. The neural generators **ship untrained** — a proof-of-life VAE -training run (`python -m ll_gen.training.proof_of_life`) is documented, but -production-quality generation requires real training. Reward is gated on -producing a **closed solid**, so the metric reflects genuine CAD validity rather -than reward-hacked non-solids. +:::tip[Generators trained; validity measured through the real kernel] +The orchestration, dispose sandbox, verification, and RL loop run end-to-end, and +ll_gen now ships **trained generators that produce measured-valid CAD** on the DeepCAD +distribution — the autoregressive command generator (0.914 valid) and the latent +diffusion (0.934 valid), each gated on real non-degenerate solids. Native-MLX trainers +run on Apple Silicon. Scope is stated honestly: these are trained on DeepCAD parametric +command sequences (sketch + extrude), and validity is measured on that distribution. ::: Use the sidebar for **Installation**, **Usage**, and the **API Reference**. diff --git a/site/src/content/docs/ll_gen/usage.md b/site/src/content/docs/ll_gen/usage.md index 6ca4c57..b1bfa55 100644 --- a/site/src/content/docs/ll_gen/usage.md +++ b/site/src/content/docs/ll_gen/usage.md @@ -60,6 +60,30 @@ All paths converge on the `DisposalEngine`, which executes the proposal, validates the result (`DisposalResult`, `GeometryReport`), and can apply `RepairAction`s. +## Trained generators that produce valid CAD (native MLX) + +Two trained generators take the **construction-program** route — generate the command +program and *execute* it — so the kernel builds a watertight solid. They run natively in +MLX on Apple Silicon and report validity **measured through the real kernel**, gated on a +non-degenerate solid (closed solid with positive volume): + +```bash +# Autoregressive command generator — trained on real DeepCAD programs. +# Result: validity 0.914 (234/256), 104 distinct, non-degenerate. +python ll_gen/mlx/ar_generator_mlx.py --mode train + +# Latent diffusion over a program autoencoder. +# Result: sampled-z validity 0.934 (239/256), 138 distinct. +python ll_gen/mlx/latent_diffusion_mlx.py --mode train +``` + +For the latent diffusion the headline metric is **sampled-z** validity +(noise → denoise → decode → execute), reported against a `z=0` predict-the-mean +baseline so a diverse generator is distinguishable from one that repeats the mean shape. +A faithful MLX port of the command-VAE (`python ll_gen/mlx/vae_mlx.py --mode parity`) +reproduces the PyTorch model exactly, but that parallel-decoder VAE is itself a weak +generator (~0–12% valid) — the program-based generators above are the valid-CAD path. + ## Train the neural generators (REINFORCE) The RL alignment loop rewards proposals that dispose into valid solids: @@ -94,12 +118,15 @@ python -m ll_gen.training.proof_of_life \ It reports `validity_rate` **and** `num_distinct_valid` at both points plus the per-epoch curve. -:::caution[Models ship untrained] -Out of the box the neural generators are randomly initialized — their prior -samples are mostly invalid. The dispose stage and the RL loop are real and run -end-to-end; meaningful generation requires training. See +:::note[Two generator generations — know which you're running] +The **program-based** generators (`ar_generator_mlx.py`, `latent_diffusion_mlx.py`) are +**trained** and produce measured-valid CAD (0.914 / 0.934 valid). The **legacy** +neural generators reachable from the orchestrator (`vae`, `vqvae`, `diffusion` via the +REINFORCE loop) are randomly initialized out of the box — their prior samples are mostly +invalid until trained, and the raw-geometry diffusion is limited by representation +(independently-sampled faces don't mate). See [The reality of AI CAD generation](/ll_toolkit/concepts/the-reality-of-ai-cad-generation/) -for why the propose→dispose design is the reliable part. +for why generating the program and executing it is the reliable route. ::: ## Related diff --git a/site/src/content/docs/ll_ocadr/overview.md b/site/src/content/docs/ll_ocadr/overview.md index 5e1c906..2112e43 100644 --- a/site/src/content/docs/ll_ocadr/overview.md +++ b/site/src/content/docs/ll_ocadr/overview.md @@ -56,12 +56,29 @@ vLLM's `ModelRegistry` and does **not** implement `SupportsMultiModal`, so it cannot be served by the vLLM engine as written. Use the HF-native path above. ::: +## Native MLX — a trained, geometry-grounded model + +`ll_ocadr/mlx/train_ocadr_mlx.py` trains the **real** geometry tower natively on Apple +Silicon: a faithful MLX port of GeometryNet (PointNet++) + ShapeNet (Point-BERT) + +projector — **forward-parity-verified at ~1e-6** against the PyTorch encoders +(`ll_ocadr/mlx/faithful_tower_mlx.py`) — projects a CAD point cloud into 256 mesh tokens +spliced into a frozen 4-bit Qwen2, with LoRA + the encoder trained jointly. On a held-out +CAD point-cloud → class task it reaches **llm-generation accuracy 0.919 vs a shuffled-mesh +control of 0.313** (majority 0.374) — i.e. the model genuinely *reads the geometry* and +verbalizes it, rather than guessing from the text prior. + +```bash +python ll_ocadr/mlx/faithful_tower_mlx.py --mode parity # prove the tower == real encoders +python ll_ocadr/mlx/train_ocadr_mlx.py --mode train # train encoder + projector + LoRA +``` + ## Status -:::note[Maturity: HF-native runnable; models untrained] -The HF-native model, encoders, and inference script are real and tested. Like the -other neural packages, ll_ocadr **ships no trained weights** — a tiny offline LM -is used in tests. vLLM serving is planned future work. +:::tip[Trained via MLX; HF-native path real; vLLM experimental] +The native-MLX path produces a **genuinely trained, geometry-grounded model** (0.919 vs +0.313 shuffled). The HF-native PyTorch model, encoders, and inference script are real and +tested (a tiny offline LM is used in unit tests). vLLM serving remains experimental / +future work (see the caution above). ::: Use the sidebar for **Installation**, **Usage**, and the **API Reference**. diff --git a/site/src/content/docs/ll_stepnet/overview.md b/site/src/content/docs/ll_stepnet/overview.md index 3287f79..32ddfb0 100644 --- a/site/src/content/docs/ll_stepnet/overview.md +++ b/site/src/content/docs/ll_stepnet/overview.md @@ -5,8 +5,8 @@ sidebar: label: Overview order: 1 badge: - text: Untrained - variant: caution + text: Trained + MLX + variant: success --- **ll_stepnet** is a neural-network package for processing STEP / B-Rep CAD @@ -56,15 +56,21 @@ print(features["entity_type"], features["numeric_params"]) ## Status -:::caution[Maturity: architectures present, models untrained] -ll_stepnet provides full model architectures and a trainer, but **ships no -trained checkpoints**. A randomly-initialized model produces meaningless -predictions until you train it on STEP data. See **Usage** (in the sidebar) for -the training loop. +:::tip[First real trained checkpoint + native MLX] +ll_stepnet now ships a **trained classifier**: `STEPForClassification` trained on real +DeepCAD models to predict face-count complexity (≤4 / 5–6 / 7+ faces, a geometric label +derived from the reconstructed solid). **Validation accuracy 0.976** vs a 0.436 +majority-class baseline (per-class 0.991 / 0.978 / 0.933). A **native-MLX port** +(`ll_stepnet/mlx/train_classification_mlx.py`) loads those exact trained weights and +reproduces the PyTorch model on Apple Silicon — verified at **100% argmax agreement, +identical 0.976 accuracy** (`--mode parity`). The other task heads (property prediction, +similarity, captioning, QA) provide architectures + trainers; train them on your data +before relying on their outputs. ::: `stepnet` also provides the generative models (`STEPVAE`, `StructuredDiffusion`, -`VQVAEModel`, `CADGenerationPipeline`) that [ll_gen](/ll_toolkit/ll_gen/overview/) -drives for neural CAD generation. +`VQVAEModel`, `CADGenerationPipeline`). The valid-CAD generation path now lives in +[ll_gen](/ll_toolkit/ll_gen/overview/) as trained, program-based generators +(autoregressive command model + latent diffusion). Use the sidebar for **Installation**, **Usage**, and the **API Reference**. diff --git a/site/src/content/docs/ll_stepnet/usage.md b/site/src/content/docs/ll_stepnet/usage.md index 51f131a..4962b85 100644 --- a/site/src/content/docs/ll_stepnet/usage.md +++ b/site/src/content/docs/ll_stepnet/usage.md @@ -80,13 +80,26 @@ trainer = STEPTrainer(model=STEPForClassification(num_classes=10), trainer.train(num_epochs=10, save_every=2) ``` -:::caution[Train before you trust outputs] -The models ship **untrained**. A randomly-initialized network produces -meaningless predictions — train it on STEP data (labels as integers for -classification, or float vectors for property prediction) before relying on -outputs. +:::tip[A trained classifier ships; train other heads before trusting them] +`STEPForClassification` ships **trained** — face-count complexity from real DeepCAD +models, **val acc 0.976** (vs 0.436 majority), with a native-MLX port that reproduces it +exactly on Apple Silicon. The other heads (property prediction, similarity, captioning, +QA) ship as architectures — a randomly-initialized network produces meaningless +predictions, so train them on STEP data (integer labels for classification, float +vectors for property prediction) before relying on outputs. ::: +## Native MLX (Apple Silicon) + +`ll_stepnet/mlx/train_classification_mlx.py` runs the classifier natively in MLX. It +converts the real PyTorch checkpoint and **proves parity** (100% argmax agreement, +identical 0.976 accuracy), and can also train the faithful architecture from scratch: + +```bash +python ll_stepnet/mlx/train_classification_mlx.py --mode parity # convert + verify vs PyTorch +python ll_stepnet/mlx/train_classification_mlx.py --mode train # native-MLX training +``` + ## Generative models `stepnet` also exports the generative models — `STEPVAE`, `StructuredDiffusion`, diff --git a/site/src/content/docs/roadmap/ll_brepnet.md b/site/src/content/docs/roadmap/ll_brepnet.md index 6de34fc..5223875 100644 --- a/site/src/content/docs/roadmap/ll_brepnet.md +++ b/site/src/content/docs/roadmap/ll_brepnet.md @@ -17,7 +17,9 @@ solids and trains a face-segmentation model end-to-end. On the **full official split** of the **Fusion 360 Gallery segmentation dataset** (official 8 manufacturing-feature classes, 5,366-solid held-out test split) it reaches **test mIoU ≈ 0.828 / accuracy ≈ 0.947** — **exceeding** the BRepNet -paper's reported ~0.65–0.72 mIoU, achieved with an MIT-clean architecture. +paper's reported ~0.65–0.72 mIoU, achieved with an MIT-clean architecture. A +**native-MLX port** (parity-verified at 100% per-face agreement with PyTorch) runs the +trained GNN on Apple Silicon. ::: See the package documentation: diff --git a/site/src/content/docs/tutorials/generate-cad.md b/site/src/content/docs/tutorials/generate-cad.md index c9ed071..5881283 100644 --- a/site/src/content/docs/tutorials/generate-cad.md +++ b/site/src/content/docs/tutorials/generate-cad.md @@ -10,13 +10,14 @@ In this tutorial you will run [ll_gen](/ll_toolkit/ll_gen/overview/)'s propose→dispose loop and then train a proof-of-life neural generator. Allow ~20 minutes (training is short and CPU-friendly). -:::caution[Set expectations first] -ll_gen's neural generators ship **untrained** — random weights produce mostly -invalid geometry. The *dispose* stage (CadQuery execution + validation) and the -RL loop are real. This tutorial shows the loop working end to end and a -before/after validity measurement, not a production model. Read +:::note[Two generator generations] +This tutorial walks the **orchestrator + RL loop** with a from-scratch generator so you +can see the propose→dispose loop and a before/after validity measurement end to end. For +generators that already **produce valid CAD**, jump to +[Generate valid CAD (trained, MLX)](#5-generate-valid-cad-trained-mlx) below — the +autoregressive command generator (0.914 valid) and latent diffusion (0.934 valid). Read [The reality of AI CAD generation](/ll_toolkit/concepts/the-reality-of-ai-cad-generation/) -for the why. +for why generating the program and executing it is the reliable route. ::: ## 1. Generate from a prompt @@ -96,6 +97,28 @@ python -m ll_gen.training.run \ The command prints a metrics JSON with reward, advantage, baseline, and loss. +## 5. Generate valid CAD (trained, MLX) + +The generators that actually produce valid CAD take the **construction-program** route — +generate the command program and execute it, so the kernel builds a watertight solid. +They train and run natively in MLX on Apple Silicon: + +```bash +# Autoregressive command generator: trains on real DeepCAD programs, then samples + executes. +# Reports validity through the real kernel, gated on a non-degenerate solid. +python ll_gen/mlx/ar_generator_mlx.py --mode train +# -> validity 0.914 (234/256), distinct 104, non-degenerate + +# Latent diffusion over a program autoencoder: sample z -> decode -> execute. +python ll_gen/mlx/latent_diffusion_mlx.py --mode train +# -> sampled-z validity 0.934 (239/256), distinct 138 (vs a z=0 mean baseline: 14 distinct) +``` + +Both report `num_distinct` alongside validity, so a high rate from one repeated shape +(mode collapse) is visible. The latent-diffusion run prints **sampled-z** validity (noise +→ denoise → decode → execute) against a `z=0` predict-the-mean baseline — the comparison +that proves the diffusion adds diversity rather than repeating the mean shape. + ## Where to next - [ll_gen Usage](/ll_toolkit/ll_gen/usage/) for the full API. diff --git a/site/src/content/docs/tutorials/index.md b/site/src/content/docs/tutorials/index.md index 711ed5f..01edece 100644 --- a/site/src/content/docs/tutorials/index.md +++ b/site/src/content/docs/tutorials/index.md @@ -14,7 +14,7 @@ the toolkit. |---|---|---| | [Parse a STEP file](/ll_toolkit/tutorials/parse-a-step-file/) | cadling | convert a CAD file to a structured document and export it | | [Tokenize a mesh](/ll_toolkit/tutorials/tokenize-a-mesh/) | geotoken | turn a mesh into tokens and measure the quantization impact | -| [Generate CAD](/ll_toolkit/tutorials/generate-cad/) | ll_gen | run the propose→dispose loop and train a proof-of-life model | +| [Generate CAD](/ll_toolkit/tutorials/generate-cad/) | ll_gen | run the propose→dispose loop and train generators that produce valid CAD | | [OCADR HF inference](/ll_toolkit/tutorials/ocadr-hf-inference/) | ll_ocadr | feed geometry into a language model and read its description | For conceptual background, see [Concepts](/ll_toolkit/concepts/); for terse diff --git a/site/src/content/docs/tutorials/ocadr-hf-inference.md b/site/src/content/docs/tutorials/ocadr-hf-inference.md index 39b632a..a9f66ca 100644 --- a/site/src/content/docs/tutorials/ocadr-hf-inference.md +++ b/site/src/content/docs/tutorials/ocadr-hf-inference.md @@ -11,12 +11,14 @@ HF-native pipeline: encode a 3D object's geometry into a language model's embedding space and have the model describe it. Allow ~15 minutes (first run downloads the chosen LM). -:::caution[Untrained projector + vLLM is not used here] -The encoders and the HF model are real and tested, but the toolkit ships **no -trained weights** for the geometry→text projector — a small object will run end -to end, but treat the text as a mechanism demo, not an accurate caption. This -tutorial uses only the **HF-native** path; the vLLM serving path is experimental -and not functional. See the [Overview](/ll_toolkit/ll_ocadr/overview/). +:::note[This HF path's projector is untrained — the trained model is the MLX one] +This tutorial exercises the **HF-native** PyTorch path with an **untrained** +geometry→text projector — a small object runs end to end, but treat the text as a +mechanism demo, not an accurate caption. A **genuinely trained, geometry-grounded model** +exists via the native-MLX path (`ll_ocadr/mlx/train_ocadr_mlx.py`): it reads the geometry +and verbalizes the correct class at **0.919 vs a 0.313 shuffled-mesh control**. The vLLM +serving path is experimental and not functional. See the +[Overview](/ll_toolkit/ll_ocadr/overview/). ::: ## 1. Run the CLI