Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions site/src/content/docs/concepts/how-neural-cad-generation-works.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,16 @@ with newer holistic-latent approaches pushing past 80%. Common failure modes:
non-watertight solids, self-intersections, dangling edges, broken
face-edge-vertex connectivity.

:::note[Measured in this toolkit]
This shows up concretely in [ll_gen](/ll_toolkit/ll_gen/overview/): a diffusion that
denoises **independent face grids** and sews them reaches **0** valid solids on the
honest solid+volume gate — the sampled faces never mate. Re-targeting the same idea to
diffuse a **construction-program** latent and decode it autoregressively (so the kernel
*builds* the solid) reaches **0.934** sampled-z validity, and the autoregressive command
model reaches **0.914** — both measured through the real kernel. These are the toolkit's
own numbers, not the literature figures above.
:::

## The role of reinforcement learning

A recurring result: **RL alignment with solver/kernel feedback** unlocks large
Expand Down
10 changes: 7 additions & 3 deletions site/src/content/docs/concepts/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,13 @@ project's research notes.

A note on honesty: where these pages cite validity rates or benchmark numbers,
those come from the **published research literature** (DeepCAD, BrepGen,
Text2CAD, and others), not from LatticeLabs' own models. The toolkit's neural
packages ship **untrained**; the numbers describe what the *field* has achieved
and frame what is realistic, not what this codebase outputs today.
Text2CAD, and others) and describe what the *field* has achieved. They are kept
distinct from **the toolkit's own measured results**, which are now real and
reproducible — e.g. [ll_brepnet](/ll_toolkit/ll_brepnet/overview/) segmentation at
test mIoU 0.828, and [ll_gen](/ll_toolkit/ll_gen/overview/)'s program-based generators
producing measured-valid CAD (0.914 / 0.934, gated on real non-degenerate solids). Where
a page shows a LatticeLabs number it is labeled as such; field numbers frame what is
realistic, package pages report what this codebase outputs.

## The pages

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ sidebar:

This page looks inside the architectures, with the concrete shapes and choices
that make them work. The numbers are drawn from the primary research papers and
describe the field — not LatticeLabs' (untrained) models.
describe the field; where the toolkit's *own* trained models are referenced, they are
labeled as such (see [ll_gen](/ll_toolkit/ll_gen/overview/) for the toolkit's measured
generation results).

## The four architectural families

Expand Down
29 changes: 22 additions & 7 deletions site/src/content/docs/concepts/the-reality-of-ai-cad-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,13 +74,28 @@ These are architecturally unrelated and routinely confused:

## What to expect from this toolkit

The toolkit's neural packages ship **untrained** — they are architectures and
training loops, not finished models. Treat generated output as a starting point
to be validated, never as a manufacturing-ready part. The honest near-term value
of neural CAD is prototyping, concept exploration, and education — with a human
doing the majority of the design effort. The toolkit is built to make the
*reliable* part (kernel execution + validation) load-bearing, and the
*unreliable* part (the neural proposal) improvable through training and RL.
The toolkit now ships **trained** generators — and they confirm the thesis above. The
[ll_gen](/ll_toolkit/ll_gen/overview/) generators that produce valid CAD are the ones
that generate the **construction program** and execute it: an autoregressive command
model (**0.914** valid) and a latent diffusion over a program autoencoder (**0.934**
valid), both measured through the real kernel and gated on a non-degenerate solid. The
route that *doesn't* work is the one that generates raw B-rep faces to be sewn — its
independently-sampled faces never mate, so honest validity is **0**. That is the
"code → kernel → validate" lesson made concrete in this codebase: validity comes from
the executable representation, not from a richer neural sampler.

Still, treat generated output as a starting point to be validated, never as a
manufacturing-ready part. These models are trained on the DeepCAD distribution of
parametric sketch-and-extrude programs; quality drops off for high-complexity designs,
and a human does the majority of real design effort. The toolkit makes the *reliable*
part (kernel execution + validation) load-bearing, and the *neural* part improvable.

:::note[Honest measurement]
Validity here is gated on a real closed solid with positive volume — `BRepCheck` alone
passes volume-less degenerates (an earlier metric scored a broken generator at 1.0 while
it produced 0 real solids). The harness also reports distinct-shape counts so a high rate
from one repeated shape is visible, not hidden.
:::

## See also

Expand Down
2 changes: 1 addition & 1 deletion site/src/content/docs/contributing/docs-site.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ site/src/content/docs/
guides/ # task-oriented how-tos
<pkg>/ # one per package: overview, installation, usage
reference/ # GENERATED API reference (do not hand-edit)
roadmap/ # planned, not-yet-built packages (e.g. ll_brepnet)
roadmap/ # roadmap + shipped milestones (e.g. ll_brepnet → done)
contributing/ # this section
```

Expand Down
13 changes: 13 additions & 0 deletions site/src/content/docs/get-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,19 @@ pip install -e ".[all]" # Everything
available through conda, not PyPI. It is included in `environment.yml`.
:::

## Native MLX (Apple Silicon)

The neural models also train and run natively in **[MLX](https://github.com/ml-explore/mlx)**
on Apple Silicon (M-series). Each package's `mlx/` trainers convert the real PyTorch
checkpoints and verify parity, or train from scratch — see each package's Usage page.

```bash
pip install mlx mlx-lm # in the conda env; mlx-lm is used by ll_ocadr's LLM splice
```

The OCC kernel (`pythonocc-core`) is still required for the generation packages, since
validity is measured by executing generated programs through the real kernel.

## Next steps

Continue to the [Quickstart](/ll_toolkit/get-started/quickstart/) for a first
Expand Down
26 changes: 21 additions & 5 deletions site/src/content/docs/get-started/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,13 +65,29 @@ topology = builder.build_complete_topology(
embedding = encoder(token_ids, topology_data=topology) # [1, 1024]
```

:::note[Models ship untrained]
The neural packages (`ll_stepnet`, `ll_ocadr`, `ll_gen`) ship with
architectures but **no trained checkpoints**. Until you train them, their
outputs are not meaningful predictions. Each package's pages explain how to
train or run a proof-of-life model.
:::note[Which models are trained]
Several models now ship **trained** with reproducible, honest metrics:
**ll_brepnet** (B-Rep segmentation, test mIoU 0.828), **ll_stepnet** (face-count
classifier, val acc 0.976), **ll_ocadr** (geometry-grounded, 0.919 vs 0.313 shuffled),
and **ll_gen**'s program-based generators (valid CAD: AR 0.914 / latent diffusion 0.934).
The neural models train and run natively in **MLX on Apple Silicon** as well as PyTorch.
Remaining task heads (e.g. ll_stepnet property prediction/QA) ship as architectures —
train them on your data before relying on their outputs.
:::

## Train / run natively on Apple Silicon (MLX)

Each neural package has an `mlx/` trainer that runs on Apple Silicon. The ones with
existing PyTorch checkpoints convert the real weights and prove parity:

```bash
python ll_stepnet/mlx/train_classification_mlx.py --mode parity # acc 0.976, argmax 1.0 vs PyTorch
python ll_brepnet/mlx/train_brepnet_mlx.py --mode parity # mIoU parity vs PyTorch
python ll_gen/mlx/ar_generator_mlx.py --mode train # valid CAD generation 0.914
python ll_gen/mlx/latent_diffusion_mlx.py --mode train # latent-diffusion generation 0.934
python ll_ocadr/mlx/train_ocadr_mlx.py --mode train # geometry-grounded multimodal
```

## Where to go next

- **[cadling](/ll_toolkit/cadling/overview/)** — full CAD parsing, chunking, and SDG.
Expand Down
33 changes: 23 additions & 10 deletions site/src/content/docs/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,11 @@ import { Card, CardGrid, LinkCard, Badge } from '@astrojs/starlight/components';

export const base = import.meta.env.BASE_URL;

The **LatticeLabs Toolkit** is a monorepo of six Python packages that bring 3D
The **LatticeLabs Toolkit** is a monorepo of seven Python packages that bring 3D
CAD geometry into the AI ecosystem — parsing CAD files, tokenizing geometry,
running neural models over STEP/B-Rep data, recognizing geometry for LLMs,
processing point clouds, and generating parametric CAD.
running neural models over STEP/B-Rep data, segmenting B-Rep faces, recognizing
geometry for LLMs, processing point clouds, and generating parametric CAD. The
neural models train and run **natively in MLX on Apple Silicon** as well as PyTorch.

Each package is independently installable and documented here. The pages below
describe **what actually exists in the code today** — maturity is called out per
Expand All @@ -38,9 +39,14 @@ what is planned.
/>
<LinkCard
title="ll_stepnet"
description="Neural networks for STEP/B-Rep files: tokenizer, feature extractor, topology builder, transformer+GNN encoder, and task heads."
description="Neural networks for STEP/B-Rep files: tokenizer, feature extractor, topology builder, transformer+GNN encoder, and task heads. Trained classifier (0.976), native MLX."
href={base + 'll_stepnet/overview/'}
/>
<LinkCard
title="ll_brepnet"
description="B-Rep face segmentation (UV-Net + coedge message passing). Trained on the full Fusion 360 split — test mIoU 0.828. Native MLX (parity-verified)."
href={base + 'll_brepnet/overview/'}
/>
<LinkCard
title="geotoken"
description="Geometric tokenizer with adaptive quantization — mesh, command-sequence, and B-Rep graph tokenization for transformer models."
Expand All @@ -53,7 +59,7 @@ what is planned.
/>
<LinkCard
title="ll_gen"
description="Generation orchestration: neural propose, deterministic dispose in a CadQuery sandbox, REINFORCE training loop. Models ship untrained."
description="Generation orchestration: neural propose, deterministic dispose in a kernel sandbox. Trained program-based generators produce measured-valid CAD (AR 0.914 / latent diffusion 0.934)."
href={base + 'll_gen/overview/'}
/>
<LinkCard
Expand All @@ -74,9 +80,16 @@ what is planned.

## Project status

<Card title="On the horizon" icon="rocket">
**ll_brepnet** — a planned B-Rep face-graph network — is not yet implemented
(the directory is an empty scaffold). It is tracked honestly on the{' '}
<a href={base + 'roadmap/ll_brepnet/'}>Roadmap</a> rather than documented as if
it worked. <Badge text="Planned" variant="caution" />
<Card title="Recently shipped" icon="rocket">
**ll_brepnet** — the B-Rep face-segmentation network — is built, trained, and
documented: **test mIoU 0.828** on the full Fusion 360 split, with a parity-verified
native-MLX port. See its <a href={base + 'll_brepnet/overview/'}>docs</a> and the{' '}
<a href={base + 'roadmap/ll_brepnet/'}>Roadmap</a>. <Badge text="Trained" variant="success" />

**ll_gen generation** now produces measured-valid CAD via the construction-program
route (autoregressive command generator + latent diffusion), gated on real
non-degenerate solids. <Badge text="Generates valid CAD" variant="success" />

Still experimental: **vLLM serving** for ll_ocadr (the HF-native path works today).
<Badge text="Planned" variant="caution" />
</Card>
15 changes: 15 additions & 0 deletions site/src/content/docs/ll_brepnet/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,21 @@ convolution. (An earlier 4,800-solid subset run reached 0.709; training on the
full set raised it to 0.828 and lifted the rare **RevolveEnd** class from 0.11
to 0.55.)

## Native MLX (Apple Silicon)

A native-MLX port (`ll_brepnet/mlx/train_brepnet_mlx.py`) reproduces the exact
architecture — UV-Net encoders with BatchNorm + the coedge message-passing encoder
(input projection, residual coedge convolutions with LayerNorm, output projection) — and
**converts the real trained checkpoint** into MLX. Driving both models from the same
`BRepDataset`, it is verified at **100% per-face argmax agreement** with PyTorch and an
**identical mIoU (0.835 on the measured subset)**, so the MLX model *is* the trained GNN
running on Apple Silicon (the conversion handles Conv `OIHW→OHWI`/`OIW→OWI` permutes and
inference-mode BatchNorm running stats):

```bash
python ll_brepnet/mlx/train_brepnet_mlx.py --mode parity # convert real weights + verify
```

:::note[Scope]
The repository ships the training/eval code and this reproducible recipe; the
trained checkpoint (~4.8 MB) is produced by the run in **Usage** rather than
Expand Down
86 changes: 56 additions & 30 deletions site/src/content/docs/ll_gen/overview.md
Original file line number Diff line number Diff line change
@@ -1,61 +1,87 @@
---
title: ll_gen — Overview
description: Generation orchestration for CAD — neural propose, deterministic dispose in a CadQuery sandbox, with a REINFORCE training loop. Models ship untrained.
description: Generation orchestration for CAD — neural propose, deterministic dispose in a sandbox. Ships trained generators that produce measured-valid CAD via the construction-program route.
sidebar:
label: Overview
order: 1
badge:
text: Untrained
variant: caution
text: Generates valid CAD
variant: success
---

**ll_gen** orchestrates generative CAD modeling around a simple idea: **propose,
then dispose**. A generator proposes a candidate (neural latent sample, an LLM
code proposal, or a deterministic template), and a deterministic *dispose* stage
executes that proposal in a sandboxed [CadQuery](https://cadquery.readthedocs.io/)
subprocess to produce — and verify — real geometry. A REINFORCE training loop
closes the alignment loop using the verification result as reward.
then dispose**. A generator proposes a candidate (a neural sample or a deterministic
template), and a deterministic *dispose* stage executes that proposal in a sandboxed
geometry kernel ([OCC](https://dev.opencascade.org/) /
[CadQuery](https://cadquery.readthedocs.io/)) to produce — and verify — real
geometry. The verification result feeds an RL alignment loop.

## How it works

```text
GenerationOrchestrator.generate(prompt)
→ propose
├─ neural VAE / diffusion / VQ-VAE (stepnet models)
├─ LLM code proposal
├─ autoregressive command generator (the construction PROGRAM)
├─ latent diffusion over a program autoencoder
└─ deterministic template
→ dispose (CadQuery subprocess sandbox) → real solid + validity
→ dispose (OCC / CadQuery sandbox) → real solid + validity
→ verification + feedback (reward)

Training:
RLAlignmentTrainer.train_step
→ generate_for_training (log-probs on the live graph)
→ reward (dispose success / validity)
→ advantage = reward − baseline → loss.backward() → optimizer.step()
```

The neural generators import their models from the **`stepnet`** package
(`STEPVAE`, `StructuredDiffusion`, `VQVAEModel`, `CADGenerationPipeline`). Code
proposals are always executed in the subprocess sandbox — never `exec`'d in
process.
## Generators that produce valid CAD

The robust route to valid CAD — the one DeepCAD/Text2CAD take — is to generate the
**construction program** (sketch + extrude commands) and *execute* it, so the kernel
builds a watertight solid, rather than generating B-rep faces that must be sewn. Two
trained generators take this route and run natively in **MLX on Apple Silicon**:

- **Autoregressive command generator** (`ll_gen/mlx/ar_generator_mlx.py`) — a causal
transformer over the CAD command vocabulary, trained on ~38k real DeepCAD programs,
sampled token-by-token → executed. **Measured validity 0.914** (234/256), **104
distinct** shapes.
- **Latent diffusion** (`ll_gen/mlx/latent_diffusion_mlx.py`) — diffuses the latent of
a program autoencoder and decodes autoregressively. **Sampled-z validity 0.934**
(239/256), **138 distinct**. The validity comes from the execution-respecting
decoder; the diffusion contributes the diverse latent prior (138 distinct vs a
predict-the-mean baseline's 14).

Two earlier routes are superseded because of their *representation*, not their
training: the command-VAE's parallel (non-autoregressive) decoder is primitive-limited
(~0–12% valid, posterior-collapses), and the raw-geometry diffusion produced **0** valid
solids — it denoised independent face grids that never mate when sewn. A faithful MLX
port of the command-VAE (`ll_gen/mlx/vae_mlx.py`) reproduces the PyTorch model exactly
(parity-verified) but inherits that limitation.

## Running a training loop
## Honest validity

ll_gen exposes a runnable training entry point:
Validity is **measured through the real kernel** and gated on a **non-degenerate solid**
(a closed solid with positive volume) — `BRepCheck` alone passes volume-less shells, so
gating on it would over-report. The harness also reports `num_distinct` (rounded
bounding boxes) as a mode-collapse guard, so a high rate from one repeated trivial shape
is visible, not hidden. (`GenerationMetrics.is_valid_solid`.)

## Running it

```bash
# train + measure the autoregressive command generator (Apple Silicon / MLX)
python ll_gen/mlx/ar_generator_mlx.py --mode train

# train the latent-diffusion generator and measure sampled-z validity
python ll_gen/mlx/latent_diffusion_mlx.py --mode train

# the orchestration / RL training entry point
python -m ll_gen.training.run --help
```

## Status

:::caution[Maturity: pipeline real, models untrained]
The orchestration, dispose sandbox, verification, and REINFORCE loop are real and
run end-to-end. The neural generators **ship untrained** — a proof-of-life VAE
training run (`python -m ll_gen.training.proof_of_life`) is documented, but
production-quality generation requires real training. Reward is gated on
producing a **closed solid**, so the metric reflects genuine CAD validity rather
than reward-hacked non-solids.
:::tip[Generators trained; validity measured through the real kernel]
The orchestration, dispose sandbox, verification, and RL loop run end-to-end, and
ll_gen now ships **trained generators that produce measured-valid CAD** on the DeepCAD
distribution — the autoregressive command generator (0.914 valid) and the latent
diffusion (0.934 valid), each gated on real non-degenerate solids. Native-MLX trainers
run on Apple Silicon. Scope is stated honestly: these are trained on DeepCAD parametric
command sequences (sketch + extrude), and validity is measured on that distribution.
:::

Use the sidebar for **Installation**, **Usage**, and the **API Reference**.
Loading
Loading