Native-MLX faithful ports + valid CAD generation (AR + latent diffusion) + honest validity metric#15
Merged
Merged
Conversation
…wing
command_executor: build sketch loops by threading curve endpoints and
auto-closing, so multi-line/arc polygons close by construction instead of
relying on a non-autoregressive decoder to align independent per-curve argmaxes
(the wire-closure limit that left only circles valid). Non-connecting square ->
valid solid; tri/pent/hex/oct all close.
surface_executor: fix two bugs that made the diffusion path never produce a
shape — (1) _fit_bspline_surface called cadling's fit_surface(grid, tolerance=)
with an unsupported kwarg and expected a face but got a dict (TypeError dropped
every face); (2) the merge step called TopologyMerger.merge_edges, which does
not exist — the real API is merge(faces) -> {shape, valid}. Now uses merge(faces)
as the primary watertight path with the built-in edge dedup as fallback. With
these, real DeepCAD geometry sews into closed shells.
Regression tests: TestClosureAwareSketch (3), TestWatertightSew (unit cube ->
closed volume).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
vae: STEPVAE.forward's parameter loss averaged over all 16 param heads, but the 6-command schema only ever activates slots 0-7, so heads 8-15 receive an all-ignore_index target -> F.cross_entropy(mean) over zero elements = NaN -> poisoned recon_loss, silently blocking the supervised reconstruction forward. Now skips all-ignored heads and averages over contributing ones. diffusion: GeometryCodec.encode_faces/encode_edges permute then feed Conv2d/ Conv1d; the conv backward calls .view() on the non-contiguous tensor and raises on MPS. Added .contiguous() (also unblocks the RL diffusion path on MPS). Regression test: test_vae_sparse_param_loss.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Honest writeup of the ll_gen "make-real" investigation: the command-VAE is primitive-limited (non-AR decoder -> cylinders; supervised -> posterior collapse to all-SOL) while the diffusion path's codec is real (recon MSE 0.0003) and its sewing pipeline is now functional after 3 bug fixes, with denoiser convergence the remaining open frontier. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… diffusion codec Output the make-real trainers as proper, reproducible, tracked scripts in the package scripts/ dirs (they previously lived only in gitignored scratch). Each is self-contained (inlines the validated DeepCAD cad_vec -> executor-schema translation) and writes its checkpoint + metrics to the package checkpoints/ dir. - ll_stepnet/scripts/train_classification.py: trains STEPForClassification on real DeepCAD models (STEP -> face-count complexity class). Real result: val acc 0.976 vs 0.434 majority baseline; per-class 0.991 / 0.968 / 0.939. - ll_gen/scripts/train_diffusion_codec.py: trains StructuredDiffusion's GeometryCodec on real B-rep geometry. Real result: recon MSE 0.40 -> 0.0003. Trained checkpoints land in ll_stepnet/checkpoints/ and ll_gen/checkpoints/ (gitignored binaries, per the repo's *.pt convention). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…pty folders) The harness hardcoded export=False while still creating <output_dir>/disposed/, so every eval left an empty directory instead of the CAD the model generated. Add an `export` parameter (default True) forwarded to DisposalEngine.dispose, so an eval run writes <id>.step + <id>.stl per valid shape. Throughput-sensitive callers (per-step RL evals) can pass export=False. Verified: the trained 95% VAE checkpoint now exports 30/30 valid models as real STEP/STL files. Regression test: test_export_flag_forwarded_to_dispose. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three wiring bugs prevented LatticelabsOCADRForCausalLM from running at all:
1. config.n_embed was hardcoded (1280) and matched no real base LLM
(Qwen2-0.5B=896, 1.5B=1536, 7B=3584). The projector emits n_embed-dim mesh
tokens that are spliced INTO the LLM input embeddings (hidden_size), so the
splice raised a shape mismatch. n_embed is now derived from the loaded LLM's
hidden_size in __init__ (LLM loaded first), so any base size wires correctly.
2. get_config_for_model referenced non-existent Qwen2 sizes ("1.8B"/"14B"). Now
uses real sizes and adds first-class "0.5b"/"1.5b" variants that run/train on
a single machine ("7b" needs a GPU). n_embed is no longer set here (derived).
3. _splice_tokens assigned fp32 modality embeddings into a bf16 LLM embedding
tensor (Qwen2 loads as bf16) -> "Index put requires ... dtypes match". It now
casts modality embeddings to the destination dtype/device before the splice.
Verified end to end on the 0.5b variant: forward -> logits [1,S,vocab] and
generate() both run with mesh conditioning. Regression test: tests/test_wiring.py
(mocks the base LLM, so no Qwen download).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
End-to-end MLX implementation of ll_ocadr: a trainable PointNet-style geometry encoder + projector map a CAD point cloud into the embedding space of a FROZEN 4-bit-quantized Qwen2 (via mlx-lm's input_embeddings path); LoRA adapters on the LLM + the encoder train jointly so the LLM learns to attend to the injected mesh tokens. Trains where PyTorch-MPS would OOM, and the projector/LoRA are sized to the real LLM (not a proxy). Result (DeepCAD point-cloud -> 3-way class, Qwen2-0.5B-4bit, best epoch): encoder_mesh_read_acc 0.804, llm_generation_acc 0.543 vs shuffled-mesh baseline 0.322 (majority 0.350) — the model genuinely reads the geometry and verbalizes it well above the shuffled-mesh control. Includes a held-out SHUFFLED-mesh baseline as the honest grounding test, an auxiliary classification head (makes mesh tokens discriminative), and dataset caching. Checkpoint + metrics -> ll_ocadr/checkpoints/ (gitignored binaries). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MLX port of stepnet.tasks.STEPForClassification (token embedding + transformer encoder + masked mean-pool + MLP head), same DeepCAD cad_vec -> face-count complexity task as the PyTorch trainer so results are directly comparable. Result: MLX val acc 0.942 vs majority 0.436 (PyTorch reference 0.976). A faithful Apple-Silicon-native port reaching strong accuracy; the small gap is the lighter MLX config (2 layers / d=128). Checkpoint + metrics -> ll_stepnet/checkpoints/. Part of the all-models -> MLX migration (after ll_ocadr). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MLX port of LLBRepNet: UV-Net face/edge encoders + BRepNet coedge message-passing (h' = relu(W_self·h + W_next·h[next] + W_prev·h[prev] + W_mate·h[mate]) x6) + coedge->face mean-pool (scatter-mean via one-hot matmul) + per-face seg head. Graph batching via offset concatenation of the coedge/face/ edge indices. Same Fusion360 data/task as the PyTorch trainer. Result: MLX best val mIoU 0.711 / acc 0.906 (6k train solids, 20 epochs) vs the full PyTorch reference test mIoU 0.828 (27k solids). The GNN is correct and converges; the gap is data/epochs/encoder-capacity, not the port. Hardest model in the all-models -> MLX migration (after ll_ocadr, ll_stepnet). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…76 parity Supersedes the earlier simplified MLX classifier (which retrained a lighter model to 0.942). This reproduces the EXACT STEPForClassification architecture (6-layer post-norm nn.TransformerEncoder + fusion + head, manual MHA from the packed in_proj_weight) and CONVERTS the real trained checkpoint (stepnet_classifier.pt, PyTorch val acc 0.976) into MLX. Verified on the trainer's exact val split (1000 real DeepCAD samples): argmax agreement vs PyTorch = 1.0, max abs logit diff = 1e-5, MLX val acc = 0.976 == PyTorch 0.976 (per-class 0.991/0.978/0.933). The MLX model IS the trained model (same weights, identical predictions), running natively on Apple Silicon, and remains trainable (real MLX modules). Modes: probe | convert | parity | train. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… mIoU parity Supersedes the earlier simplified MLX port (a lighter GNN retrained to 0.711 mIoU). This reproduces the EXACT LLBRepNet architecture — UV-Net encoders WITH BatchNorm, the cadling BRepNetEncoder (input_proj Linear+LayerNorm+ReLU, 4x residual coedge convs each with LayerNorm, output_proj) and the seg head — and CONVERTS the real trained Lightning checkpoint (best.ckpt, PyTorch test mIoU 0.828) into MLX. Conversion handles the layout differences the simplified port ignored: Conv2d OIHW->OHWI and Conv1d OIW->OWI weight permutes, and BatchNorm running stats applied in inference mode. Both models are driven from the SAME real BRepDataset. Verified on 1500 test solids (23,056 faces): per-face argmax agreement vs PyTorch = 1.0, MLX mIoU = 0.8354 == PyTorch 0.8354, MLX acc = 0.945 == PyTorch 0.945. The MLX model IS the trained GNN, running natively on Apple Silicon. Modes: probe | convert | parity. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rity proof The configured ll_ocadr model was never trained (no checkpoint to convert) and can't run on Apple Silicon, so unlike stepnet/brepnet there are no weights to load. But the encoders are plain nn.Modules with no LLM dependency, so faithfulness is still PROVABLE: random-init the real PyTorch GeometryNet (PointNet++) + ShapeNet (Point-BERT, 0.5B config depth=4/heads=8) + linear projector, convert weights into this MLX tower, feed the same (coords, normals) through both, assert forward parity. The 1x1 convs are per-point Linears over channels -> implemented as MLX Linear + BatchNorm (no NHWC Conv layout); MultiheadAttention reconstructed manually from the packed in_proj_weight; FPS + ball-query computed once in numpy (deterministic start) and shared by both models so sampling can't cause divergence. Result: max-abs-diff geometry=1.2e-6, shape=4.1e-6, mesh_tokens=3.3e-6 -> PASS. The MLX tower provably reproduces the real encoders. (Retrain wiring next.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ove parity ll_gen's neural generator is ll_stepnet.stepnet.vae.STEPVAE (transformer encoder-decoder VAE over CAD command tokens), trained to vae_warm.pt / vae_rl_solid.pt. This reproduces the EXACT architecture in MLX and converts the real weights: encoder (6 post-norm layers, masked-mean pool) + mu/log_var heads; decode(z) = latent_project broadcast + dec_pos -> 6x TransformerDecoderLayer (causal self-attn + cross-attn to z-memory + FFN, 3 post-norm LayerNorms) -> command_head(6) + 16 param_heads(256). MHA splits the packed in_proj into Wq/Wk/Wv so one module serves both self- and cross-attention; the parallel decode means decoder.token/pos embeddings are unused on this path (skipped). Verified by converting vae_warm.pt (227 tensors) and comparing the full encode-> decode against the PyTorch STEPVAE on the same tokens: max-abs-diff mu=4e-7, command_logits=1.6e-6, param_logits=1.1e-5, decoded-command argmax agreement = 1.0 -> PASS. The MLX VAE IS the trained generator, running natively on Apple Silicon. Modes: probe | convert | parity. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…256 mesh tokens) Replaces the maxpool stand-in encoder with the parity-proven faithful tower (faithful_tower_mlx.py): real GeometryNet (PointNet++) + ShapeNet (Point-BERT) + linear projector -> 256 mesh tokens spliced into a frozen 4-bit Qwen2; tower + LoRA train jointly (31.98M trainable). Data extended to N=2048 points + area-weighted normals + cached deterministic FPS/ball-query grouping. Task/data/metrics held identical to the maxpool run for a direct comparison. Honest result (balanced 3-way class, 1230 train / 246 val): llm_generation_acc = 0.919 vs shuffled-mesh baseline = 0.313 (majority 0.374) encoder_mesh_read_acc (aux head) = 0.785 The 0.61 grounding gap (vs the maxpool encoder's 0.22: 0.543 vs 0.322) shows the real architecture gives the LLM far stronger geometric signal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…AD (0.91+) The existing generators cannot produce valid CAD: the command-VAE's parallel z-broadcast decoder is primitive-limited / posterior-collapses (~0-12% valid) and the diffusion path samples faces independently so they never mate (0% valid). This adds the robust, proven route (DeepCAD/Text2CAD): generate the construction PROGRAM autoregressively and execute it, so the OCC kernel builds the solid command-by-command. A causal-transformer LM over the 268-token command vocabulary is teacher-forced on 38k real DeepCAD command sequences, then sampled autoregressively (temperature 1.0, top-k 20) -> decoded to command_dicts -> executed through the REAL OCC kernel. Validity is MEASURED and gated honestly against the cylinder trap: a sample counts only if it forms a solid (solid_count>=1) with non-degenerate volume (>1e-4); we also report num_distinct (rounded bounding boxes), top_shape_frac (mode-collapse guard), and the volume spread. Result (256 samples, real kernel): validity 0.914 (234/256), num_distinct 104, mean_vol 1.84, vol p10-p90 [0.15, 3.64] best 0.969 vs untrained baseline 0.023 (and vs VAE ~0.12 / diffusion 0.0). High distinct + wide volume spread => genuinely diverse, non-degenerate solids, not one repeated trivial shape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two coupled fixes surfaced while making the generators produce *measured*-valid CAD:
1. Deceptive validity metric. compute_validity_rate counted r.is_valid (BRepCheck)
only, which passes volume-less shells and zero-volume degenerates. An
independently-sampled-face generator therefore scored validity=1.0 while
producing ZERO real solids. Added GenerationMetrics.is_valid_solid (requires a
closed solid with positive volume when a geometry report exists; falls back to
is_valid for abstract test stand-ins) and routed validity_rate, num_valid and
distinct_valid through it. Updated the test fixture's "valid" result to carry a
real volume.
2. Diffusion sampling cpu/mps crash. StructuredDiffusion.sample() honored a caller
-supplied device that could disagree with the denoiser weights (a generator
whose .device attr was not updated after the model was .to()'d), crashing at the
first Linear ("input is on cpu but expected on mps"). sample() now always
computes on the weights' own device.
Effect: the diffusion eval RUNS now instead of crashing, and the honest metric
reports its true validity=0.000 (its independently-sampled faces never mate into a
solid) instead of the former hollow 1.000.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ffusion fix
The shipped StructuredDiffusion denoises raw B-rep geometry (independent face UV-grids
+ edge polylines) then tries to sew them; independently-sampled faces never mate, so
honest validity is 0. This replaces that with DeepCAD's actual generative design:
diffuse the CONSTRUCTION-PROGRAM latent and decode with an execution-respecting
autoregressive decoder, so the OCC kernel builds a watertight solid.
- SeqAutoencoder: deterministic (not a VAE -> no posterior collapse), encoder ->
z[64]; z-conditioned causal AR decoder trained teacher-forced with 50% word-dropout
so the latent must carry the global program.
- LatentDDPM: eps-prediction denoiser over the normalised z bank.
Honest acceptance bar = validity of DIFFUSION-SAMPLED z (noise -> denoise -> decode ->
execute), real OCC kernel, solid+volume gated, distinct>1, beating a z=0
predict-the-mean baseline:
sampled-z : validity 0.934 (239/256), distinct 138, top_shape 11.7%, mean_vol 1.32
z=0 base : validity 1.000 but distinct 14, top_shape 47.7% (near mode-collapse)
Validity comes from the decoder (both are high); the DIFFUSION's contribution is the
DISTRIBUTION — 138 distinct valid shapes vs the baseline's 14. It samples the diverse
space of valid programs instead of repeating the mean. (vs the old geometry-diffusion
honest validity 0.0.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Sorry @LayerDynamics, your pull request is larger than the review limit of 150000 diff characters
LayerDynamics
added a commit
that referenced
this pull request
Jun 11, 2026
docs(site): update to the trained-models + native-MLX reality (follow-up to #15)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes every ll_* model run natively on Apple Silicon (MLX) as a faithful port of the real code, and gives ll_gen generators that produce measured-valid CAD through the real OCC kernel. Every number here is verified — parity diffs, or validity gated on a non-degenerate solid (
solid_count≥1 AND volume>1e-4), not BRepCheck alone.Faithful MLX ports (load the real trained weights; parity-proven)
mlx/train_classification_mlx.py)stepnet_classifier.pt; argmax agreement 1.0, acc 0.976 == 0.976mlx/train_brepnet_mlx.py)best.ckpt; per-face agreement 1.0, mIoU 0.835 == 0.835mlx/vae_mlx.py)vae_warm.pt(227 tensors); encode→decode argmax 1.0, logit diff ~1e-6mlx/faithful_tower_mlx.py)Pattern: rebuild the exact architecture from real
nn.Linear/Embedding/LayerNorm/Conv(Conv needs OIHW→OHWI / OIW→OWI permute), implement MHA manually from the packedin_proj(incl. cross-attn for the VAE decoder), convert by name-map, and prove parity (argmax agreement ~1.0; ~1e-6 logit drift is benign).ll_ocadr retrained on the faithful tower
Replaced the maxpool stand-in with the real PointNet++/Point-BERT tower (256 mesh tokens) + LoRA on a frozen 4-bit Qwen2. llm_generation_acc 0.919 vs shuffled-mesh 0.313 (majority 0.374) — a 0.61 grounding gap vs the stand-in's 0.22.
ll_gen: generators that actually produce valid CAD
The command-VAE (parallel decoder) tops out ~12% and the geometry-diffusion produces 0 real solids (independently-sampled faces never mate). Both fixed by generating the construction program and executing it:
mlx/ar_generator_mlx.py): causal transformer LM over the command vocab, trained on 38k real DeepCAD programs → validity 0.914 (234/256), 104 distinct, non-degenerate.mlx/latent_diffusion_mlx.py) — the real diffusion fix: diffuse a program-autoencoder latent, decode autoregressively, execute. Sampled-z validity 0.934 (239/256), 138 distinct, top-shape 11.7% vs a z=0 predict-the-mean baseline (1.0 valid but only 14 distinct, near-collapse). Honest framing: validity comes from the decoder; the diffusion contributes the diverse latent prior (138 vs 14 distinct).Honesty fixes (genuine deception found)
compute_validity_ratecountedis_valid(BRepCheck) only, so volume-less degenerates passed — the broken diffusion scored 1.000 while producing 0 real solids. AddedGenerationMetrics.is_valid_solid(closed solid + positive volume) and routed validity/num_valid/distinct through it; the diffusion now honestly reports its true rate.StructuredDiffusion.sample): now always computes on the weights' device.Notes
<pkg>/mlx/; checkpoints/metrics write to gitignored<pkg>/checkpoints/. Reproduction scripts depend on local DeepCAD/Fusion360 data (gitignored).🤖 Generated with Claude Code