[arch] ADR-014 — Inter-card link architecture choice (multi-card parallelism)

[marcos-mendez]

User requirement (2026-05-05): multiple PopSolutions Sails cards must aggregate compute capacity. 'várias destas placas precisam funcionar em paralelo ou seja deve existir algum mecanismo de paralelização de processos para que estas placas possam juntas somar capacidade de processamento.'

This is a first-class architectural requirement on equal footing with cooperative governance (ADR-010), license stack (ADR-001), and verification methodology (Verilator + cocotb).

Decision needed

Which inter-card link architecture does MAST commit to?

Option	Cost	Bandwidth	Open-source IP	Notes
PCIe peer-to-peer	cheapest (already in PCIe shim)	host-mediated, 8 GB/s on Gen4 x4	LitePCIe + standard kernel drivers	starts free; bottlenecks on host CPU
CXL.cache / CXL.mem	high (datacenter spec)	very high, coherent	nascent open implementations	aligns with future trajectory but immature
Custom LVDS over backplane	low (we own the protocol)	moderate, scales with lane count	we write it; minimal upstream	maximum freedom; requires software stack from scratch
Hybrid	medium	best of both	mixed	PCIe for setup/control, custom for fast path

Tasks

• Survey state of open-source CXL implementations (LiteX, Migen-based, OpenCAPI-derived)
• Bandwidth model for the three reference workloads (TinyLlama-1.1B, Llama-7B, Llama-70B) under tensor parallelism with N=2/4/8 cards
• Decision ADR (docs/popsolutions/adr/0014-intercard-link.md)
• Spawn follow-up issues for the chosen path (RTL interconnect skeleton, PHY constraints, connector spec)

References: project memory entry project_multicard_parallelism.md.

[marcos-mendez]

PR #16 opened — research-only, no decision committed.

Per the operating model, architectural decisions on equal footing with ADR-001 / ADR-010 need cooperative ratification. The PR is a research document with my recommendation (Hybrid: GbE control + custom LVDS data for rev-A) plus four explicit ratification-gate questions for the cooperative.

Headline takeaways:

• Bandwidth model says GbE is sufficient for TinyLlama / Llama-7B at 30 t/s ×N=8, marginal for Llama-70B (~700 Mbps on a 940 Mbps practical link).
• ECP5 SerDes (4 lanes × 3.125 Gbps) gives ~0.8–1.0 GB/s effective — 10× headroom over Llama-70B inference need.
• CXL is structurally unavailable on ECP5 (needs PCIe Gen3+). Possibly available on CertusPro-NX rev-B+ if the open-source CXL FPGA ecosystem matures (uncertain).
• Recommendation locks no protocol details; the intercard_link skeleton (PR feat(interconnect): inter-card link RTL skeleton + cocotb tests #14, merged) already covers the interface contract for the Stays PCB.

Status stays DRAFT — NOT YET ACCEPTED until Marcos + cooperative answer the four §8 questions.

Per Agent R's prior directive, I'm flagging this as needing a human-attention label on top of the review-pending/stream-1 ones I attached. (I can't self-add that label cleanly without approving the merge surface; Agent R / human can edit.)

Authored by Agent 1 (RTL Architect).