BPF compatibility, compiler fixes, and Nix build infrastructure#13
Open
randomizedcoder wants to merge 15 commits intoxdp2-dev:num_argsfrom
Open
BPF compatibility, compiler fixes, and Nix build infrastructure#13randomizedcoder wants to merge 15 commits intoxdp2-dev:num_argsfrom
randomizedcoder wants to merge 15 commits intoxdp2-dev:num_argsfrom
Conversation
Safe, self-contained build system hygiene with no functional change. Boost.System is header-only since 1.69 — try linking without -lboost_system first, falling back for older versions. - src/configure.sh: backtick→$(), quote basename args, exit -1→exit 1, PLATFORMS=($(ls))→mapfile -t - src/configure.sh: Boost Wave/Thread/System/Filesystem tests try header-only link first - .gitignore: add install/ directory (Nix build output) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Nix bypasses the cc-wrapper, so xdp2-compiler's ClangTool API misses system includes. extract_struct_constants() had no include path config at all, causing proto table extraction failures on Nix. Adds environment-variable-driven ClangToolConfig loaded from XDP2_C_INCLUDE_PATH, XDP2_GLIBC_INCLUDE_PATH, XDP2_LINUX_HEADERS_PATH. No-op on systems where these env vars are unset (Ubuntu/Fedora). - NEW src/tools/compiler/include/xdp2gen/clang-tool-config.h - NEW src/tools/compiler/src/clang-tool-config.cpp - NEW src/tools/compiler/include/xdp2gen/assert.h - src/tools/compiler/src/main.cpp: from_environment() + apply_config() for both create_clang_tool() and extract_struct_constants() - src/tools/compiler/Makefile: add clang-tool-config.o, drop -lboost_system (header-only since 1.69) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Nix clang 18.1.8 handles C tentative definitions differently from Ubuntu clang 18.1.3: hasInit() returns true with void-type InitListExpr. getAs<RecordType>() returns nullptr → segfault on ->getDecl(). Also, XDP2_MAKE_PROTO_TABLE creates two declarations per group, but HandleTopLevelDecl used isSingleDecl() which missed the second one. - proto-tables.h: iterate all decls in HandleTopLevelDecl; null-check getAs<RecordType>() before ->getDecl() - graph_consumer.h: add "proto_def" to fields of interest - log_handler.h: make static members inline (ODR fix, multi-TU) - processing_utilities.h: debug logging in connect_vertices - template.cpp: import sys + debug logging for graph vertex info Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root parse function expects pointer to frame, but template passed frame by value. Causes segfault only in optimized (-O) builds where the compiler optimizes away the copy. - src/templates/xdp2/common_parser.template.c: frame → &frame Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
XDP2 headers now compile when __bpf__ is defined (clang -target bpf). Replaces libc functions with __builtin_memset/__builtin_memcpy, maps htons/ntohs to bpf_htons/bpf_ntohs via new bpf_compat.h, and provides minimal struct definitions to avoid heavy kernel header dependency chains that fail under the BPF backend. Core change pattern: #ifndef __KERNEL__ becomes #if !defined(__KERNEL__) && !defined(__bpf__) - NEW src/include/xdp2/bpf_compat.h: byte-order compat for BPF/kernel/userspace; IPPROTO_* via linux/in.h - arrays.h, parser_metadata.h, tlvs.h: guard stddef.h, sys/types.h - bitmap.h: XDP2_BITMAP_BITS_PER_WORD from __SIZEOF_LONG__ (literal 32/64) — __BITS_PER_LONG is computed, breaks token pasting - parser.h: BPF memset/memcpy builtins; guard siphash - parser_types.h: BPF stddef.h+stdint.h from clang resource dir; ssize_t as __s64, bool as _Bool - utility.h: split includes into 3 contexts - proto_defs/proto_*.h: arpa/inet.h → bpf_compat.h; BPF-specific minimal struct definitions for ARP, ICMP, ICMPv6 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Allows sample binaries to find libxdp2.so at runtime without
LD_LIBRARY_PATH, which is required for Nix-based test automation.
- samples/parser/{simple,offset,ports}_parser/Makefile: -Wl,-rpath
- samples/xdp/flow_tracker_combo/Makefile: -Wl,-rpath
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replaces monolithic ~1000-line flake.nix with modular imports from nix/.
This commit covers native x86_64 build and devshell only — tests,
cross-compilation, and MicroVMs added in subsequent commits.
derivation.nix has patches = [] — source fixes from commits 2-3
supersede the patch files, which are kept as documentation of the
Nix-specific issues encountered.
- NEW nix/llvm.nix, packages.nix, env-vars.nix, derivation.nix,
devshell.nix
- NEW nix/shell-functions/{ascii-art,build,clean,configure,
navigation,validation}.nix
- NEW nix/patches/01-nix-clang-system-includes.patch,
02-tentative-definition-null-check.patch (not applied, docs only)
- flake.nix: refactored; outputs: xdp2, xdp2-debug, devShell
- flake.lock: updated nixpkgs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds Nix-based test infrastructure that builds samples from source
(native mode) or uses pre-built binaries (cross-compilation mode),
then runs them against pcap test data with expected output validation.
XDP sample compilation uses unwrapped clang to avoid Nix cc-wrapper
hardening flags that are incompatible with BPF target.
Tests: simple-parser, offset-parser, ports-parser, flow-tracker-combo
xdp-build test currently SKIPPED (blocked on BPF stack/API fixes).
- NEW nix/xdp-samples.nix: BPF bytecode compilation
- NEW nix/samples/default.nix: pre-built sample binaries
- NEW nix/tests/{default,simple-parser,offset-parser,ports-parser,
flow-tracker-combo,xdp-build,simple-parser-debug}.nix
- flake.nix: add xdp-samples, tests, run-sample-tests, per-test
convenience aliases
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Generates .deb package from nix build output for x86_64 distribution.
- NEW nix/packaging/{default,metadata,deb}.nix
- flake.nix: add deb-staging, deb-x86_64 outputs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
QEMU-based MicroVMs for full-system testing on x86_64, aarch64, and
riscv64. Expect-based automation handles VM boot, login, command
execution, and shutdown — required for cross-arch test runners that
cannot use binfmt for kernel-level XDP testing.
- NEW nix/microvms/{default,mkVm,lib,constants}.nix
- NEW nix/microvms/scripts/{vm-expect,vm-verify-service,vm-debug}.exp
- flake.nix: add microvm input; add microvms.* outputs + legacy flat
aliases for lifecycle scripts
- flake.lock: add microvm + spectrum inputs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cross-compiles xdp2 + samples for riscv64-linux. xdp2-compiler runs natively on x86_64 to generate .p.c files, which are then compiled with the RISC-V GCC toolchain. Tests run via binfmt/QEMU user-mode or inside MicroVMs. - NEW nix/cross-tests.nix: reusable cross-compilation module (included for future DRY refactoring; flake.nix currently inlines this) - flake.nix: add pkgsCrossRiscv, xdp2-debug-riscv64, prebuiltSamplesRiscv64, testsRiscv64, run-riscv64-tests (guarded by system == "x86_64-linux") Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Same cross-compilation pattern as RISC-V, targeting aarch64-linux. - flake.nix: add pkgsCrossAarch64, xdp2-debug-aarch64, prebuiltSamplesAarch64, testsAarch64, run-aarch64-tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Convenience make targets wrapping nix build/run commands for the full
build/test/cross-compilation matrix. Includes binfmt prerequisite
checks for cross-arch targets.
- NEW Makefile (332 lines): build, build-debug, samples, test,
test-{simple,offset,ports,flow}, riscv64, riscv64-samples,
test-riscv64, test-riscv64-vm, aarch64, aarch64-samples,
test-aarch64, test-aarch64-vm, vm-{x86,aarch64,riscv64},
vm-test-all, deb, dev, shell, check, eval, clean, gc
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- documentation/nix/nix.md: update with cross-compilation, MicroVM integration testing, sample test infrastructure, Makefile targets, and Debian packaging sections. Fix patches description (now historical, not applied). Fix file listing for new nix/ modules. - documentation/cpp-style-guide.md: C++ style guide for xdp2-compiler Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Documents verified test results across x86_64, RISC-V, and AArch64. All 38 parser/XDP sample tests pass on all architectures. xdp_build tests are SKIPPED pending BPF stack/API fixes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This was referenced Mar 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Commits (15)
Bug fixes (commits 1-4):
build:shellcheck fixes and Boost 1.69+ header-only compat in configure.shcompiler:ClangTool environment-based include path config (XDP2_C_INCLUDE_PATH, etc.)compiler:fix proto-table multi-decl and tentative definition null-check (Nix clang 18.1.8)templates:fix optimized parser segfault — pass &frame not frameBPF compatibility (commit 5):
5.
headers:add BPF target compatibility — __builtin_memset/memcpy, bpf_compat.h byte-order, minimal struct defs for ARP/ICMP/ICMPv6Build support (commit 6):
6.
samples:add rpath to LDFLAGS for runtime library resolutionNix infrastructure (commits 7-12):
7.
nix:refactor flake.nix into modular build infrastructure (llvm.nix, packages.nix, derivation.nix, devshell.nix, env-vars.nix, shell-functions/)8.
nix:add sample build, XDP BPF compilation, and test infrastructure (tests/, samples/, xdp-samples.nix)9.
nix:add Debian packaging (packaging/)10.
nix:add MicroVM test infrastructure — QEMU-based VMs with expect automation for x86_64, aarch64, riscv6411.
nix:add RISC-V cross-compilation and tests (HOST compiler generates .p.c, TARGET GCC compiles)12.
nix:add AArch64 cross-compilation and testsConvenience and docs (commits 13-15):
13. Top-level Makefile with build/test/cross-compilation targets
14. Documentation: updated nix.md with cross-compilation and MicroVM sections, C++ style guide
15. Cross-architecture test summary (38/38 PASS on all architectures)
Test Results
xdp_build is blocked on BPF stack limits and template API mismatch (documented in test-summary.md).
Test plan
nix run .#run-sample-tests— all native x86_64 tests passnix build .#riscv64-tests.all+ binfmt execution — all RISC-V tests passnix build .#aarch64-tests.all+ binfmt execution — all AArch64 tests passnix flake check— flake validationmake eval— all flake outputs evaluate🤖 Generated with Claude Code