Skip to content

feat: deepen FromELF into ProbeELF + RequireMinKernel + check --from-elf#56

Open
leodido wants to merge 11 commits intomainfrom
feat/from-elf-deepen
Open

feat: deepen FromELF into ProbeELF + RequireMinKernel + check --from-elf#56
leodido wants to merge 11 commits intomainfrom
feat/from-elf-deepen

Conversation

@leodido
Copy link
Copy Markdown
Owner

@leodido leodido commented May 4, 2026

Motivation

FromELF only returned what Check(...) consumes (program / map types and helper-per-program requirements). That's enough to gate, but not enough to explain why an object will or won't load on a target kernel: which helpers are superseded, which loads aren't CO-RE-protected, what minimum kernel version the object actually requires.

This PR keeps FromELF frozen against its 4-point contract and adds ProbeELF (and ProbeELFWith(opts...)) as the strict superset, plus a parameterized RequireMinKernel(major, minor) requirement, plus the CLI affordances to drive both.

What this changes

Library

  • ProbeELF(path) / ProbeELFWith(path, opts...): returns a richer *ELFProbes snapshot — programs, maps, helper-per-program requirements, derived minimum kernel version, advisory warnings (superseded helpers), and (with WithCOREChecks()) a per-program memory-access classification distinguishing context loads, map-value loads, CO-RE-protected loads, and unprotected kernel-direct loads. Requirements() projects to a FeatureGroup consumable by Check(...).
  • RequireMinKernel(major, minor): parameterized requirement gated against uname -r. Composes with the kernel-version snapshot so Probes.Requirements() automatically rejects objects whose helpers, program types, or map types were introduced after the running kernel.
  • internal/kernelversions: generated, pure-Go tables of the minimum kernel version that introduced each BPF helper, program type, and map type. Sourced from BCC's kernel-versions.md and Linux UAPI bpf.h at pinned commits via internal/kernelversions/cmd/kvgen. The generator cross-validates UAPI against BCC and refuses to emit a partial snapshot; audited cross-validation gaps live in known_gaps.go with one-line rationales.
  • FromELF itself is unchanged — its 4-point contract (cilium parser, parser-only, returns FeatureGroup, deterministic + fail-closed) stays frozen. New extraction surface goes on ProbeELF.

CLI

  • kfeatures probe is now a parent command. probe host is the canonical name for the live-kernel probe; bare probe keeps working as an alias by sharing flag state with the host leaf.
  • kfeatures probe bpf <path.o> runs the ELF probe on a compiled object. --with-core opts in to CO-RE memory-access classification; --requirements prints only the Check-compatible projection; --json switches both to JSON.
  • kfeatures check --from-elf <path.o> accepts an ELF object as a requirement source; --require and --from-elf are now both optional and at-least-one-required.
  • MCP tool list: check, config, probe-bpf, probe-host. The bare probe alias is a CLI ergonomic affordance only; agents see the canonical names.

CI

  • New .github/workflows/refresh-kernel-versions.yml: weekly cron (Mon 06:00 UTC) plus workflow_dispatch with optional SHA overrides. Resolves upstream HEAD for both repos, bumps the pinned defaults in cmd/kvgen/main.go, regenerates the snapshot, verifies with go vet + kernelversions test package, and opens a dependencies-labelled PR (titled deps(kernelversions): refresh BCC + UAPI snapshot) only when the snapshot drifted.

Coverage gate

  • New make cover-check target plus internal/tools/covercheck enforce a per-file coverage threshold (default 90%) on the files listed in the makefile's COVER_FILES variable. The checker honours a single-line // coverage:ignore marker placed in the doc comment of a func declaration or a var foo = func(...) declaration. Wired into CI as a new "Coverage gate" step.
  • golang.org/x/tools promoted from indirect to direct because covercheck imports golang.org/x/tools/cover.

Documentation

  • README: runnable examples for ProbeELFWith(WithCOREChecks()), RequireMinKernel, the new CLI surface; updated MCP tools list and detection / comparison tables.
  • CONTRIBUTING: requirement-items table extended; new sections on the kernel-version snapshot lifecycle and the coverage gate.
  • AGENTS: agent-actionable rules for the kernel-version snapshot (don't hand-edit, use the bot, when to grow known_gaps.go) and for the coverage gate.
  • CHANGELOG: five new [Unreleased] Added entries.

Stability

  • FromELF contract is unchanged (still parser-only, still cross-platform, still returns FeatureGroup, still deterministic + fail-closed).
  • ProbeELF is additive — no existing call site of FromELF needs to migrate.
  • RequireMinKernel is additive — composes with the existing Requirement model.
  • probe CLI alias is preserved byte-for-byte; agents that hard-code probe keep working.
  • check: --require is no longer marked flagrequired, and the runtime check now permits --from-elf instead. Old --require=… invocations still work; the missing-flag exit code changes from 10 (structcli missing_required_flag) to 1 (typed no features specified).

Verification

  • go vet ./... clean
  • go test -race -count=1 ./... 4/4 packages green
  • make cover-check all 6 gated files ≥ 90%
  • bats test/cli_common.bats test/cli_linux.bats test/cli_mcp.bats 44/44
  • CLI smoke (six invocation paths including probe, probe host, probe bpf, check --from-elf, bare check, unknown subcommand) returns the right exit codes and envelopes
  • MCP tools/list returns ['check', 'config', 'probe-bpf', 'probe-host']

Commits

11 logical commits, one per spec step. The two CLI commits split cmd/kfeatures/main.go and the bats files between probe-restructuring (#7) and check --from-elf (#8).

leodido and others added 11 commits May 4, 2026 23:39
Generated, pure-Go tables mapping every BPF helper, program type, and
map type to the minimum kernel version that introduced it. Sourced
from BCC's kernel-versions.md and Linux UAPI bpf.h at pinned commits
via internal/kernelversions/cmd/kvgen, which cross-validates UAPI
against BCC and refuses to emit a partial snapshot.

Audited cross-validation gaps live in known_gaps.go with one-line
rationales. Consumers go through HelperKernelVersion /
MapTypeKernelVersion / ProgramTypeKernelVersion accessors, never
import tables.go directly.

Co-authored-by: Ona <no-reply@ona.com>
Adds ELFProbes / ELFProgram / ELFMap / ELFHelperRequirement /
ELFProgramTypeRequirement / ELFMapTypeRequirement / KernelVersion /
MemoryAccessSummary / ELFWarning value types and the Requirements()
method that projects them to a FeatureGroup consumable by Check(...).

Requirements() is the only path callers should use to gate on an ELF
snapshot; the per-row metadata (versions, warnings, memory-access
summaries) is for inspection and reporting. nil-receiver guard returns
an empty FeatureGroup so callers can chain through Probe() failures.

Co-authored-by: Ona <no-reply@ona.com>
ProbeELF(path) and ProbeELFWith(path, opts...) parse a compiled eBPF
object via cilium/ebpf and populate an *ELFProbes snapshot.
probesFromCollectionSpec is the inner helper that operates on an
already-parsed *ebpf.CollectionSpec, so unit tests can drive every
branch programmatically without clang or a real .o on disk.

Output is deterministic (program / map names sorted, helpers deduped
per program and globally, helpers sorted by version desc with helper
id as secondary key). Unknown program / map / helper kinds fail
closed.

ProbeELFWith carries // coverage:ignore: its disk-load wrapper needs
clang to exercise; the per-branch behaviour is covered through
programmatic CollectionSpec fixtures.

Co-authored-by: Ona <no-reply@ona.com>
Flags calls to bpf_probe_read / bpf_probe_read_str (split into
bpf_probe_read_kernel* / bpf_probe_read_user* in 5.5) and
bpf_get_current_task (subsumed by bpf_get_current_task_btf in 5.11).
init() binds the rule table to supersededHelperWarnings so the
extractor in probe_elf_extract.go picks it up without a circular
import.

Advisory only: warnings populate ELFProbes.Warnings but never gate
Check(...) verdicts.

Co-authored-by: Ona <no-reply@ona.com>
ProbeELFWith(path, kfeatures.WithCOREChecks()) opts in to a per-
program memory-access classifier. Walks instructions once tracking
per-register provenance (Context / MapValue / KernelDirect /
COREProtected), classifies each load by source, and clobbers R1-R5
after every helper call.

Produces a MemoryAccessSummary per program (Context / MapValue / CORE
Protected / KernelDirect / Uncategorized / Total) plus per-program
ELFWarning entries for unprotected kernel-direct loads. The
accessNotLoad zero-value sentinel ensures non-load instructions are
not counted in Total.

cilium/ebpf v0.20 does not expose btf.LineInfoMetadata, so lineInfo()
is currently a stub and warnings drop file/line attribution. To be
revisited when cilium/ebpf re-exposes the metadata.

Co-authored-by: Ona <no-reply@ona.com>
RequireMinKernel(major, minor) is a new Requirement item gated by
Check(...) against uname -r. Composes with the kernel-version
snapshot under internal/kernelversions, so ELFProbes.Requirements()
can automatically reject an object whose helpers, program types, or
map types were introduced after the running kernel.

parseKernelRelease handles the common release-string formats
("6.1.0-generic", "6.1.0-1.el9.x86_64", etc.). Constructor panics
on negative version components.

Adds the requirementSet plumbing (minKernels slice, seenMinKernels
dedup map, MinKernelRequirement case in add) and the gating loop in
Check that emits a FeatureError when uname -r is below the required
version.

Co-authored-by: Ona <no-reply@ona.com>
`kfeatures probe` is now a parent command. `probe host` is the
canonical name for the live-kernel probe and exposes the same
`--json` flag as the v0.5.x `probe`. Bare `kfeatures probe` keeps
working byte-for-byte by reusing the host leaf's RunE and mirroring
the leaf's flag set onto the parent (cmd.Flags().AddFlagSet); the two
share a process-global `probeHostOpts` pointer so flag values land in
the same memory regardless of which surface parsed them.

`probe bpf <path.o>` runs ProbeELF on a compiled eBPF object and
prints the resulting *ELFProbes snapshot. Flags: `--with-core` opts
in to the CO-RE memory-access classifier, `--requirements` prints
only the FeatureGroup projection consumed by Check(...), `--json`
switches both to JSON.

MCP exposure: structcli's MCP registry only registers runnable
leaves, so the parent `probe` is auto-excluded by
shouldIncludeMCPCommand and tools/list returns
`probe-host` / `probe-bpf` (alongside check / config). The bare
`probe` alias is a CLI ergonomic affordance only; agents see the
canonical names.

Bats coverage: cli_common updated for the new help text and host/bpf
help leaves; cli_linux gains parity, --json, missing-arg, and
nonexistent-file tests for the new commands; cli_mcp asserts the new
tools/list shape and uses `probe-host` for live-kernel tools/call
cases.

Co-authored-by: Ona <no-reply@ona.com>
`kfeatures check --from-elf <path.o>` derives the FeatureGroup from
a compiled eBPF object via kfeatures.FromELF and gates on it. Composes
with --require: the union of both is gated. The two flags are now
both optional and at-least-one-required (drops `flagrequired:"true"`
from --require; the runtime check returns the same "no features
specified" message wrapped in the structcli envelope).

Extracted assembleCheckRequirements out of the RunE so the union
logic can be unit-tested without going through cobra.

Bats coverage: cli_common gains a missing-file test for --from-elf
and a bare-invocation test asserting the new error message; the old
"missing required flag exit 10" test is removed (the flag is no
longer flagrequired).

Co-authored-by: Ona <no-reply@ona.com>
Weekly cron (Mon 06:00 UTC) plus workflow_dispatch with optional SHA
overrides. Resolves iovisor/bcc and torvalds/linux HEAD via
`git ls-remote`, rewrites the defaultBCCCommit / defaultKernelCommit
constants in cmd/kvgen/main.go, regenerates the snapshot, and
verifies the result with `go vet` plus the kernelversions test
package.

Opens a PR via peter-evans/create-pull-request@v6.1.0 only when
internal/kernelversions actually drifted; the
`deps(kernelversions): refresh BCC + UAPI snapshot` title prefix
routes the PR into the dependencies bucket of release notes via the
release.yml configuration, and the `dependencies` label is also
applied explicitly as a belt-and-braces measure.

The PR body surfaces both pinned SHAs and references
internal/kernelversions/cmd/kvgen/known_gaps.go as the escape hatch
when UAPI/BCC drift causes cross-validation failures.

Co-authored-by: Ona <no-reply@ona.com>
…on snapshot

README: adds runnable examples for ProbeELFWith(WithCOREChecks()),
RequireMinKernel, the new `probe host` / `probe bpf` /
`check --from-elf` invocations; updates the MCP tools list to
`probe-host` / `probe-bpf`; refreshes the detection table and
comparison table to mention CO-RE memory-access classification and
min-kernel derivation.

CONTRIBUTING: adds RequireMinKernel and ProbeELF / ProbeELFWith to
the requirement-items table; clarifies that FromELF stays frozen
against its 4-point contract while new extraction surface (warnings,
CO-RE) belongs on ProbeELF; new section on the kernel-version
snapshot lifecycle (auto-refresh workflow, manual go generate, the
known_gaps.go allow-list policy); documents the make cover-check
gate.

AGENTS: adds the agent-actionable rules for the kernel-version
snapshot (don't hand-edit, use the bot, when to grow known_gaps.go,
go through the accessor functions) and for the coverage gate (when
to extend COVER_FILES, when a // coverage:ignore marker is
acceptable, the rule against bumping COVER_THRESHOLD per-file).

CHANGELOG: five new `[Unreleased] Added` entries covering ProbeELF,
RequireMinKernel, the kernel-version snapshot + auto-refresh
workflow, superseded-helper warnings, and the new CLI surface.

Co-authored-by: Ona <no-reply@ona.com>
`make cover-check` runs the test suite with -coverprofile=coverage.out
and then runs internal/tools/covercheck against that profile, failing
if any gated source file falls below COVER_THRESHOLD (default 90).
The gated set lives in the COVER_FILES makefile variable; it covers
the public/library files added in this branch only. Internal tools
(kvgen, covercheck itself) are intentionally excluded — their happy
paths are exercised by the scheduled refresh workflow against live
data.

The checker honours a single-line `// coverage:ignore` marker placed
in the doc comment of a function declaration or of a
`var foo = func(...)` declaration. The marker excludes every
statement attributed to that function from both numerator and
denominator. Used for the disk-bound ProbeELFWith wrapper (branches
exercised through programmatic CollectionSpec fixtures against the
inner probesFromCollectionSpec) and for the test-time stub variables
overwritten by package init functions.

Wired into ci.yml as a new "Coverage gate" step that runs after the
test step. `golang.org/x/tools` promoted from indirect to direct
because covercheck imports `golang.org/x/tools/cover`.

Co-authored-by: Ona <no-reply@ona.com>
@leodido leodido added the enhancement New feature or request label May 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant