Skip to content

feat(epaxos): conflict engine core with model-equivalence tests#2

Merged
metaphorics merged 12 commits into
mainfrom
feat/conflict-engine-core
Jul 14, 2026
Merged

feat(epaxos): conflict engine core with model-equivalence tests#2
metaphorics merged 12 commits into
mainfrom
feat/conflict-engine-core

Conversation

@metaphorics

@metaphorics metaphorics commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Solution / proposal

Standalone sparse 64-ary radix per-lane conflict engine with per-key O(log) postings, foldRecord/advanceFold, retired floors, and property tests (oracle equivalence, fold domination).

Diagnosis addressed

D1 seq scan, D5 string keys, foundation for D2/D6 elimination.

Closes #

Summary by Sourcery

Introduce a per-lane conflict engine and related safety hardening while tightening validation, error handling, and linting across EPaxos, examples, and test harnesses.

Enhancements:

  • Add a standalone sparse radix-based per-lane conflict engine with model-equivalence and fold-domination property tests.
  • Tighten EPaxos message, timing, configuration, and bootstrap validation logic and make previously unchecked paths return or propagate errors instead of panicking silently.
  • Refine timestamp-bounds handling, checkpoint manifest parsing, and backup/restore verification to handle edge cases, overflows, and invalid metadata more defensively.
  • Improve test harnesses, fault-simulation, and refinement tracing code for determinism, clearer errors, and stricter invariants without changing external behavior.
  • Document and codify stricter development practices via golangci-lint configuration, CI integration, and agent task goals.

CI:

  • Integrate golangci-lint into CI for the main module and examples/kv to enforce a stricter static-analysis baseline.

Documentation:

  • Add AGENTS and task-goal documentation describing required local validations and per-task goals for EPaxos conflict GC work.

Tests:

  • Add comprehensive conflict engine property tests plus additional boundary, overflow, and malformed-input tests for codecs, bootstrap messages, ballots, and timing logic.
  • Extend campaign, lifecycle, and simulation tests to assert stronger invariants around metrics, HTTP probes, TLS, artifacts, and trace integrity.

Closes #9

Drop govet enable-all; keep revive/gosec/exhaustive enabled for R9.
Make epaxos package lint-clean under the U1 strict config: errcheck on Tick
paths, revive renames, intentional test nolints for G115/G304/exhaustive
subsets, and leave production wire G115 checks intact.
Root golangci-lint does not load the nested module; add a second CI step
and document the local dual-module lint commands in AGENTS.md.
Mirror parser.uvarint8 for bootstrap envelopes so oversized wire types
cannot truncate into the valid BootstrapMessageType band.
Re-check depsCount against maxWireDeps before int conversion and
arena slicing so a hostile evidence frame cannot panic the node.
MkdirAll/WriteFile already use restrictive 0o750/0o600; the G115
suppressions were false and would hide future mode regressions.
Standalone per-lane sparse radix max-seq index and per-key postings with
fold summaries; property tests prove oracle equivalence and fold domination.
@sourcery-ai

sourcery-ai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Reviewer's Guide

Introduce a standalone per-lane conflict engine with radix/posting data structures and property-style tests, while tightening protocol/message/bootstrap validation, improving error handling and security hardening across tests, examples, and tools, and wiring in stricter linting via golangci-lint and CI updates.

Sequence diagram for conflict engine apply and foldRecord/advanceFold

sequenceDiagram
    participant RawNode
    participant conflictEngine
    participant laneIndex
    participant laneTree
    participant keyLane
    participant postingSet

    RawNode->>conflictEngine: apply(prev *InstanceRecord, rec InstanceRecord)
    conflictEngine->>conflictEngine: laneFor(rec.Ref)
    conflictEngine->>conflictEngine: slotForRecord(rec)
    conflictEngine->>conflictEngine: recordConflictEligible(rec)
    conflictEngine->>conflictEngine: commandHasGlobalConflictScope(rec.Command.Kind)
    conflictEngine->>conflictEngine: ensureLane(lane)
    conflictEngine->>laneTree: set(rec.Ref.Instance, slot)
    alt recordConflictEligible(rec) && !commandHasGlobalConflictScope(rec.Command.Kind)
        loop for each key in rec.Command.ConflictKeys
            conflictEngine->>conflictEngine: ensureKeyLane(rec.Ref.Conf, key, lane)
            conflictEngine->>keyLane: postings.insert(rec.Ref.Instance)
        end
    end

    RawNode->>conflictEngine: foldRecord(rec InstanceRecord)
    conflictEngine->>conflictEngine: laneFor(rec.Ref)
    conflictEngine->>conflictEngine: ensureLane(lane)
    conflictEngine->>conflictEngine: remove(rec.Ref, rec)
    conflictEngine->>postingSet: insert(rec.Ref.Instance)
    conflictEngine->>conflictEngine: recordConflictEligible(rec)
    alt recordConflictEligible(rec)
        conflictEngine->>conflictEngine: addRetiredSeq(rec.Ref.Instance, rec.Seq)
        alt commandHasGlobalConflictScope(rec.Command.Kind)
            conflictEngine->>conflictEngine: update retiredGlobal
        else non-global keys
            loop for each key in rec.Command.ConflictKeys
                conflictEngine->>conflictEngine: ensureKeyLane(rec.Ref.Conf, key, lane)
                conflictEngine->>keyLane: update retiredFloor
            end
        end
    end

    RawNode->>conflictEngine: advanceFold(lane, through InstanceNum)
    conflictEngine->>conflictEngine: ensureLane(lane)
    conflictEngine->>postingSet: contains(instance)
    conflictEngine->>postingSet: remove(instance)
    conflictEngine->>conflictEngine: update folded
Loading

File-Level Changes

Change Details Files
Add a per-lane conflict engine with radix-based lane trees and posting sets, plus model-equivalence and fold-domination property tests.
  • Implement conflictEngine with radixNode/laneTree and postingSet structures to index instances per lane and per key without linear scans.
  • Support foldRecord/advanceFold and retired sequence breakpoint tracking for executed-instance retirement while preserving conflict ordering semantics.
  • Provide verification helpers and randomized tests (model-equivalence against a naïve model and fold-domination properties) to assert correctness and monotonicity of conflict results.
epaxos/conflict_engine.go
epaxos/conflict_engine_test.go
.agent-tasks/epaxos-conflict-gc/GOALS.md
Harden message/bootstrap/record codecs and validation against overflows, oversized enums, and malformed inputs.
  • Add bounded uvarint8 parsing for enum-like fields and enforce max lengths/counts in message and bootstrap decoders.
  • Tighten Message.Validate, bootstrap/membership validators, and configuration-history reconstruction to check previously implicit cases and reject inconsistent or unspecified outcomes.
  • Improve EPaxos record decode/encode paths (including timing metadata and command kinds) with additional range checks and negative-size protections for checkpoint manifests and files.
epaxos/codec.go
epaxos/bootstrap.go
epaxos/bootstrap_fence_test.go
epaxos/bootstrap_core_test.go
examples/kv/epaxos_storage.go
examples/kv/backup.go
examples/kv/checkpoint_hard_state_test.go
examples/kv/epaxos_record_timing_codec_test.go
epaxos/message.go
epaxos/bootstrap_recovery_test.go
Make Tick/timer/start* paths error-returning and plumb error handling through tests and helper code.
  • Change RawNode.Tick, timer handlers, and scheduling helpers to return errors and update tests to assert on errors instead of ignoring them.
  • Adjust startAccept/startTryPreAccept and various recovery/coverage paths to panic only in logically unreachable clock-exhaustion cases, relying on Tick to guard logical time.
  • Update simulation, stress tests, and DST harnesses to check Tick errors and propagate failures via t.Fatal or panic in test-only code.
epaxos/node.go
epaxos/recovery_test.go
epaxos/remaining_test.go
epaxos/protocol_coverage_test.go
epaxos/branch_test.go
epaxos/hard_state_ready_test.go
epaxos/internal_test.go
epaxos/sim_test.go
epaxos/stress_test.go
epaxos/dst_test.go
epaxos/toq_test.go
Improve error handling, security hardening, and diagnostics in faultcampaign, lifecyclecollector, and kvcheckpoint tooling.
  • Propagate and wrap errors with %w, split status and error conditions, and refine messages for metrics/admission/retention checks and readiness probes.
  • Add gosec suppressions where appropriate and tighten file/dir permissions, path handling, and subprocess execution in test harnesses and campaign tools.
  • Make kvcheckpoint CLI usage and error printing I/O-safe, ensure report/manifest writes create parent dirs with restricted modes, and refine configuration-history reconstruction errors.
tests/faultcampaign/campaign.go
tests/faultcampaign/process.go
tests/faultcampaign/artifacts.go
tests/faultcampaign/trace.go
tests/faultcampaign/main.go
tests/faultcampaign/proxy.go
tests/faultcampaign/proxy_test.go
tests/faultcampaign/native_darwin.go
tests/faultcampaign/native_other.go
tests/faultcampaign/history.go
tests/faultcampaign/invariants.go
tests/faultcampaign/artifacts_test.go
tests/lifecyclecollector/collect.go
tests/lifecyclecollector/model.go
tests/lifecyclecollector/finalize.go
tests/lifecyclecollector/collector_test.go
examples/kv/cmd/kvcheckpoint/main.go
examples/kv/cmd/kvcheckpoint/main_test.go
Tighten protocol/bootstrap/test helpers and add or refine property/fixture tests for edge conditions.
  • Annotate non-exhaustive switch statements with nolint and add explicit default/unsupported handling for Status/MessageType/RejectReason cases in tests and helpers.
  • Refine bootstrap/membership tests (including oversized bootstrap type and malformed message fuzzing) to assert correct error types and residue-free decode failures.
  • Clarify and harden refinementtrace and fault-sim trace handling, including git metadata discovery, JSON reading/writing, and replay invariants.
epaxos/recovery_boundary_test.go
epaxos/faultsim_harness_test.go
epaxos/faultsim_oracle_test.go
epaxos/faultsim_trace_test.go
epaxos/fault_campaign_test.go
epaxos/codec_evidence_scratch_test.go
epaxos/malformed_fuzz_test.go
tests/refinementtrace/scenarios.go
tests/refinementtrace/trace.go
tests/refinementtrace/semantic.go
tests/refinementtrace/inventory_test.go
Introduce and enforce stricter linting configuration and CI wiring, plus minor structural cleanups.
  • Add a repository-wide golangci-lint configuration and wire golangci-lint into the GitHub Actions CI workflow for both the root module and examples/kv.
  • Adjust code to satisfy linters (e.g., gosec, gocritic, staticcheck) via minor refactors, explicit nolint annotations, and clearer variable naming.
  • Document agent tasks and goals for the EPaxos conflict-engine/GC work and add AGENTS.md with standard test/lint commands.
.github/workflows/ci.yml
.golangci.yml
AGENTS.md
.agent-tasks/epaxos-conflict-gc/GOALS.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@metaphorics

Copy link
Copy Markdown
Contributor Author

Closes #9

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="examples/kv/kv.go" line_range="122-128" />
<code_context>

 func (b TimestampBounds) classify(ts uint64) timestampMatch {
 	switch b.mode {
+	case timestampLatest:
</code_context>
<issue_to_address>
**issue (bug_risk):** TimestampBounds.classify lacks a return for timestampLatest, which will not compile.

The new `case timestampLatest` branch only has a comment and does not return a `timestampMatch`, unlike the other cases and with no default/trailing return. With `mode == timestampLatest`, this path has no return and will not compile in Go. Add an explicit return (e.g., `timestampVisible` or the intended value) in this case.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread examples/kv/kv.go
Comment on lines 122 to 128
func (b TimestampBounds) classify(ts uint64) timestampMatch {
switch b.mode {
case timestampLatest:
// All timestamps are visible under latest mode.
case timestampAtOrBefore:
if ts > b.max {
return timestampTooNew

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): TimestampBounds.classify lacks a return for timestampLatest, which will not compile.

The new case timestampLatest branch only has a comment and does not return a timestampMatch, unlike the other cases and with no default/trailing return. With mode == timestampLatest, this path has no return and will not compile in Go. Add an explicit return (e.g., timestampVisible or the intended value) in this case.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a per-lane conflict engine using radix trees and posting sets to optimize conflict and attribute computation, replacing the previous O(all instances) scan machinery. It also adds extensive linting annotations, refactors error handling, and ensures exhaustive enum dispatching across the codebase. A review comment suggests a performance improvement in keyLaneSet by hoisting a map lookup out of nested loops to avoid redundant operations.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread epaxos/conflict_engine.go
Comment on lines +644 to +664
func (e *conflictEngine) keyLaneSet(conf ConfID, keys [][]byte, yield func(instanceLane) bool) {
for lane := range e.laneIndex {
if lane.conf != conf {
continue
}
for _, key := range keys {
byConf := e.byKey[conf]
if byConf == nil {
break
}
byLane := byConf[string(key)]
if byLane == nil || (*byLane)[lane] == nil {
continue
}
if !yield(lane) {
return
}
break
}
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The map lookup e.byKey[conf] only depends on conf, which is constant throughout the execution of keyLaneSet. Hoisting this lookup outside of the loops avoids repeating the lookup for every single lane and key, improving the query's efficiency.

func (e *conflictEngine) keyLaneSet(conf ConfID, keys [][]byte, yield func(instanceLane) bool) {
	byConf := e.byKey[conf]
	if byConf == nil {
		return
	}
	for lane := range e.laneIndex {
		if lane.conf != conf {
			continue
		}
		for _, key := range keys {
			byLane := byConf[string(key)]
			if byLane == nil || (*byLane)[lane] == nil {
				continue
			}
			if !yield(lane) {
				return
			}
			break
		}
	}
}

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4295ec2bf8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread epaxos/conflict_engine.go
Comment on lines +491 to +495
type conflictEngine struct {
laneIndex map[instanceLane]*laneIndex
byKey map[ConfID]map[string]*keyLanes
resident int
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Wire the conflict engine into RawNode

When this commit is used in production, this new engine is unreachable: RawNode still stores and queries the old conflicts/allConflicts/globalConflicts maps, and computeAttrsAt still falls back to scanning n.instances; there is also no VisitConflicts API exposed. This means the added O(log n) conflict structure only runs in conflict_engine_test.go, so workloads with many resident instances keep the old O(all instances) behavior and embedders cannot use the promised conflict query surface.

Useful? React with 👍 / 👎.

@metaphorics
metaphorics merged commit 643dc75 into main Jul 14, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(epaxos): conflict engine core

1 participant