Gloas lookup sync by dapplion · Pull Request #68 · dapplion/lighthouse

dapplion · 2026-03-09T05:22:29Z

Summary

Fixes for Gloas lookup sync + PayloadEnvelopesByRoot RPC wiring.

Rebased on current sigp/unstable.

Changes

Rewrite single block lookup state machine for Gloas (AwaitingParent, PayloadRequest states)
Fix genesis parent check + infinite retry loop
Address full parent missing payload (envelope_is_known_to_fork_choice)
Fix on_completed_request no-op + clear all peer sets on disconnect
Wire PayloadEnvelopesByRoot RPC: replaces the NoRequestNeeded placeholder with actual request sending, response routing, and state machine updates

Test plan

Compiles on current unstable
Test on local Kurtosis ePBS devnet
Verify envelope downloads during single block lookups at head

pawanjay176

I like the simplicity of the new approach and the removal of the Request state trait.
First pass, will continue tomorrow.
Also, curious if you want to wire up the payload processing in this PR or leave it to a future PR. I'm fine with both.

pawanjay176 · 2026-04-18T00:50:49Z

+            peers,
+            peer_type,
+            cx.next_id(),
+            awaiting_parent.map(AwaitingParent::PreGloas),


This is a bit confusing.

Could AwaitingParent be a struct instead of an enum?

struct AwaitingParent { parent_hash: Hash256, parent_root: Option<ExecutionBlockHash> }

I find the pattern of creating an AwaitingParent::PreGloas variant by default and adjusting it later very confusing. a gloas parent_hash not containing a parent_root is conceptually confusing as well.

With the struct approach, adding a parent root later on as we figure out its gloas seems clearer to me.

pawanjay176 · 2026-04-18T01:14:04Z

+            "Block download response"
+        );
+
+        let result = lookup.on_block_download_response(id.req_id, response.map_err(|_| ()), cx);


I don't get why we are swallowing the errors here and in the other on_*download_response functions.

pawanjay176 · 2026-04-18T01:26:17Z

+        peer: PeerId,
+    },
+    /// Block processing complete
+    Done {


Complete instead of Done maybe to be consistent with naming in other parts of the codebase?

pawanjay176 · 2026-04-18T01:32:57Z

+        if let BlockRequest::Downloading { state, .. } = self {
+            state.insert_verified_response(result)
+        } else {
+            false


Should we log something here? Might be hard to debug in a later refactor if we accidentally modify the state and the invalid state error gets swallowed

pawanjay176 · 2026-04-18T01:35:19Z

+        peer_group: PeerGroup,
+    },
+    /// Data sent for processing, awaiting result
+    Processing {


Note to self: re review why we need DataDownloadKind here?

pawanjay176 · 2026-04-18T02:55:08Z

+            match &mut self.data_request {
+                DataRequest::WaitingForBlock => {
+                    if let Some(block) = self.block_request.peek_block() {
+                        let block = block.clone();


Unnecessary clone here?

pawanjay176 · 2026-04-18T03:03:07Z

+                        Duration::ZERO,
+                    )
+                    .map_err(LookupRequestError::SendFailedProcessor)?;
+                    self.block_request = BlockRequest::Processing { block, peer };


can't we break after this too? processing needs async trigger

pawanjay176 · 2026-04-18T03:03:25Z

+                    }
+                    let kind = data.kind();
+                    let peer_group = peer_group.clone();
+                    self.data_request = DataRequest::Processing { kind, peer_group };


can break after this too?

pawanjay176 · 2026-04-18T03:03:40Z

+            match &mut self.payload_request {
+                PayloadRequest::WaitingForBlock => {
+                    if let Some(block) = self.block_request.peek_block() {
+                        let block = block.clone();


unnecessary clone again?

pawanjay176 · 2026-04-18T03:04:55Z

+                                state.on_download_start(req_id)?;
+                            }
+                            Ok(LookupRequestResult::NoRequestNeeded(_reason)) => {
+                                // Payload RPC not wired yet — skip download, mark as done


We have it now right? Can add it in maybe? I'm also fine with deferring that to another PR though

dapplion · 2026-04-21T22:38:54Z

Rebased onto sigp/unstable and squashed to a single commit. Pushed c23b1af56.

Addressed @pawanjay176's comments from 2026-04-18 (17/18):

AwaitingParent enum → struct with Option<ExecutionBlockHash> parent_hash + pre_gloas()/post_gloas() constructors (mod.rs:489)
Done → Complete across BlockRequest / DataRequest / PayloadRequest; is_done() → is_complete() (sbl.rs:118)
on_*_download_response handlers now debug! log the error before collapsing to Err(()) (mod.rs:550)
insert_verified_response logs before dropping on bad state (sbl.rs:184)
Docstring added to DataDownloadKind explaining why it caches the request kind (sbl.rs:203)
Docstring on the Arc<RwLock<..>> peer sets explaining sharing with long-running ActiveCustodyRequest (sbl.rs:412)
Docstring on is_full_payload for intended usage; conservative false + log when block not downloaded (sbl.rs:470, :475)
Docstring on continue_requests loop explaining per-stream semantics (sbl.rs:567)
Pass block_root directly; drop the local br (sbl.rs:572)
if false dead branch → if state.is_completed() (sbl.rs:574)
"Single peer" comment added to block download peer handling (sbl.rs:580)
let Some(_) = ... else {} → .is_some() + bool flag (sbl.rs:600)
break after transitioning to Processing (sbl.rs:673, :733) — "processing needs async trigger"
.clone() of Arc<Block> eliminated by extracting (slot, expected_blobs) before the &mut self helper call (sbl.rs:684, :744); helpers renamed create_data_request/create_payload_request

Deferred (per your "fine with deferring that to another PR" note):

Wiring payload-envelope processing (sbl.rs:759). Adds a send_payload_for_processing in SyncNetworkContext plus a beacon_processor Work variant; keeping this PR focused on the download path.

Test status (FORK_NAME=gloas): 59/59 lookup tests pass.

Will open the upstream PR against sigp:unstable next.

Rewrites the single block lookup state machine for Gloas, where block, data (blobs/columns), and execution payload envelope are independent components that can arrive and import out of order. - Three additive-only sub-state-machines for block / data / payload streams. Peer sets start empty for data/payload and grow as children arrive — the parent lookup's completion requirement can widen over time without mutating any state machine. - `AwaitingParent` becomes a struct carrying the child's `parent_block_hash` so the parent can be classified empty/full from the child's bid reference. - Wires `PayloadEnvelopesByRoot` RPC end-to-end through `SyncNetworkContext`: request sending, response routing (`SingleLookupReqId::SinglePayloadEnvelope`), and integration into `PayloadRequest`. Envelope *processing* is still a TODO; only the download path is wired. - Test rig: serves envelopes from a `network_envelopes_by_root` cache populated from the external harness; bumps test validator count to 8 so `proposer_lookahead` can populate at the Fulu → Gloas upgrade. - Enables gloas in `TEST_NETWORK_FORKS`. - Fixes: genesis parent check, infinite retry loop on repeated download failure, no-op in `on_completed_request`, and peer sets not being cleared on disconnect.

- sigp#9107 Remove all instances of `test_logger` in the code Co-Authored-By: Tan Chee Keong <tanck@sigmaprime.io>

We are not submitting ptc votes that we produce to our lcoal ptc op pool. So when we are the block producer we don't include our own ptc votes! Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>

Get the Gloas HTTP API tests passing, partly through fixes and partly through disabling tests that don't fit the Gloas paradigm. Co-Authored-By: Michael Sproul <michael@sigmaprime.io> Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu> Co-Authored-By: dapplion <35266934+dapplion@users.noreply.github.com> Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>

We yolo'd to alpha 7. We're just changing the proposer preference to include dependent root, instead of checkpoint root. This way we can actually construct it within the VC without needing a view of fork choice. Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>

Update the ci workflow to use warpbuild snapshot image and test suit uses `Swatinew/rust-cache` to utilize warpbuild cache Co-Authored-By: lemon <snyxmk@gmail.com>

- Avoid sending 0x00 block hashes for the safe and finalized block hashes post-Gloas. - Add code to check this inside the mock EL, which will be reached in all Gloas beacon chain tests Co-Authored-By: Michael Sproul <michael@sigmaprime.io>

Fixes a flaky CI failure in `data_column_reconstruction_at_deadline` where 2 `column_reconstruction` events are emitted instead of the expected 1. - Change `queued_column_reconstructions` from `HashMap<Hash256, DelayKey>` to `HashMap<Hash256, Option<DelayKey>>`, where `None` indicates reconstruction was already dispatched. - On dispatch (`ReadyColumnReconstruction`), set the entry to `None` instead of removing it. This prevents a subsequent gossip column from inserting a fresh reconstruction request into the now-vacant slot. - Prune stale `None` entries on each dispatch to keep the map bounded. Co-Authored-By: Josh King <josh@sigmaprime.io>

…gp#9257) Two audit failures for `hickory-proto` which is used upstream in `libp2p-dns` and `libp2p-mdns` - https://rustsec.org/advisories/RUSTSEC-2026-0118.html - https://rustsec.org/advisories/RUSTSEC-2026-0119.html Tracking Issue: sigp#9258 Since RUSTSEC-2026-0118 does not even have any non-patched versions available and RUSTSEC-2026-0119 requires a major version bump I think we would need to wait on a release from libp2p in both cases. So for now, add an ignore for each so we can at least unblock CI Co-Authored-By: Mac L <mjladson@pm.me>

Allow for the vc to submit its proposer preferences to the network Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu> Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>

) Addresses issue sigp#9220 The `payload_attestation_data` endpoint returns 400 when no block has been received for the requested slot. This causes the VC to log at CRIT level for what is expected behaviour per spec: validators should simply not submit a payload attestation when no block has been seen. - Return 404 (Not Found) instead of 400 from `payload_attestation_data` when no block exists for the slot. This is consistent with other beacon api endpoints. - Downgrade the VC log from `crit` to `debug` when a 503 is received, since this is an expected no-op per spec. - Add `BlockNotFound` rejection type to `warp_utils`. - Add a test asserting the 404 response for an empty slot. Co-Authored-By: Josh King <josh@sigmaprime.io> Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu> Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>

N/A libssl download seems to be failing on [CI](https://github.com/sigp/lighthouse/actions/runs/25346412432/job/74316275231?pr=9126). This was originally added to unblock CI in sigp#6777, but we may not need this anymore. Co-Authored-By: Pawan Dhananjay <pawandhananjay@gmail.com>

We have a legacy `TestRandom` trait which generates random types for testing and fuzzing. This function overlaps with `arbitrary` which is used very commonly in the ecosystem. Remove `TestRandom` and generate random type instances using `Arbitrary`. Co-Authored-By: Mac L <mjladson@pm.me> Co-Authored-By: Michael Sproul <michael@sigmaprime.io>

Co-Authored-By: dapplion <35266934+dapplion@users.noreply.github.com>

dapplion · 2026-05-12T03:46:30Z

Closing for sigp#9155

dapplion force-pushed the unstable branch from eb54291 to efe43f7 Compare March 9, 2026 05:45

ethDreamer reviewed Mar 9, 2026

View reviewed changes

Comment thread .ai/plans/gloas-lookup-levels.md Outdated

dapplion force-pushed the gloas-lookup-sync-fixes branch from f551bd3 to bae92bd Compare March 10, 2026 02:23

dapplion requested a review from michaelsproul as a code owner March 10, 2026 02:23

dapplion force-pushed the gloas-lookup-sync-fixes branch 6 times, most recently from a6a37aa to f28fed4 Compare March 16, 2026 17:09

dapplion force-pushed the gloas-lookup-sync-fixes branch from f28fed4 to 8f4a5f0 Compare April 2, 2026 04:34

dapplion mentioned this pull request Apr 7, 2026

Gloas lookup sync fixes (rebased on unstable) #73

Closed

3 tasks

dapplion force-pushed the gloas-lookup-sync-fixes branch from 8f4a5f0 to d7af091 Compare April 7, 2026 04:28

pawanjay176 reviewed Apr 18, 2026

View reviewed changes

dapplion force-pushed the gloas-lookup-sync-fixes branch from 5533da0 to c23b1af Compare April 21, 2026 22:38

dapplion mentioned this pull request Apr 21, 2026

Gloas lookup sync sigp/lighthouse#9155

Open

dapplion force-pushed the gloas-lookup-sync-fixes branch 3 times, most recently from eb40ff0 to 3ff8491 Compare April 22, 2026 06:56

dapplion force-pushed the gloas-lookup-sync-fixes branch from 3ff8491 to ebe9fe2 Compare April 22, 2026 07:32

chong-he and others added 8 commits April 30, 2026 04:57

Remove test_logger feature (sigp#9125)

8d77b1c

- sigp#9107 Remove all instances of `test_logger` in the code Co-Authored-By: Tan Chee Keong <tanck@sigmaprime.io>

Submit ptc votes that we produce to the ptc op pool (sigp#9231)

728356a

We are not submitting ptc votes that we produce to our lcoal ptc op pool. So when we are the block producer we don't include our own ptc votes! Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>

Update CI: warp runnner to use snapshot and use warm (sigp#9217)

5384ab8

Update the ci workflow to use warpbuild snapshot image and test suit uses `Swatinew/rust-cache` to utilize warpbuild cache Co-Authored-By: lemon <snyxmk@gmail.com>

Avoid 0x00 block hashes in fcU (sigp#9233)

8b8124d

- Avoid sending 0x00 block hashes for the safe and finalized block hashes post-Gloas. - Add code to check this inside the mock EL, which will be reached in all Gloas beacon chain tests Co-Authored-By: Michael Sproul <michael@sigmaprime.io>

eserilev and others added 7 commits May 4, 2026 11:33

Implement gloas proposer preference vc duty (sigp#9208)

9cf155a

Allow for the vc to submit its proposer preferences to the network Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu> Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>

Generalise reconstruct_historic_states for ranged replay (sigp#9222)

31e5f30

Co-Authored-By: dapplion <35266934+dapplion@users.noreply.github.com>

Implement beacon_blocks_by_head (sigp#9237)

7148bfc

Co-Authored-By: dapplion <35266934+dapplion@users.noreply.github.com>

Merge branch 'unstable' into gloas-lookup-sync-fixes

e0effdb

dapplion closed this May 12, 2026

Conversation

dapplion commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Test plan

Uh oh!

Uh oh!

pawanjay176 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dapplion commented Apr 21, 2026

Uh oh!

dapplion commented May 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

dapplion commented Mar 9, 2026 •

edited

Loading