Skip to content

Core FCR Implementation#2

Open
harsh-ps-2003 wants to merge 92 commits intounstablefrom
coreFCR
Open

Core FCR Implementation#2
harsh-ps-2003 wants to merge 92 commits intounstablefrom
coreFCR

Conversation

@harsh-ps-2003
Copy link
Owner

@harsh-ps-2003 harsh-ps-2003 commented Jul 30, 2025 •
edited
Loading

Core FCR Implementation:

  • Q-indicator calculation (is_one_confirmed)
  • Committee weight calculations with cross-epoch boundary handling and safety adjustments
  • Weight separation - properly separates attestation weight from proposer boost as required by FCR specification
  • Integration hooks - HOOK-A (after find_head), HOOK-B (after pruning), HOOK-C (slot transitions)
  • O(depth) ancestor scanning for efficient confirmation inheritance along the canonical chain, leveraging existing ProtoArray weight calculations
  • DAG-aligned pruning - FCR metadata pruned in sync with ProtoArray preventing metadata leaks
  • Confirmation of inheritance and advancement logic - find_latest_confirmed_descendant() and get_latest_confirmed()
  • Legacy API preservation - get_weight_legacy() maintains existing test compatibility
    With Tests
  • Calculates FFG support using LMD-GHOST votes - get_checkpoint_weight()
  • Checks if validator vote supports checkpoint - validator_vote_supports_checkpoint()
  • Pro-rata FFG weight calculation - get_ffg_weight_till_slot()
  • FFG justification prediction with β and slashing thresholds - will_current_epoch_checkpoint_be_justified()
  • FFG safety check for conflicting checkpoints - will_no_conflicting_checkpoint_be_justified()
  • StateProvider trait - Abstracted historical state access for FFG analysis. Decouples fork_choice from store crate for FFG analysis
  • HotColdDBStateProvider - Newtype wrapper implementing StateProvider for store integration

Slight Deviation from spec

The spec compliance is nearly complete.

  • S vs W data source - We use proto_array.get_weight(..., None, /*no boost*/) for S (attestation support), and the weighting checkpoint state only for W (maximum possible support). Because in Lighthouse, S is the authoritative node.weight recomputed on every get_head() via compute_deltas() under the justified view. This avoids a second, ad-hoc vote-eval pipeline and keeps S consistent with fork-choice. The _checkpoint_state parameter in proto_array.get_weight is intentionally ignored.
  • W computation fallback - We compute W using the weighting checkpoint state’s get_total_active_balance() when available; otherwise we fall back to fc_store.justified_balances().total_effective_balance. Python spec always uses a state; Lighthouse can sometimes lack a historical checkpoint state through the StateProvider. We fail-safe by falling back to the justified balances rather than block the algorithm.
  • Optional state provider instead of spec’s mandatory checkpoint_states - The spec assumes store.checkpoint_states[...] is available. We abstracted state access via a StateProvider and treat missing states conservatively (returning false for justification checks if state unavailable). This is to keep fork_choice crate independent of store and avoid a DB/schema change. It also lets us operate even when historical states are not cached.
  • Voting source epoch - now computed from latest messages (exact) with a fallback to prev_slot_unrealized_justified_checkpoint.epoch when the signal is missing. So deviation reduced to a fallback only.
  • The spec has no opinion about storage; Lighthouse avoids SSZ/db changes by keeping FCR state ephemeral and pruned alongside the DAG. We maintain confirmation metadata in a side-table (HashMap<Hash256, FcrMeta>) and prune it in lockstep with ProtoArray.

Functions that are NOT needed in Lighthouse:

  • get_forkchoice_store() - Lighthouse uses ForkChoice::new() with FCR parameters
  • get_block_slot() - Access via proto_array.get_block().slot
  • get_block_epoch() - Use slot.epoch(E::slots_per_epoch())
  • get_weight() - Use proto_array.get_weight() with FCR parameters
  • on_tick_per_slot_after_attestations_applied() - Integrated into ForkChoice::on_tick()

@harsh-ps-2003
core FCR implementation :
4b47f70 
* is_one_confirmed
* committee weight calculations with epoch
* separating attestation weight from proposer boost as required by FCR
* integration hooks
* ancestor scanning
* DAG-aligned pruning

and property based tests along with backward compatibility of tests
@harsh-ps-2003 harsh-ps-2003 self-assigned this Jul 30, 2025
@harsh-ps-2003
nits
40a5aa7
@harsh-ps-2003
Add is_block_confirmed method and enhance confirmation inheritance …
5a654f6 
…in FCR

Key changes include:
- Introduced `is_block_confirmed` method to check the confirmation status of a block without exposing internal metadata.
- Implemented `mark_descendants_confirmed` method to recursively mark all descendants of a confirmed block, ensuring consistency across the block DAG.
@harsh-ps-2003
Implement is_ancestor method for block relationship checks in FCR
7550e87 
Key changes include:
- Added `is_ancestor` method in both `FastConfirmation` and `ForkChoice` to determine if one block is an ancestor of another, enhancing the confirmation logic.
- Updated documentation to clarify the purpose and arguments of the `is_ancestor` method.
@harsh-ps-2003
committee weight adjustments and checkpoint support
bb2aaeb 
Key changes include:
- Added `COMMITTEE_WEIGHT_ESTIMATION_ADJUSTMENT_FACTOR` to apply a safety margin in committee weight estimates, ensuring FCR safety guarantees.
- Implemented `adjust_committee_weight_estimate_to_ensure_safety` method to encapsulate the logic for adjusting committee weight estimates.
- Introduced `get_checkpoint_weight` method to estimate FFG support for checkpoints using LMD-GHOST votes, ensuring confirmed blocks are not filtered out by FFG justification changes.
- Added comprehensive tests for both `adjust_committee_weight_estimate_to_ensure_safety` and `get_checkpoint_weight` functionalities, covering various scenarios including edge cases and precision handling.
@harsh-ps-2003
- Improved FCR confirmation logic by simplifying various checks, allo…
a9b9a98 
…wing confirmation to proceed based on LMD-GHOST support only.

- Added detailed documentation regarding architectural limitations and the current state of FFG integration, clarifying the need for future enhancements.
- Removed unused `ffg_support_lru` cache from `FcrStore` to streamline the implementation and reduce complexity.
- Introduced new tests for `get_ffg_weight_till_slot` and `will_current_epoch_checkpoint_be_justified` functionalities, ensuring comprehensive coverage of FCR confirmation logic.
- Enhanced existing tests for consistency and clarity, particularly in the context of Byzantine threshold adjustments.
@harsh-ps-2003
nits
b6e8aa9
@harsh-ps-2003
Refactor ancestor check in FastConfirmation
58b8ede 
- Simplified the ancestor verification logic by utilizing the existing `is_descendant` method with swapped arguments, enhancing code clarity and maintainability.
@harsh-ps-2003
nit
9820d3e
@harsh-ps-2003
fmt nit
7207662
@harsh-ps-2003
- Updated the Byzantine threshold calculation to include the current …
1bc3a39 
…epoch for improved FCR logic clarity.

- Added checks to prevent confirmation of blocks from future epochs, ensuring safety in the confirmation process.
- Revised comments to better reflect the purpose of `checkpoint_state`.
- Removed redundant variable declarations to streamline the code and improve readability.
@harsh-ps-2003 harsh-ps-2003 changed the title Core FCR Implementation Core FCR Implementation, till LMD GHOST Aug 4, 2025
@harsh-ps-2003 harsh-ps-2003 changed the title Core FCR Implementation, till LMD GHOST Core FCR Implementation Aug 5, 2025
@harsh-ps-2003
add FFG
576221c
@harsh-ps-2003
clippy nits
7485507
@harsh-ps-2003
fmt nit
25fe71e
@harsh-ps-2003
nits
5242f0b
@harsh-ps-2003
spec change
c8e2d18
@harsh-ps-2003
add some property tests
c58b73f
@harsh-ps-2003
Refactor get_voting_source_epoch to accept fc_store parameter for…
9e69d4a 
… improved balance handling and voting source calculation. Enhanced logic to derive voting source epoch based on effective balances and added fallback mechanism. Updated related function calls for consistency. Cleaned up comments and removed unused variables in tests for clarity.
@harsh-ps-2003
Add tests for FFG checkpoint validation and cross-epoch weight progre…
d9d5c22 
…ssion

- Introduced a test to ensure that a checkpoint is only supported if the vote's target checkpoint matches the expected target, verifying the correct handling of mismatched target roots.
- Added a test for minimal slot progression across epoch boundaries, confirming that the estimator behaves as expected when transitioning between epochs.
@harsh-ps-2003
Enhance get_checkpoint_state and related methods for improved state…
3c7aa59 
… retrieval

- Updated the `get_checkpoint_state` method to return `Arc<BeaconState<E>>` instead of `&BeaconState<E>`, improving memory management and ownership semantics.
- Enhanced logic to retrieve checkpoint states from both hot and cold storage, ensuring more efficient state access and fallback mechanisms.
- Refactored `get_total_active_balance_at_epoch` to utilize cold state roots directly, optimizing balance retrieval at epoch boundaries.
@harsh-ps-2003
nits
8434c84
@harsh-ps-2003
test nits
d6a43df
@harsh-ps-2003
nit
7134c64
@harsh-ps-2003
add some logging
af094d2
@harsh-ps-2003
Enhance fast confirmation logic to promote confirmed checkpoints at e…
25a801d 
…poch boundaries

- Added logic to promote the confirmed checkpoint to the unrealized justified checkpoint if it belongs to the previous epoch and is later than the current confirmed block, aligning with spec-defined safety measures.
- Improved the conditional checks for advancing the confirmed root by incorporating checks for unrealized justification epochs, ensuring robust handling of checkpoint conditions across epoch transitions.
@harsh-ps-2003
nit
ad8671f
@harsh-ps-2003
Enhance FastConfirmation with safe head caching and metrics. Introduc…
9d7f859 
…ed `safe_head_root` for O(1) access, improving performance and leveraging structural sharing for efficient updates. Added `SafeHeadMetrics` struct for monitoring, and updated methods to utilize the new safe head functionality. This refactor optimizes the confirmation logic while maintaining compliance with specifications and improving sync safety.
@harsh-ps-2003
Implement depth limiting in is_ancestor to prevent infinite recursion…
5e14389 
… during sync.
@harsh-ps-2003
stackoverflow issue still not gone
14145e6
@harsh-ps-2003
nit
fd6d1e3
@harsh-ps-2003
nit
45614da
@harsh-ps-2003
nit
4ff036c
@harsh-ps-2003
temporarily disable FCR during sync for testing.
3564c53
@harsh-ps-2003
enable metrics
bfbebc6
@harsh-ps-2003
monitoring setup
5f85eea
@harsh-ps-2003
Add FCR-specific Prometheus recording rules and update configuration
522bc15
@harsh-ps-2003
remove grafana
3c9b971
@harsh-ps-2003
nit
9ff4c41
@harsh-ps-2003
nit
a19bc46
@harsh-ps-2003
nit
aeb169c
@harsh-ps-2003
nit
a7c235f
@harsh-ps-2003
nit
6d4beda
@harsh-ps-2003
improve benchmarking
c69cd65
@harsh-ps-2003
nits
f48df62
@harsh-ps-2003
monitor reorgs
c92bebf
@harsh-ps-2003
- Introduced new metrics for tracking confirmation slot delay and hea…
7c75928 
…d-to-confirmed gap slots.

- Added a counter for confirmation rule restarts, labeled by reason (stale or reorg).
- Implemented tail-case confirmation metrics, categorized by epoch boundary and delay bucket.
- Updated the `mark_confirmed` function to utilize the new metrics for improved observability.
@harsh-ps-2003
Added a new metric FCR_IN_SYNC to track the synchronization state b…
fbce468 
…etween the head slot and the current slot. This metric is represented as a gauge, indicating 1 when the head slot equals the current slot, and 0 otherwise.
@harsh-ps-2003
logs
1bd18fd
@harsh-ps-2003
FCR logs for 1 day
4b458b2
@harsh-ps-2003
remove previous logs
a5ea6eb
@harsh-ps-2003
Enhance block creation tracking in FastConfirmation to prevent 0-slot…
04453d1 
… delay artifacts in metrics. Updated `block_creation_slot` to use the block's real slot when available, ensuring accurate metadata creation for confirmed blocks.
@harsh-ps-2003
Add metrics for StateProvider performance in FastConfirmation
efd8149 
also, fixed proposer delta
@harsh-ps-2003
refactoring
e70fabe
@harsh-ps-2003
remove sync safety
f9868f5
@harsh-ps-2003
spec updates + fmt nit
220f701
@harsh-ps-2003
nit
a97b549
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@harsh-ps-2003 harsh-ps-2003

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@harsh-ps-2003