poc gloas builder settings - #17205
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates Prysm’s validator/keymanager and builder-preference plumbing to align with recent upstream spec work (keymanager-APIs #87, beacon-APIs #625, builder-specs 5078eab), adding a unified validator config document endpoint and shifting builder preferences to a JIT (inline) model for Gloas block production.
Changes:
- Add
GET/POST /eth/v1/validator/configkeymanager endpoints to manage per-validator proposer + builder preferences as one document, including builder entry lists and per-field inheritance fromdefault_config. - Update proposer/builder configuration handling to be presence-tracked (notably
enabledandmax_execution_payment), normalize legacy v1 presence semantics, and ensure schema version precedence during merges. - Replace builder “request auth” semantics to sign opaque
auth_data(not URL), and send builder preferences inline on block requests (including REST v4 POST flow).
Reviewed changes
Copilot reviewed 56 out of 58 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| validator/rpc/structs.go | Adds JSON structs for validator config keymanager API responses/requests. |
| validator/rpc/server.go | Adds proposer settings mutex and wires new /eth/v1/validator/config routes. |
| validator/rpc/handlers_validator_config.go | Implements GET/POST validator config handlers with per-key replacement semantics. |
| validator/rpc/handlers_validator_config_test.go | Adds tests covering validator config endpoint semantics and edge cases. |
| validator/rpc/handlers_keymanager.go | Serializes proposer-settings writers with a shared lock. |
| validator/rpc/handlers_keymanager_test.go | Updates tests for presence-tracked builder enabled fields. |
| validator/rpc/BUILD.bazel | Registers new validator config handler + tests and adds proto dep. |
| validator/db/kv/proposer_settings_test.go | Updates DB tests for presence-tracked builder fields. |
| validator/db/kv/BUILD.bazel | Adds protobuf dep for updated tests. |
| validator/db/convert_test.go | Updates conversion tests for presence-tracked builder fields. |
| validator/db/BUILD.bazel | Adds protobuf dep for updated tests. |
| validator/client/validator.go | Implements field-level builder config inheritance and JIT builder prefs/auth warming. |
| validator/client/validator_test.go | Updates/extends tests for new builder config semantics and registration behavior. |
| validator/client/runner_test.go | Updates runner tests for presence-tracked builder enabled. |
| validator/client/request_auth.go | Changes request-auth cache keying/signing to use opaque auth_data, and adds lookup helper. |
| validator/client/request_auth_test.go | Adds tests for auth rotation safety + per-entry inheritance resolution. |
| validator/client/propose.go | Sends inline builder preferences and (pre-Gloas) builder boost factor in block requests. |
| validator/client/BUILD.bazel | Adds deps for new validator types and wrapperspb; registers request_auth tests. |
| validator/client/beacon-api/get_beacon_block.go | Supports POSTing inline builder preferences to /eth/v4/validator/blocks/{slot}. |
| validator/client/beacon-api/get_beacon_block_test.go | Updates beacon-api client tests for the new beaconBlock signature. |
| validator/client/beacon-api/builder_preferences.go | Adds JSON marshaling for inline builder preferences POST body. |
| validator/client/beacon-api/BUILD.bazel | Registers new builder_preferences source and wrapperspb dep. |
| validator/client/beacon-api/beacon_api_validator_client.go | Wires BlockRequest fields through to REST beaconBlock call. |
| proto/prysm/v1alpha1/validator.proto | Replaces builder_request_auths with builder_preferences on BlockRequest. |
| proto/prysm/v1alpha1/validator-client/keymanager.proto | Makes builder fields presence-tracked and adds BuilderEntry + per-entry overrides. |
| proto/prysm/v1alpha1/gloas_builder_api.proto | Makes SubmitBuilderPreferences URL-explicit and adds BuilderPreferenceV1 message. |
| proto/prysm/v1alpha1/gloas_builder_api.pb.go | Regenerated Go bindings for updated gloas builder API protos. |
| config/proposer/settings.go | Adds presence normalization, field-level builder inheritance, and new builder fields/entries. |
| config/proposer/settings_test.go | Updates tests for pointer-presence semantics and adds legacy normalization coverage. |
| config/proposer/loader/loader.go | Ensures schema-version precedence and promotes v1 payloads when merging into v2. |
| config/proposer/loader/loader_test.go | Updates loader tests for v2 promotion semantics and per-key overlay merging. |
| config/proposer/loader/BUILD.bazel | Adds protobuf dep for updated tests. |
| config/proposer/effective_builder_config_test.go | Adds dedicated test vectors for EffectiveBuilderConfig behavior. |
| config/proposer/BUILD.bazel | Registers new effective builder config test and adds protobuf dep. |
| cmd/prysmctl/validator/proposer_settings.go | Updates CLI builder config construction for presence-tracked enabled field. |
| changelog/james-prysm_validator-config-endpoint.md | Adds changelog entry describing new endpoints and builder pref behavior changes. |
| beacon-chain/rpc/prysm/v1alpha1/validator/server.go | Removes now-unneeded maxExecutionPayments cache field. |
| beacon-chain/rpc/prysm/v1alpha1/validator/proposer.go | Threads builder preferences through block build path instead of request-auth list. |
| beacon-chain/rpc/prysm/v1alpha1/validator/proposer_gloas.go | Extracts inline prefs/auths for builder bidding and bid selection. |
| beacon-chain/rpc/prysm/v1alpha1/validator/proposer_gloas_prefs_test.go | Adds tests for builderAuthsAndPrefs extraction rules. |
| beacon-chain/rpc/prysm/v1alpha1/validator/proposer_bid.go | Adds boost/min-bid preferences and changes builder bid querying to use authsByURL. |
| beacon-chain/rpc/prysm/v1alpha1/validator/proposer_bid_test.go | Updates proposer bid tests for new prefs struct. |
| beacon-chain/rpc/prysm/v1alpha1/validator/proposer_bid_prefs_test.go | Adds tests covering boost/min-bid behavior and overflow-safe effective value logic. |
| beacon-chain/rpc/prysm/v1alpha1/validator/proposer_bid_builder_test.go | Updates builder bid tests for auth map + prefs plumbing. |
| beacon-chain/rpc/prysm/v1alpha1/validator/builder_preferences.go | Makes AOT SubmitBuilderPreferences URL-explicit and removes maxPayment side effects. |
| beacon-chain/rpc/prysm/v1alpha1/validator/builder_preferences_test.go | Updates tests to match URL-explicit AOT forwarding semantics. |
| beacon-chain/rpc/prysm/v1alpha1/validator/BUILD.bazel | Registers new proposer bid prefs test file. |
| beacon-chain/rpc/eth/validator/handlers_block_gloas.go | Accepts POST body for inline builder preferences for v4 produce-block. |
| beacon-chain/rpc/eth/validator/builder_preferences.go | Adds REST-side JSON parsing for inline builder preferences POST body. |
| beacon-chain/rpc/eth/validator/BUILD.bazel | Registers new REST builder preferences parser source. |
| beacon-chain/rpc/endpoints.go | Allows POST on /eth/v4/validator/blocks/{slot}. |
| beacon-chain/rpc/endpoints_test.go | Updates endpoint method expectations to include POST for v4 blocks. |
| beacon-chain/builder/testing/mock.go | Updates mock builder interface (auth map + URL-explicit SubmitBuilderPreferences). |
| beacon-chain/builder/service.go | Updates builder service interface + routing for opaque auth data and URL-explicit calls. |
| beacon-chain/builder/service_gloas_test.go | Updates multiplex builder service tests for authsByURL and URL-explicit preference submission. |
| api/client/builder/client_gloas.go | Updates comments around SubmitBuilderPreferences semantics (still unwired). |
Files not reviewed (1)
- proto/prysm/v1alpha1/gloas_builder_api.pb.go: Generated file
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| settings := s.validatorService.ProposerSettings() | ||
| resp := &GetValidatorConfigResponse{Data: &ValidatorConfigData{Configs: map[string]*ValidatorConfig{}}} |
| func (p *builderPreferenceJson) toConsensus(i int) (*eth.BuilderPreferenceV1, error) { | ||
| if p == nil || p.SignedRequestAuth == nil || p.SignedRequestAuth.Message == nil { | ||
| return nil, errors.Errorf("builder_preferences[%d] missing signed_request_auth", i) | ||
| } |
| data, err := hexutil.Decode(p.SignedRequestAuth.Message.Data) | ||
| if err != nil { | ||
| return nil, errors.Errorf("builder_preferences[%d].signed_request_auth.message.data is not valid hex", i) | ||
| } |
| if !baselineSet { | ||
| bp.maxPayment = uint64(p.Request.Preferences.GetMaxExecutionPayment()) | ||
| if p.MinBid != nil { | ||
| bp.minBid = uint64(p.GetMinBid()) | ||
| } |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 63 out of 67 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
beacon-chain/rpc/prysm/v1alpha1/validator/proposer_gloas.go:130
builderAuthsAndPrefscollapses all inlineBuilderPreferenceV1entries into a singlebidPreferencesderived from the first valid entry. That means per-builder overrides (e.g. differentmax_execution_payment,min_bid, orbuilder_boost_factorper URL) are ignored during bid validation/selection, even though the request format carries them per entry.
// builderAuthsAndPrefs extracts the signed request auths and the effective
// proposer-local bid preferences from the inline builder preferences carried on
// the block request. The effective preferences use the first valid entry as the
// proposal baseline. The beacon node enforces the caps locally during bid
// selection; communicating preferences to the builder is left to the bid request.
| for _, p := range prefs { | ||
| if p == nil || p.Request == nil || p.Request.Auth == nil || p.Request.Auth.Message == nil { | ||
| continue | ||
| } | ||
| entry := &builderEntryReq{ | ||
| Url: p.Url, | ||
| Auth: signedAuthFromConsensus(p.Request.Auth), | ||
| MaxExecutionPayment: strconv.FormatUint(uint64(p.Request.Preferences.GetMaxExecutionPayment()), 10), | ||
| } |
| func (c *beaconApiValidatorClient) submitBuilderPreferences(ctx context.Context, in *ethpb.SubmitBuilderPreferencesRequest) error { | ||
| if in == nil || in.Request == nil || in.Request.Auth == nil || in.Request.Auth.Message == nil { | ||
| return errors.New("builder preferences request is empty") | ||
| } | ||
| body, err := json.Marshal([]*builderPreferenceEntryReq{{ | ||
| Url: in.Url, | ||
| Auth: signedAuthFromConsensus(in.Request.Auth), | ||
| MaxExecutionPayment: strconv.FormatUint(uint64(in.Request.Preferences.GetMaxExecutionPayment()), 10), | ||
| }}) | ||
| if err != nil { | ||
| return errors.Wrap(err, "could not marshal builder preference entry") | ||
| } | ||
| return c.handler.Post(ctx, "/eth/v1/validator/builder_preferences/"+hexutil.Encode(in.ValidatorPubkey), nil, bytes.NewBuffer(body), nil) | ||
| } |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 63 out of 67 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (2)
validator/client/beacon-api/builder_preferences.go:64
marshalBuilderPreferencesassumesp.Request.Preferencesis non-nil and callsp.Request.Preferences.GetMaxExecutionPayment(). If a caller constructs aBuilderPreferenceV1withRequest.Preferences == nil, this will panic during block production (v4 POST body). Consider defaultingmax_execution_paymentto 0 when preferences are absent (consistent with other code treating unset as 0/trustless-only).
validator/client/beacon-api/builder_preferences.go:86submitBuilderPreferencesalso assumesin.Request.Preferencesis non-nil and callsin.Request.Preferences.GetMaxExecutionPayment(), which can panic on malformed/partial inputs. Returning a validation error (or treating it as 0) would prevent a crash in the beacon-api validator client.
| // Empty means self-build or a P2P bid; the builder learns those via gossip. | ||
| if block.Version() >= version.Gloas && req.BuilderUrl != "" { | ||
| go vs.submitBlockToBuilder(block, req.BuilderUrl) | ||
| } |
| for _, p := range prefs { | ||
| if p == nil || p.Request == nil || p.Request.Auth == nil || p.Url == "" { | ||
| continue | ||
| } | ||
| if _, ok := authsByURL[p.Url]; !ok { | ||
| authsByURL[p.Url] = p.Request.Auth | ||
| } | ||
| if !baselineSet { | ||
| bp.maxPayment = uint64(p.Request.Preferences.GetMaxExecutionPayment()) | ||
| if p.MinBid != nil { | ||
| bp.minBid = uint64(p.GetMinBid()) | ||
| } | ||
| if p.BuilderBoostFactor != nil { | ||
| bp.boostFactor = p.GetBuilderBoostFactor() | ||
| } | ||
| baselineSet = true | ||
| } | ||
| } |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 63 out of 67 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
beacon-chain/rpc/prysm/v1alpha1/validator/proposer_gloas.go:131
- builderAuthsAndPrefs collapses per-builder preferences (max_execution_payment/min_bid/builder_boost_factor) into a single “baseline” taken from the first valid entry. That drops per-entry overrides emitted by the validator client (BuilderPreferenceV1 carries per-builder min_bid/boost/max payment) and can cause the beacon node to apply the wrong caps/boost when selecting between bids from different builders.
// builderAuthsAndPrefs extracts the signed request auths and the effective
// proposer-local bid preferences from the inline builder preferences carried on
// the block request. The effective preferences use the first valid entry as the
// proposal baseline. The beacon node enforces the caps locally during bid
// selection; communicating preferences to the builder is left to the bid request.
func builderAuthsAndPrefs(prefs []*ethpb.BuilderPreferenceV1) (map[string]*ethpb.SignedRequestAuthV1, bidPreferences) {
| if len(eff.Builders) == 0 { | ||
| eff.Builders = def.Builders | ||
| } |
| if len(eff.Relays) == 0 { | ||
| eff.Relays = def.Relays | ||
| } |
| func (s *Server) proposeBlock(ctx context.Context, w http.ResponseWriter, r *http.Request, blk *eth.GenericSignedBeaconBlock) { | ||
| blk.BuilderUrl = r.Header.Get(api.EthBuilderUrlHeader) | ||
| _, err := s.V1Alpha1ValidatorServer.ProposeBeaconBlock(ctx, blk) |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 63 out of 65 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (2)
config/proposer/settings.go:165
- EffectiveBuilderConfig currently inherits default Builders/Relays when the effective slice length is 0. This makes it impossible to explicitly clear an inherited list (e.g., JSON
"builders":[]decodes to a non-nil empty slice, but will be treated as “inherit default”). To honor “lists replace rather than merge”, inheritance should depend on nil vs non-nil, not len==0.
beacon-chain/rpc/prysm/v1alpha1/validator/proposer_gloas.go:131 - builderAuthsAndPrefs collapses all per-builder preferences (max_execution_payment, min_bid, builder_boost_factor) down to the first valid entry and uses that as a global baseline. However, the client and config types support per-builder overrides (BuilderEntry.{max_execution_payment,min_bid,builder_boost_factor}), so ordering can change behavior and later entries’ overrides are silently ignored. This can lead to the beacon node enforcing the wrong caps/floors/boost when selecting among builder bids.
// builderAuthsAndPrefs extracts the signed request auths and the effective
// proposer-local bid preferences from the inline builder preferences carried on
// the block request. The effective preferences use the first valid entry as the
// proposal baseline. The beacon node enforces the caps locally during bid
// selection; communicating preferences to the builder is left to the bid request.
func builderAuthsAndPrefs(prefs []*ethpb.BuilderPreferenceV1) (map[string]*ethpb.SignedRequestAuthV1, bidPreferences) {
| s.proposerSettingsLock.Lock() | ||
| defer s.proposerSettingsLock.Unlock() | ||
| settings := s.validatorService.ProposerSettings() | ||
| if err := settings.SetGasLimit(bytesutil.ToBytes48(pubkey), validator.Uint64(gasLimit)); err != nil { |
| s.proposerSettingsLock.Lock() | ||
| defer s.proposerSettingsLock.Unlock() | ||
| settings := s.validatorService.ProposerSettings() | ||
| if !settings.ResetGasLimit(bytesutil.ToBytes48(pubkey)) { |
| "bytes" | ||
| "context" | ||
| "encoding/json" | ||
| "github.com/OffchainLabs/prysm/v7/api" | ||
| "net/http" |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 63 out of 65 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (2)
validator/rpc/handlers_validator_config.go:43
GetValidatorConfigreads and iterates oversettings.ProposeConfigwithout synchronizing with writers. Other handlers mutate proposer settings in-place unders.proposerSettingsLock, so this endpoint can race (and potentially panic) if a write occurs concurrently.
Consider cloning proposer settings under the lock (then unlocking) before building the response, so iteration is race-free without holding the mutex during JSON encoding.
beacon-chain/rpc/prysm/v1alpha1/validator/proposer_gloas.go:153
builderAuthsAndPrefscollapses all per-builder preferences into a single baseline taken from the first valid entry. However the validator client now emits per-entry overrides (max_execution_payment, min_bid, builder_boost_factor), so enforcing a single baseline can under-/over-enforce caps and apply the wrong boost/min-bid when comparing bids from different builders.
To preserve the new per-builder semantics, the beacon node likely needs to keep preferences keyed by URL (or builder index) and apply the correct cap/boost/min-bid for each bid during validation and best-bid selection.
// builderAuthsAndPrefs extracts the signed request auths and the effective
// proposer-local bid preferences from the inline builder preferences carried on
// the block request. The effective preferences use the first valid entry as the
// proposal baseline. The beacon node enforces the caps locally during bid
// selection; communicating preferences to the builder is left to the bid request.
func builderAuthsAndPrefs(prefs []*ethpb.BuilderPreferenceV1) (map[string]*ethpb.SignedRequestAuthV1, bidPreferences) {
authsByURL := make(map[string]*ethpb.SignedRequestAuthV1, len(prefs))
bp := bidPreferences{boostFactor: neutralBuilderBoostFactor}
baselineSet := false
for _, p := range prefs {
if p == nil || p.Request == nil || p.Request.Auth == nil || p.Url == "" {
continue
}
if _, ok := authsByURL[p.Url]; !ok {
authsByURL[p.Url] = p.Request.Auth
}
if !baselineSet {
bp.maxPayment = uint64(p.Request.Preferences.GetMaxExecutionPayment())
if p.MinBid != nil {
bp.minBid = uint64(p.GetMinBid())
}
if p.BuilderBoostFactor != nil {
bp.boostFactor = p.GetBuilderBoostFactor()
}
baselineSet = true
}
}
return authsByURL, bp
| func (v *validator) upgradeProposerSettingsToV2(ctx context.Context) { | ||
| ps := v.ProposerSettings() | ||
| ps := v.ProposerSettings().Clone() | ||
| if !ps.UpgradeToV2() { | ||
| return | ||
| } |
| func builderEntryFromJSON(in *BuilderEntryJson, i int) (*proposer.BuilderEntry, error) { | ||
| if in.Url == "" { | ||
| return nil, errors.Errorf("builder.builders[%d].url is required", i) | ||
| } |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 62 out of 64 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (3)
validator/client/validator.go:1145
upgradeProposerSettingsToV2can panic whenv.proposerSettingsis nil (fresh start / no settings loaded), because it callsv.ProposerSettings().Clone()unconditionally. This is reachable on the post-Gloas path (PushProposerSettings), so a nil settings pointer would crash the validator client.
validator/rpc/handlers_keymanager.go:774s.validatorService.ProposerSettings()can return nil (it returns nil when the underlying validator has no settings).settings.SetGasLimit(...)will panic in that case; this endpoint should return a structured HTTP error instead of crashing.
validator/rpc/handlers_keymanager.go:803s.validatorService.ProposerSettings()can return nil. Callingsettings.ResetGasLimit(...)without checking will panic; this should return a 404/400 instead of crashing.
| if len(builderPreferences) > 0 { | ||
| // JIT (beacon-APIs #625): POST the per-builder preferences inline. | ||
| body, mErr := marshalBuilderPreferences(builderPreferences) | ||
| if mErr != nil { | ||
| return nil, errors.Wrap(mErr, "could not marshal builder preferences") | ||
| } | ||
| headers := map[string]string{"Content-Type": api.JsonMediaType} | ||
| data, header, err = c.handler.PostSSZ(ctx, queryUrl, headers, bytes.NewBuffer(body)) | ||
| } else { | ||
| data, header, err = c.handler.GetSSZ(ctx, queryUrl) | ||
| } |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r public key, removing relay field
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 64 out of 66 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
validator/client/validator.go:1145
- upgradeProposerSettingsToV2 dereferences v.ProposerSettings() without a nil check; if proposer settings haven’t been loaded yet this will panic on .Clone(). Guard against nil before cloning/upgrading.
| e := &BuilderEntry{ | ||
| URL: from.Url, | ||
| Proxy: from.Proxy, | ||
| MinBid: from.MinBid, | ||
| MaxExecutionPayment: from.MaxExecutionPayment, | ||
| BuilderBoostFactor: from.BuilderBoostFactor, | ||
| } |
…ty builder are identical and unset across validator backends
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 67 out of 69 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (4)
validator/client/validator.go:1144
- upgradeProposerSettingsToV2 now calls v.ProposerSettings().Clone() without a nil check; if proposer settings are still unset, this will panic. The previous version safely handled nil by returning early.
validator/client/beacon-api/builder_preferences.go:65 - marshalBuilderPreferences dereferences p.Request.Preferences directly (p.Request.Preferences.GetMaxExecutionPayment()), which will panic if Preferences is nil. It should either skip entries missing Preferences or use the safe getters.
validator/client/beacon-api/builder_preferences.go:90 - submitBuilderPreferences also dereferences in.Request.Preferences directly; if Preferences is nil this will panic. The input validation should include Preferences (and ideally also ensure Url is present since it is required by the REST endpoint).
config/proposer/settings.go:190 - EffectiveBuilderConfig claims the builders list "replaces rather than merges", but the current logic uses len(eff.Builders)==0 to inherit the default list. That makes an explicitly configured empty list (e.g. JSON/YAML "builders": []) inherit the default instead of replacing it.
|
significant work was also done on #17124, i'll be breaking out the keymanager api stuff and validator components from this one |
| if p.Request == nil { | ||
| p.Request = new(BuilderPreferencesRequestV1) | ||
| } | ||
| offset += p.Request.SizeSSZ() |
What type of PR is this?
What does this PR do? Why is it needed?
ethereum/keymanager-APIs#87
beacon apis ethereum/beacon-APIs#625 and then there’s ethereum/builder-specs@5078eab
Which issue(s) does this PR fix?
Fixes #
Other notes for review
Acknowledgements