feat(speculation): add path scorer extension

Add the scorer seam from the speculation RFC, as a vendor-agnostic extension interface under submitqueue/extension/speculation/scorer/.

The scorer computes each speculation path's predicted-success score from the current state: the per-batch scores of the path's base batches (entity.Batch.Score) and which of those dependencies have resolved (landed or build-passed), plus optionally other signals. It is a prediction over live state, so the controller re-runs it on every respeculate right after reconciling status, and persists the result; the scorer owns only the formula.

The controller hands it the batch's speculation tree directly — the subject it scores. Any richer signal an implementation needs (dependency batch scores, historical pass rates) is injected at its Factory, not put in the signature. It never writes: it returns the tree with Score recomputed and the controller persists, staying the single writer of tree state.

This is the per-path scorer, distinct from the existing per-batch score stage (extension/scorer) that sets entity.Batch.Score — the path scorer consumes those to score whole paths.

Follows the repo extension contract: Factory.For(Config) (Scorer, error) with Config carrying only QueueName. Includes README, gomock package, and a programmable fake. Interface only; concrete impls and controller wiring are deferred.

Author: behinddwalls

Makefile

@@ -364,7 +364,7 @@ local-stovepipe-stop: ## Stop the Stovepipe service
 
 mocks: ## Generate mock files using mockgen
 	@echo "Generating mocks..."
-	@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./platform/extension/counter/... ./platform/extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/extension/speculation/enumerator/... ./submitqueue/extension/speculation/dependencylimit/... ./platform/consumer/... ./submitqueue/core/changeset/... ./stovepipe/extension/storage/... ./stovepipe/extension/sourcecontrol/...
+	@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./platform/extension/counter/... ./platform/extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/extension/speculation/enumerator/... ./submitqueue/extension/speculation/dependencylimit/... ./submitqueue/extension/speculation/scorer/... ./platform/consumer/... ./submitqueue/core/changeset/... ./stovepipe/extension/storage/... ./stovepipe/extension/sourcecontrol/...
 	@echo "Mocks generated successfully!"
 
 proto: ## Generate protobuf files from .proto definitions

submitqueue/extension/speculation/scorer/BUILD.bazel

@@ -0,0 +1,9 @@
+load("@rules_go//go:def.bzl", "go_library")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["scorer.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/scorer",
+    visibility = ["//visibility:public"],
+    deps = ["//submitqueue/entity:go_default_library"],
+)

submitqueue/extension/speculation/scorer/README.md

@@ -0,0 +1,17 @@
+# Speculation Path Scorer
+
+Vendor-agnostic interface for scoring the paths in a batch's **speculation tree** — the predicted-success probability of each candidate bet, recomputed as the batch's world changes.
+
+See the [Speculation RFC](../../../../doc/rfc/submitqueue/speculation.md) for the end-to-end design and how scoring fits into the orchestrator pipeline.
+
+## Scorer
+
+A path's score is a **prediction**: *how likely is this bet to pay off, right now?* The scorer answers it from the current state — the per-batch success probabilities of a path's base batches (`entity.Batch.Score`, set by the score stage), which of those dependencies have already landed or had their build pass (resolved assumptions raise confidence), and optionally other signals such as how long the batch has waited or historical pass rates. The score is the common currency the [selector](../selector) and prioritizer both rank on, so keeping it current is what makes both act on the latest reality.
+
+Because it is a prediction over live state, the scorer is **re-run on every respeculate**, right after the controller reconciles path status — so when a dependency lands, its build passes, or a sibling path fails, the surviving paths' scores are recomputed before anything is selected or prioritized. The controller drives *when* to rescore (it is part of reconciliation) and persists the result; the scorer owns the *formula*.
+
+This is the per-**path** scorer, distinct from the per-**batch** [score stage](../../scorer), which sets `entity.Batch.Score`. The path scorer consumes those batch scores to score whole paths. The controller hands it the batch's **speculation tree** directly — the subject it scores — and any richer signal an implementation needs (the dependency batches' scores, historical pass rates) is injected at its factory, not passed in. It never writes: it returns the scored tree and the controller persists the scores, so the controller stays the single writer of tree state. Path structure and status pass through unchanged; only `Score` is recomputed.
+
+## Factory
+
+A per-queue factory returns the scorer for a queue, following the repo's extension contract. It is handed only the queue identity; scoring knobs and read access to any extra signals are injected at construction by the integrator in the wiring layer, which resolves per-queue settings through `queueconfig`. Scoring itself stays config-free.

submitqueue/extension/speculation/scorer/fake/BUILD.bazel

@@ -0,0 +1,23 @@
+load("@rules_go//go:def.bzl", "go_library", "go_test")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["fake.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/scorer/fake",
+    visibility = ["//visibility:public"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "//submitqueue/extension/speculation/scorer:go_default_library",
+    ],
+)
+
+go_test(
+    name = "go_default_test",
+    srcs = ["fake_test.go"],
+    embed = [":go_default_library"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "@com_github_stretchr_testify//assert:go_default_library",
+        "@com_github_stretchr_testify//require:go_default_library",
+    ],
+)

submitqueue/extension/speculation/scorer/fake/fake.go

@@ -0,0 +1,67 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package fake provides a programmable scorer.Scorer for tests and examples. By
+// default Score echoes the tree it is given; Returns overrides that with a
+// canned scored tree, and FailWith injects an error on every call. It is
+// intended for examples and tests only, never production.
+package fake
+
+import (
+	"context"
+
+	"github.com/uber/submitqueue/submitqueue/entity"
+	"github.com/uber/submitqueue/submitqueue/extension/speculation/scorer"
+)
+
+// Scorer is a programmable scorer.Scorer.
+type Scorer struct {
+	tree    entity.SpeculationTree
+	hasTree bool
+	err     error
+}
+
+// New returns a fake Scorer that echoes the tree it is given. Override the
+// returned tree with Returns.
+func New() *Scorer {
+	return &Scorer{}
+}
+
+// Returns makes every Score call return tree instead of echoing its input.
+func (s *Scorer) Returns(tree entity.SpeculationTree) *Scorer {
+	s.tree = tree
+	s.hasTree = true
+	return s
+}
+
+// FailWith makes every Score call return err.
+func (s *Scorer) FailWith(err error) *Scorer {
+	s.err = err
+	return s
+}
+
+// Score returns the canned tree if one was set with Returns, otherwise the tree
+// it was given, unchanged.
+func (s *Scorer) Score(_ context.Context, tree entity.SpeculationTree) (entity.SpeculationTree, error) {
+	if s.err != nil {
+		return entity.SpeculationTree{}, s.err
+	}
+	if s.hasTree {
+		return s.tree, nil
+	}
+	return tree, nil
+}
+
+// ensure the fake satisfies the interface.
+var _ scorer.Scorer = (*Scorer)(nil)

submitqueue/extension/speculation/scorer/fake/fake_test.go

@@ -0,0 +1,56 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package fake
+
+import (
+	"context"
+	"errors"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"github.com/uber/submitqueue/submitqueue/entity"
+)
+
+func TestScore_EchoesInputByDefault(t *testing.T) {
+	tree := entity.SpeculationTree{
+		BatchID: "q/batch/2",
+		Paths: []entity.SpeculationPathInfo{
+			{Path: entity.SpeculationPath{Head: "q/batch/2"}, Score: 0.9},
+		},
+	}
+	got, err := New().Score(context.Background(), tree)
+	require.NoError(t, err)
+	assert.Equal(t, tree, got)
+}
+
+func TestScore_ReturnsCanned(t *testing.T) {
+	in := entity.SpeculationTree{BatchID: "q/batch/2"}
+	scored := entity.SpeculationTree{
+		BatchID: "q/batch/2",
+		Paths: []entity.SpeculationPathInfo{
+			{Path: entity.SpeculationPath{Head: "q/batch/2"}, Score: 0.75},
+		},
+	}
+	got, err := New().Returns(scored).Score(context.Background(), in)
+	require.NoError(t, err)
+	assert.Equal(t, scored, got)
+}
+
+func TestScore_FailWith(t *testing.T) {
+	sentinel := errors.New("boom")
+	_, err := New().FailWith(sentinel).Score(context.Background(), entity.SpeculationTree{BatchID: "q/batch/1"})
+	require.ErrorIs(t, err, sentinel)
+}

submitqueue/extension/speculation/scorer/mock/BUILD.bazel

@@ -0,0 +1,13 @@
+load("@rules_go//go:def.bzl", "go_library")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["scorer_mock.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/scorer/mock",
+    visibility = ["//visibility:public"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "//submitqueue/extension/speculation/scorer:go_default_library",
+        "@org_uber_go_mock//gomock:go_default_library",
+    ],
+)

submitqueue/extension/speculation/scorer/mock/scorer_mock.go

@@ -0,0 +1,73 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package scorer
+
+//go:generate mockgen -source=scorer.go -destination=mock/scorer_mock.go -package=mock
+
+import (
+	"context"
+
+	"github.com/uber/submitqueue/submitqueue/entity"
+)
+
+// Scorer computes the predicted-success score of every path in a batch's
+// speculation tree.
+//
+// A path's score is a prediction — "how likely is this bet to pay off?" — and
+// predictions must move as evidence arrives. The scorer answers "how good is
+// each path right now" from the current state: the per-batch success
+// probabilities of a path's base batches (entity.Batch.Score, set by the score
+// stage), which of those dependencies have already landed or had their build
+// pass (resolved assumptions raise confidence), and optionally other signals
+// (how long the batch has waited, historical pass rates).
+//
+// The controller re-runs the scorer on every respeculate, right after it
+// reconciles path status — so when a dependency lands, its build passes, or a
+// sibling path fails, the surviving paths' scores are recomputed against the new
+// reality before anything is selected or prioritized. The controller drives
+// *when* to rescore and persists the result; the scorer owns the *formula*.
+//
+// This is the per-*path* scorer, distinct from the per-*batch* score stage
+// (extension/scorer), which sets entity.Batch.Score. The path scorer consumes
+// those batch scores to score whole paths.
+//
+// The controller hands the scorer the batch's speculation tree directly — the
+// subject it scores. Any richer signal an implementation needs (the dependency
+// batches' scores, historical pass rates) is injected at its Factory, not passed
+// here. It never writes: it returns the scored tree and the controller persists
+// the scores, keeping the controller the single writer of tree state.
+type Scorer interface {
+	// Score returns the given tree with each path's Score set to its freshly
+	// computed predicted-success value. Path structure and controller-stamped
+	// Status are carried through unchanged; only Score is (re)computed. The
+	// combination formula is the implementation's concern.
+	Score(ctx context.Context, tree entity.SpeculationTree) (entity.SpeculationTree, error)
+}
+
+// Config carries the per-queue identity handed to a Factory. The system knows
+// only the queue name; everything an implementation needs (scoring knobs, and
+// read access to any extra signals such as the dependency batches' scores) is
+// injected at construction by the integrator.
+type Config struct {
+	// QueueName identifies the queue this Scorer serves.
+	QueueName string
+}
+
+// Factory builds the Scorer for a queue. Implementations are provided by
+// integrators (and tests) and inject whatever they need at construction.
+type Factory interface {
+	// For returns the Scorer for the given queue.
+	For(cfg Config) (Scorer, error)
+}