feat(speculation): add enumerator + dependency-limit extensions

Add the first speculation seam and its limit counterpart from the speculation RFC, as vendor-agnostic extension interfaces under submitqueue/extension/speculation/.

enumerator: given a batch and its ordered active dependencies, mechanically lists the candidate Base/Head paths — pure, deterministic, and purely structural. It sets no score and no status; the controller stamps status on persist and the scorer fills score. (Its earlier scoring responsibility is dropped here, per the RFC, and moves to the scorer seam.)

dependencylimit: the "how much" policy bounding how many active (in-flight) dependencies a batch may speculate over. It is the eligibility gate before enumeration; unlike the other speculation limits it is controller-held rather than injected into a seam, keeping the enumerator pure. The value is signal-driven, not a fixed constant.

Each follows the repo extension contract (conflict.Analyzer reference shape): Factory.For(Config) (T, error) with Config carrying only QueueName; behavioral knobs and limit signals are integrator-injected at construction. Includes READMEs, gomock packages, and Makefile mock-gen wiring. Interfaces only; concrete impls and controller wiring are deferred.

Author: behinddwalls

Makefile

@@ -364,7 +364,7 @@ local-stovepipe-stop: ## Stop the Stovepipe service
 
 mocks: ## Generate mock files using mockgen
 	@echo "Generating mocks..."
-	@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./platform/extension/counter/... ./platform/extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/extension/validator/... ./platform/consumer/... ./stovepipe/extension/storage/... ./stovepipe/extension/sourcecontrol/...
+	@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./platform/extension/counter/... ./platform/extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/extension/speculation/enumerator/... ./submitqueue/extension/speculation/dependencylimit/... ./submitqueue/extension/validator/... ./platform/consumer/... ./submitqueue/core/changeset/... ./stovepipe/extension/storage/... ./stovepipe/extension/sourcecontrol/...
 	@echo "Mocks generated successfully!"
 
 proto: ## Generate protobuf files from .proto definitions

submitqueue/extension/speculation/dependencylimit/BUILD.bazel

@@ -0,0 +1,8 @@
+load("@rules_go//go:def.bzl", "go_library")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["dependencylimit.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/dependencylimit",
+    visibility = ["//visibility:public"],
+)

submitqueue/extension/speculation/dependencylimit/README.md

@@ -0,0 +1,17 @@
+# Speculation Dependency Limit
+
+Vendor-agnostic "how much" policy that bounds how many **active** (in-flight, non-terminal) dependencies a batch may speculate over.
+
+See the [Speculation RFC](/doc/rfc/submitqueue/speculation.md) for the end-to-end design and how limits fit into the two-layer speculation model.
+
+## Dependency Limit
+
+Speculation splits into *decision seams* (what to build) and *limit policies* (how much to allow). The dependency limit is the first limit: it is the **eligibility gate** for speculation. A batch becomes eligible to enumerate only when its count of active dependencies is at or below the current limit; otherwise it waits. Nothing is dropped — as dependencies land they leave the active set, the count shrinks, and the batch is admitted. The gate applies even to the fully-stacked happy path, so a very long chain is not speculated in full at once.
+
+The value is **signal-driven**, not a fixed constant. Its primary input is the build system's available capacity, so a period of CI pressure can shrink how deep the queue speculates, but a policy may also weigh historical pass rates, cost budgets, time of day, or an experiment toggle. Because the value is dynamic, a change to the limit alone — not only a landing dependency or a DAG change — can newly admit a waiting batch.
+
+Unlike the selection and prioritization limits, the dependency limit is **not injected into a decision seam**. It gates eligibility *before* enumeration and needs active-dependency reconciliation, which is controller orchestration — so the controller holds it, consults it on every respeculate, and applies it, keeping the enumerator pure.
+
+## Factory
+
+A per-queue factory returns the limit policy for a queue, following the repo's extension contract. It is handed only the queue identity; the signals a policy weighs — a capacity feed, historical metrics, config — are injected at construction by the integrator in the wiring layer. Computing the limit itself takes no further inputs.

submitqueue/extension/speculation/dependencylimit/dependencylimit.go

@@ -0,0 +1,61 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package dependencylimit
+
+//go:generate mockgen -source=dependencylimit.go -destination=mock/dependencylimit_mock.go -package=mock
+
+import "context"
+
+// DependencyLimit is the "how much" policy that bounds how many active
+// (in-flight, non-terminal) dependencies a batch may speculate over.
+//
+// It is the eligibility gate for speculation: a batch becomes eligible to
+// enumerate only when its count of active dependencies is at or below the
+// current limit; otherwise it waits, and is admitted later as predecessors land
+// and leave the active set. The limit is a bound, not a trim — nothing is
+// dropped from a batch's base.
+//
+// The value is dynamic: it may change between calls — not only when a
+// dependency lands — so a change alone can newly admit a waiting batch, and the
+// controller re-consults it on every respeculate rather than caching it.
+//
+// This limit is the exception among the speculation limits: it gates eligibility
+// *before* enumeration and needs active-dependency reconciliation, which is
+// controller orchestration — so the controller holds and applies it, rather than
+// it being injected into a decision seam. The enumerator stays pure.
+type DependencyLimit interface {
+	// Limit returns the current maximum number of active dependencies a batch
+	// may speculate over. The controller compares a batch's active-dependency
+	// count against this to decide eligibility. It takes no parameters; anything
+	// an implementation needs is injected at construction.
+	Limit(ctx context.Context) (int, error)
+}
+
+// Config carries the per-queue identity handed to a Factory. The system knows
+// only the queue name; everything a policy needs to compute the limit (a
+// capacity feed, historical metrics, config) is injected at construction by the
+// integrator.
+type Config struct {
+	// QueueName identifies the queue this DependencyLimit serves.
+	QueueName string
+}
+
+// Factory builds the DependencyLimit for a queue. Implementations are provided
+// by integrators (and tests) and inject whatever signals they need at
+// construction.
+type Factory interface {
+	// For returns the DependencyLimit for the given queue.
+	For(cfg Config) (DependencyLimit, error)
+}

submitqueue/extension/speculation/dependencylimit/fake/BUILD.bazel

@@ -0,0 +1,19 @@
+load("@rules_go//go:def.bzl", "go_library", "go_test")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["fake.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/dependencylimit/fake",
+    visibility = ["//visibility:public"],
+    deps = ["//submitqueue/extension/speculation/dependencylimit:go_default_library"],
+)
+
+go_test(
+    name = "go_default_test",
+    srcs = ["fake_test.go"],
+    embed = [":go_default_library"],
+    deps = [
+        "@com_github_stretchr_testify//assert:go_default_library",
+        "@com_github_stretchr_testify//require:go_default_library",
+    ],
+)

submitqueue/extension/speculation/dependencylimit/fake/fake.go

@@ -0,0 +1,52 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package fake provides a programmable dependencylimit.DependencyLimit for tests
+// and examples. New sets the value returned by Limit; FailWith injects an error
+// on every call. It is intended for examples and tests only, never production.
+package fake
+
+import (
+	"context"
+
+	"github.com/uber/submitqueue/submitqueue/extension/speculation/dependencylimit"
+)
+
+// DependencyLimit is a programmable dependencylimit.DependencyLimit.
+type DependencyLimit struct {
+	limit int
+	err   error
+}
+
+// New returns a fake DependencyLimit whose Limit returns the given value.
+func New(limit int) *DependencyLimit {
+	return &DependencyLimit{limit: limit}
+}
+
+// FailWith makes every Limit call return err.
+func (l *DependencyLimit) FailWith(err error) *DependencyLimit {
+	l.err = err
+	return l
+}
+
+// Limit returns the configured value, or the injected error if FailWith was set.
+func (l *DependencyLimit) Limit(_ context.Context) (int, error) {
+	if l.err != nil {
+		return 0, l.err
+	}
+	return l.limit, nil
+}
+
+// ensure the fake satisfies the interface.
+var _ dependencylimit.DependencyLimit = (*DependencyLimit)(nil)

submitqueue/extension/speculation/dependencylimit/fake/fake_test.go

@@ -0,0 +1,36 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package fake
+
+import (
+	"context"
+	"errors"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestLimit_ReturnsConfiguredValue(t *testing.T) {
+	got, err := New(3).Limit(context.Background())
+	require.NoError(t, err)
+	assert.Equal(t, 3, got)
+}
+
+func TestLimit_FailWith(t *testing.T) {
+	sentinel := errors.New("boom")
+	_, err := New(3).FailWith(sentinel).Limit(context.Background())
+	require.ErrorIs(t, err, sentinel)
+}

submitqueue/extension/speculation/dependencylimit/mock/BUILD.bazel

@@ -0,0 +1,12 @@
+load("@rules_go//go:def.bzl", "go_library")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["dependencylimit_mock.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/dependencylimit/mock",
+    visibility = ["//visibility:public"],
+    deps = [
+        "//submitqueue/extension/speculation/dependencylimit:go_default_library",
+        "@org_uber_go_mock//gomock:go_default_library",
+    ],
+)

submitqueue/extension/speculation/dependencylimit/mock/dependencylimit_mock.go

@@ -0,0 +1,9 @@
+load("@rules_go//go:def.bzl", "go_library")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["enumerator.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/enumerator",
+    visibility = ["//visibility:public"],
+    deps = ["//submitqueue/entity:go_default_library"],
+)