Consolidate Multiple Queueable Chains Created Within the Same Execution Context

[Mateusz7410]

Consolidate Multiple Queueable Chains Created Within the Same Execution Context

Depends on / Follow-up to #15

This builds on #15 — Smarter Async Execution: Context-Aware Queueable Invocation. #15 introduces context-aware decision logic (run inline vs. enqueue, based on context/limits/metadata). That capability is the foundation for the "smart jobs" behavior described here, so #15 should land first. This issue extends the idea from "should I enqueue at all?" to "if several enqueues are happening in the same context, can we merge them?"

Motivation

When multiple pieces of code independently enqueue Queueables within the same execution context, the framework fires a separate async job for each — even when the work is small and could safely share one job.

The most common source is trigger batching. Salesforce processes DML in chunks of 200 records:

• Update 5,000 records → 25 chunks → trigger fires 25 times
• If the trigger logic enqueues one Queueable per invocation → 25 separate Queueable jobs

Even for a trivial change, this produces 25 async jobs that:

• Consume the async/Queueable job limits unnecessarily
• Add scheduling and monitoring overhead (25 AsyncResult__c rows, 25 chains to trace)
• Push toward the overflow threshold (50 queueables → QueueableChainSchedulable) far faster than the actual workload warrants
• Fragment work that could be processed together far more efficiently

The same pattern appears outside triggers wherever several independent callers enqueue in one transaction (orchestration layers, bulk service calls, loops).

Proposed Idea

Introduce a mechanism to collect, merge, or consolidate Queueable chains enqueued within the same execution context, so that N small independent enqueues become a smaller number (ideally 1–5) of consolidated jobs that process the same total work.

Rather than "one enqueue → one job," the framework could:

• Buffer enqueue requests during a transaction and flush them as a consolidated set at the end (e.g., via a trigger-friendly commit/flush hook)
• Coalesce compatible jobs — same job type / same target logic — into a single job carrying the merged payload (e.g., accumulated record IDs)
• Group work up to a size or governor-limit budget, spilling into additional jobs only when a group is "full"

Decision Logic (open questions)

Consolidation is only safe when the merged jobs are genuinely equivalent. Behavior could be driven by:

• Job identity / compatibility — same Queueable type and configuration → mergeable; different types → keep separate
• Payload semantics — jobs whose payload is an ID/record collection are naturally mergeable; jobs with opaque or order-dependent state may not be
• Group size budget — max records / max CPU per consolidated job before spilling to the next
• Configurable defaults via Custom Metadata (reuse the QueueableJobSetting__mdt pattern), plus per-call overrides:
  • Always consolidate
  • Never consolidate (preserve one-job-per-enqueue)
  • Let the framework decide (mirrors Smarter Async Execution: Context-Aware Queueable Invocation #15's "let the framework decide" mode)

Open Design Questions

• Buffer boundary — where is the "same execution context" boundary and when do we flush? Trigger context has no single natural end-of-transaction hook for async; a finalizer, a @future, or an explicit flush call may be needed.
• Merge key — how does a caller declare "these jobs are mergeable"? (Interface method, marker, or automatic by type + payload shape.)
• Payload combining — framework-provided contract for merging payloads (e.g., merge(other) on the job) vs. framework only concatenating record collections.
• Ordering & idempotency — must consolidated work preserve per-record ordering? What happens to chained/sequenced jobs?
• Failure isolation — if one merged unit fails, does the whole consolidated job fail? Partial-failure semantics need definition.
• Interaction with overflow — consolidation should reduce pressure on the 50-queueable overflow path; confirm the two mechanisms compose cleanly.

Benefits

• Fewer async jobs — 25 trigger-driven Queueables collapse to 1–5
• Better limit utilization — one job doing more useful work vs. many jobs doing little
• Less overflow pressure — stay well under the 50-queueable threshold for the same workload
• Simpler observability — fewer chains and AsyncResult__c rows to trace
• Foundation for smart jobs — combined with Smarter Async Execution: Context-Aware Queueable Invocation #15, the framework can decide both whether to go async and how many jobs to actually create

Out of Scope (for now)

• Cross-transaction consolidation (merging jobs across separate triggers/transactions)
• Automatic payload merging for jobs with opaque or non-collection state — initial version can require callers to opt in and declare mergeability