Chunked processing across chained Queueables (source-agnostic)

[Mateusz7410]

Motivation

There is no first-class way to hand the framework a large data set and process it in tuned chunks
across chained Queueables, with per-chunk tracking and retry. Users hit this when they need to
process thousands of records in the background but do not want a Database.Batchable (heavier,
separate lifecycle) or one QueueableJob per record (blows limits, no coordination).

The data set can come from different places: an in-memory List / Set<Id>, a Database.Cursor
(SOQL cursor) for sets too large to hold in memory, or a raw SOQL string. The framework should be
agnostic to the source.

Proposal

A chunk entry point on Async that reuses the existing chaining + finalizer + retry machinery,
with a pluggable record source. The source is mandatory, so it goes in the entry method (cannot be
forgotten):

// in-memory ids or records
Async.chunk(new AccountRecalcJob(), ChunkSource.of(records))
    .chunkSize(50)
    .priority(5)
    .retry(2)
    .enqueue();

// large set via SOQL cursor - never held in memory at once
Async.chunk(new AccountRecalcJob(), ChunkSource.query('SELECT Id FROM Account WHERE ...'))
    .chunkSize(200)
    .enqueue();

The framework:

• Pages the source chunkSize records at a time via fetch(offset, count), carrying only the
  integer position across the chain.
• Drives chunks via chained Queueables (reusing existing chain + finalizer code, not a parallel
  class). Pages are enqueued lazily (self-chaining), so a huge set never materializes a giant job
  list; only one integer of state moves forward.
• Tracks each chunk in AsyncResult__c; a failed chunk retries per the configured policy + backoff.

ChunkSource seam

public abstract class ChunkSource {
    public abstract Integer size();
    public abstract List<SObject> fetch(Integer offset, Integer count);

    public static ChunkSource of(List<SObject> records);   // in-memory, also the test fake
    public static ChunkSource ofIds(Set<Id> ids);          // id-only source
    public static ChunkSource cursor(Database.Cursor c);   // wraps Database.Cursor
    public static ChunkSource query(String soql);          // convenience over Database.getCursor
}

size() + fetch(offset, count) mirror Database.Cursor, so the cursor adapter is a thin
delegate. ChunkSource.of(records) lets tests exercise chunking with in-memory records instead of
inserting data and opening a live cursor. Stays SOQL-Lib-free: callers pass their own cursor
(for example SOQL.of(...).toCursor()) into ChunkSource.cursor(...).

Verified on a scratch org that a Database.Cursor held as Queueable member state survives
re-enqueue across the chain (each link deserializes the cursor and fetches the correct page), so
no per-page re-open is needed.

Job base - two override shapes

Both are compile-enforced (one abstract method each), so a wrong override cannot become a silent
no-op:

public abstract class ChunkJob extends QueueableJob {        // records
    public abstract void work(List<SObject> chunk);
}

public abstract class IdChunkJob extends ChunkJob {          // ids convenience
    public abstract void work(Set<Id> ids);
    // framework plucks ids from the fetched records and calls work(ids)
}

Change from the original issue: the record base overrides work(List<SObject> chunk); the id base
overrides work(Set<Id> ids). Pick whichever fits the job.

Failure handling - two independent levels

• Outer chain (existing, reused from QueueableBuilder): continueOnJobExecuteFail /
  rollbackOnJobExecuteFail / fail-chain (see Opt-in failChainOnJobExecuteFail to actually stop the chain on job failure #36) govern whether jobs after the chunk run still
  execute. Unchanged.
• Within the chunk run (new): .stopRemainingChunksOnFailure(). Default off = keep processing
  the remaining chunks even if one fails (best-effort, Batchable-like). When set, a chunk that
  exhausts retries stops the remaining chunks of this run, then hands back to the outer chain per
  the chain-level setting (so later chain jobs still run if the chain is set to continue).

Notes

• Position against native Batchable: the value is a Queueable-based chunked run that keeps our
  tracking, retry / backoff, dependsOn, and finalizer semantics.
• Reuses QueueableJob chaining and the work() convention; a chunked job overrides
  work(List<SObject> chunk) (or work(Set<Id> ids)) instead of the no-arg work().

Acceptance criteria

• New builder path; no change to any existing global signature. New globals: ChunkSource,
  ChunkJob, IdChunkJob, plus the Async.chunk entry.
• The new globals are added to the public -> global rewrite list in
  scripts/create-unlocked-package-version.sh, and the built package is installed on a
  no-namespace scratch and its tests pass (source deploy alone does not prove the rewrite).
• Apex tests: chunking (in-memory), cursor delegation, id source, partial failure + retry,
  .stopRemainingChunksOnFailure() behavior, empty source is a safe no-op, invalid chunkSize
  rejected.
• website/api doc, including the cursor governor caps (rows, lifespan, daily cursor opens) so
  users can choose chunk vs Batchable.