Skip to content

docs(config): rewrite max_partitions_per_task description to be timeless #2093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
avantgardnerio merged 1 commit into from
Jul 19, 2026
Merged

docs(config): rewrite max_partitions_per_task description to be timeless #2093

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 6 additions & 7 deletions ballista/core/src/config.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -359,13 +359,12 @@ static CONFIG_ENTRIES: LazyLock<HashMap<String, ConfigEntry>> = LazyLock::new(||
ConfigEntry::new(
BALLISTA_SCHEDULER_MAX_PARTITIONS_PER_TASK.to_string(),
"Upper bound on the number of input partitions packed into a single \
task's `partition_slice`. `1` (default) is the pre-multi-partition-tasks \
execution model — one task per partition, matching the Spark user \
persona and preserving master's behaviour on merge. Raise to enable \
multi-partition tasks (fewer tasks, parallel-sort / parallel-join wins); \
`0` means unbounded — the scheduler fills each task up to the executor's \
free vcore count. Does not apply to collapse stages, which must pack \
their full pending queue into a single task for correctness."
task's `partition_slice`. `1` (default) means one task per input \
partition. Raise to enable multi-partition tasks (fewer tasks, \
parallel-sort / parallel-join wins); `0` means unbounded — the \
scheduler fills each task up to the executor's free vcore count. \
Does not apply to collapse stages, which must pack their full \
pending queue into a single task for correctness."
.to_string(),
DataType::UInt64,
Some(1.to_string()),
Expand Down