Add parallel execution for bash script

[Trial97]

if make forge and neoforge concurrent why not do the same for the bash script?

[Ryex]

did you test this and the generate steps don't walk over each other?

[Trial97]

Running that right now: but the only issue I found is that the minecraft generate needs to be done before because neoforge and forge checks for mc version to exist

[Trial97]

something about wait doesn't work right now

[Ryex]

perhaps it needs to wait on spesfic pids?

#!/bin/bash

# Track exit status of multiple processes
declare -A process_status
run_and_track() {
    local name=$1
    shift
    "$@" &
    process_status[$!]="$name"
}

# Start processes
run_and_track "Task 1 (sleep 2)" sleep 2
run_and_track "Task 2 (sleep 1)" sleep 1
run_and_track "Task 3 (sleep 3)" sleep 3

echo "Background processes started: ${!process_status[@]}"

# Collect results
for pid in "${!process_status[@]}"; do
    wait "$pid"
    status=$?
    echo "${process_status[$pid]} (PID $pid) exited with status: $status"
done
process_status=()

echo "All processes complete"

[Trial97]

Nah this PR actually works: the issue was the neoforge cache stuff and I marked it as draft to not make it harder to debug.

ID is the process or job ID. If no ID is specified, the command waits until all child background jobs are completed.

source: https://linuxize.com/post/bash-wait/

[Trial97]

And right now the only issue I have with it is that I can't Ctrl^C it

[Ryex]

tracking the pids should let you trap and exit the pids

[Trial97]

did not worked that way on my testing
I tested with something like:

pids=()

function cleanup() {
  for pid in "${pids[@]}"; do
    kill "$pid" 2>/dev/null || true
  done
}

function fail_in() {
    cleanup
    upstream_git reset --hard HEAD
    exit 1
}

function fail_out() {
    cleanup
    launcher_git reset --hard HEAD
    exit 1
}

function run_in() {
  python -m "$@" || fail_in &
  pids+=($!)      # record PID
}

...
run_in meta.run.update_mojang 
run_in meta.run.update_forge 
run_in meta.run.update_neoforge 
run_in meta.run.update_fabric 
run_in meta.run.update_quilt 
run_in meta.run.update_liteloader 
run_in meta.run.update_java 
wait

But it is enough meta for today, I will give it a shot another time.
This is not urgent and with sha1 caching and concurrent runs already in place it should be pretty fast

[Scrumplex]

Maybe this would be reason enough to rewrite the script in Python? That way we could just use a single multiprocessing Pool for all update steps as well as a Pool for all generate steps