Skip to content

Renovate pipeline task hangs ~15 minutes after Renovate exits with an error #16603

@lbussell

Description

@lbussell

When Renovate encounters an unhandledRejection error during postUpgradeTasks execution (e.g. a missing tool in the container), the "Run Renovate" pipeline task hangs for ~15 minutes between the Renovate process exiting and the bash wrapper script completing.

Observed behavior

Build #2929718 (dotnet-docker-renovate pipeline):

  • 15:20:46 — Renovate logs its final output and exits with code 1
  • 15:35:46 — The bash task reports Bash exited with code '1'15 minutes later

During the gap, the only log output is periodic Agent environment resources debug lines. No useful work is being done.

Copilot's root cause hypothesis

The pipeline template runs Renovate with this wrapper:

. /__w/1/s/dotnet-arcade/eng/common/renovate.env
renovate 2>&1 | tee /tmp/renovate.out
renovateExit=${PIPESTATUS[0]}
if grep -q '^ WARN:' /tmp/renovate.out; then
  echo "##vso[task.logissue type=warning]Renovate produced warnings."
  echo "##vso[task.complete result=SucceededWithIssues]"
fi
exit $renovateExit

The renovate 2>&1 | tee ... pipeline may keep the bash process alive if the Renovate container's child processes (spawned via postUpgradeTasks) don't fully terminate or close their inherited file descriptors. The unhandledRejection from the missing pwsh binary suggests the spawned process errored out but may not have been cleaned up cleanly, leaving tee waiting on an open pipe.

Underlying Renovate error (for context)

The postUpgradeTasks tried to run pwsh which is not installed in the azurelinux-3.0-renovate-43-amd64 container image:

ERROR: unhandledRejection
  "code": "ENOENT",
  "syscall": "spawn pwsh",
  "path": "pwsh",
  "spawnargs": ["./eng/dockerfile-templates/Get-GeneratedDockerfiles.ps1"],
  "message": "spawn pwsh ENOENT"

The missing PowerShell is being addressed in dotnet/dotnet-buildtools-prereqs-docker#1608. This issue is specifically about the pipeline hanging when postUpgradeTasks fail.

Expected behavior

When Renovate exits (whether successfully or with an error), the bash wrapper script should complete promptly, not hang for 15 minutes.

Environment

  • Pipeline: dotnet-docker-renovate
  • Build: #2929718
  • Container image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-renovate-43-amd64
  • Renovate version: 43.77.3
  • Template: /eng/common/core-templates/stages/renovate.yml@arcade

/cc @mthalman

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions