Skip to content

DOWNLOAD_FIX_GUIDE.md

Latest commit

 

History

History
98 lines (80 loc) · 2.62 KB

DOWNLOAD_FIX_GUIDE.md

File metadata and controls

98 lines (80 loc) · 2.62 KB

Download Fix Guide

Problem

You encountered this error during Docker build:

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Root Cause

The issue occurs when:

  1. Network downloads fail silently (curl with -s flag)
  2. Incomplete or corrupted tar.gz files are downloaded
  3. The tar command receives empty or malformed input

Solutions Applied

✅ Fixed Download Pattern

Changed from:

curl -sfL URL | tar zxf - -C /tmp/dir --strip-components=1

To:

for i in 1 2 3; do \
  curl -fL URL -o /tmp/file.tar.gz && break || sleep 5; \
done \
&& tar zxf /tmp/file.tar.gz -C /tmp/dir --strip-components=1 \
&& rm -rf /tmp/file.tar.gz

✅ Key Improvements

  • Retry Logic: Downloads retry up to 3 times with 5-second delays
  • Error Detection: -f flag makes curl fail on HTTP errors
  • File Validation: Downloads to file first, then extracts
  • Cleanup: Removes temporary tar.gz files after extraction

✅ Fixed Downloads

  • libdeflate: Fixed with retry logic and proper error handling
  • libpng: Fixed with retry logic and proper error handling
  • GDAL: Fixed with retry logic and proper error handling

How It Works

Before (Problematic)

curl -sfL https://github.com/user/repo/archive/v1.0.tar.gz | tar zxf - -C /tmp/dir --strip-components=1
  • -s flag hides errors
  • Pipe can fail silently
  • No retry mechanism

After (Robust)

for i in 1 2 3; do \
  curl -fL https://github.com/user/repo/archive/v1.0.tar.gz -o /tmp/file.tar.gz && break || sleep 5; \
done \
&& tar zxf /tmp/file.tar.gz -C /tmp/dir --strip-components=1 \
&& rm -rf /tmp/file.tar.gz
  • -f flag shows HTTP errors
  • Downloads to file first (validates download)
  • Retry loop with delays
  • Proper cleanup

Testing the Fix

Build the Image

docker buildx build --platform=linux/amd64 -f dockerfiles/Dockerfile -t test-gdal .

If Still Having Issues

Try building with more verbose output:

docker buildx build --platform=linux/amd64 --progress=plain -f dockerfiles/Dockerfile -t test-gdal .

Use the Retry Script

./scripts/build-with-retry.sh 3.8.3 3.12

Additional Improvements

Network Resilience

  • Downloads now retry automatically
  • Better error reporting
  • Graceful failure handling

Build Optimization

  • Temporary files are cleaned up properly
  • Better layer caching
  • Reduced image size

The download issues should now be resolved! The retry logic and proper error handling will make your builds much more reliable. 🚀