Skip to content

Bug: PermissionError on config migration write - temp files created read-only (0o444) #1012

@DevRickyCst

Description

@DevRickyCst

Description

When running source-facebook-marketing (v5.0.0) via PyAirbyte in a Docker-in-Docker setup, the connector's MigrateIncludeDeletedToStatusFilters.migrate() fails with a PermissionError when attempting to write back the migrated config to the temp file.

Related to #656 (same root cause, different connector).

Note: This issue does not reproduce on macOS with Docker Desktop, because Docker Desktop's VirtioFS/grpcfuse layer relaxes Unix permission enforcement on bind-mounted volumes. It manifests on Linux environments (e.g. ECS, CI) where permissions are strictly enforced.

Error

PermissionError: [Errno 13] Permission denied: '/airbyte/tmp/tmphpz2m88c.json'
  File "/usr/local/lib/python3.11/site-packages/airbyte_cdk/connector.py", line 67, in write_config
    with open(config_path, "w") as fh:

Root cause

In airbyte/_util/temp_files.py, temp config files are created with read-only permissions for all users (0o444):

# Grant "read" permission to all users
Path(temp_file.name).chmod(stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH)

In a Docker-in-Docker setup on Linux:

  • The outer container (PyAirbyte) runs as root and creates the temp file with 0o444
  • The inner connector container runs as airbyte user (uid 1000)
  • The connector's config migration (MigrateIncludeDeletedToStatusFilters) tries to write to the temp file → PermissionError

PR #637 introduced world-readable permissions on temp files but did not include write permissions, so config migrations that need to write back to the temp file still fail on Linux.

Possible fix (proposal)

One option could be to also grant write permissions to all users on the temp file:

Path(temp_file.name).chmod(
    stat.S_IRUSR | stat.S_IWUSR |
    stat.S_IRGRP | stat.S_IWGRP |
    stat.S_IROTH | stat.S_IWOTH
)  # 0o666

There may be other approaches (e.g. running the inner container as the same user, or a different temp file strategy) — happy to discuss what fits best.

Environment

  • airbyte: 0.44.1
  • Connector: airbyte/source-facebook-marketing:5.0.0
  • Execution context: Docker-in-Docker on Linux (ECS)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions