Description
When running source-facebook-marketing (v5.0.0) via PyAirbyte in a Docker-in-Docker setup, the connector's MigrateIncludeDeletedToStatusFilters.migrate() fails with a PermissionError when attempting to write back the migrated config to the temp file.
Related to #656 (same root cause, different connector).
Note: This issue does not reproduce on macOS with Docker Desktop, because Docker Desktop's VirtioFS/grpcfuse layer relaxes Unix permission enforcement on bind-mounted volumes. It manifests on Linux environments (e.g. ECS, CI) where permissions are strictly enforced.
Error
PermissionError: [Errno 13] Permission denied: '/airbyte/tmp/tmphpz2m88c.json'
File "/usr/local/lib/python3.11/site-packages/airbyte_cdk/connector.py", line 67, in write_config
with open(config_path, "w") as fh:
Root cause
In airbyte/_util/temp_files.py, temp config files are created with read-only permissions for all users (0o444):
# Grant "read" permission to all users
Path(temp_file.name).chmod(stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH)
In a Docker-in-Docker setup on Linux:
- The outer container (PyAirbyte) runs as
root and creates the temp file with 0o444
- The inner connector container runs as
airbyte user (uid 1000)
- The connector's config migration (
MigrateIncludeDeletedToStatusFilters) tries to write to the temp file → PermissionError
PR #637 introduced world-readable permissions on temp files but did not include write permissions, so config migrations that need to write back to the temp file still fail on Linux.
Possible fix (proposal)
One option could be to also grant write permissions to all users on the temp file:
Path(temp_file.name).chmod(
stat.S_IRUSR | stat.S_IWUSR |
stat.S_IRGRP | stat.S_IWGRP |
stat.S_IROTH | stat.S_IWOTH
) # 0o666
There may be other approaches (e.g. running the inner container as the same user, or a different temp file strategy) — happy to discuss what fits best.
Environment
airbyte: 0.44.1
- Connector:
airbyte/source-facebook-marketing:5.0.0
- Execution context: Docker-in-Docker on Linux (ECS)
Description
When running
source-facebook-marketing(v5.0.0) via PyAirbyte in a Docker-in-Docker setup, the connector'sMigrateIncludeDeletedToStatusFilters.migrate()fails with aPermissionErrorwhen attempting to write back the migrated config to the temp file.Related to #656 (same root cause, different connector).
Error
Root cause
In
airbyte/_util/temp_files.py, temp config files are created with read-only permissions for all users (0o444):In a Docker-in-Docker setup on Linux:
rootand creates the temp file with0o444airbyteuser (uid 1000)MigrateIncludeDeletedToStatusFilters) tries to write to the temp file →PermissionErrorPR #637 introduced world-readable permissions on temp files but did not include write permissions, so config migrations that need to write back to the temp file still fail on Linux.
Possible fix (proposal)
One option could be to also grant write permissions to all users on the temp file:
There may be other approaches (e.g. running the inner container as the same user, or a different temp file strategy) — happy to discuss what fits best.
Environment
airbyte: 0.44.1airbyte/source-facebook-marketing:5.0.0