HelpersTask1207_Debug_dind_and_sibling_container_issue_on_containerized_workflow

[heanhsok]

Task #1207
Pre-commit checks:
All checks passed ✅

[heanhsok]

Adding this check back (https://github.com/causify-ai/helpers/pull/1203/changes#r3113612752), as we need to ensure it does not cause issues when running on ECS Fargate through Airflow, since ECS Fargate does not support either DinD or sibling containers.

See: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/fargate-security-considerations.html

• No privileged containers or access - Features such as privileged containers or access are currently unavailable on Fargate. This will affect uses cases such as running Docker in Docker.

• Limited access to Linux capabilities - The environment in which containers run on Fargate is locked down. Additional Linux capabilities, such as CAP_SYS_ADMIN and CAP_NET_ADMIN, are restricted to prevent a privilege escalation. Fargate supports adding the CAP_SYS_PTRACE Linux capability to tasks to allow observability and security tools deployed within the task to monitor the containerized application.

• No access to the underlying host - Neither customers nor AWS operators can connect to a host running customer workloads. You can use ECS exec to run commands in or get a shell to a container running on Fargate. You can use ECS exec to help collect diagnostic information for debugging. Fargate also prevents containers from accessing the underlying host’s resources, such as the file system, devices, networking, and container runtime.

• Networking - You can use security groups and network ACLs to control inbound and outbound traffic. Fargate tasks receive an IP address from the configured subnet in your VPC.

FYI @PomazkinG @gpsaggese @Shayawnn

[PomazkinG]

Overall LG since it solves the problem at hand

Leaving a non-blocking comment

[PomazkinG]

I think here we want to return False if we are inside AirFlow.

This is a proxy for "if we are inside AirFlow"

if not has_dind_support() and not use_docker_sibling_containers():

We used to use is_inside_ecs_container() previously for that. However, I am not sure if that still would work. Or maybe we can create a new function that detects if we are running smth via AirFlow

Just leaving a TODO is fine