Skip to content

awsdataarchitect/s3files-mount

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 

Repository files navigation

s3files-mount

Mount S3 buckets locally on macOS via Amazon S3 Files — the missing local mount.

S3 Files (launched April 2026) provides NFS-based file system access to S3 buckets, but only supports AWS compute (EC2, Lambda, EKS, ECS). This tool brings S3 Files to your Mac.

Architecture

Architecture

S3 Files requires NFSv4.2 + TLS + IAM authentication (via efs-utils/efs-proxy). macOS only supports NFSv4.0, so Docker is required to run the Linux NFS client. The container re-exports the mount via WebDAV for native Mac access.

Prerequisites

Quick Start

# 1. Deploy infrastructure (VPC, bucket, IAM role, S3 Files, NLB)
cd infra && npm install && npx cdk deploy -c region=ca-central-1

# 2. Mount S3 bucket (auto-detects region + FS ID from NLB)
./docker/docker-mount.sh up <NLB_DNS_from_CDK_output>

# 3. Use it — it's just a folder
ls /tmp/s3files/
echo "hello" > /tmp/s3files/test.txt
open /tmp/s3files  # opens in Finder

# 4. Tear down
./docker/docker-mount.sh down

That's it. docker-mount.sh up builds the container, starts efs-proxy with TLS + IAM auth, mounts S3 Files via NFS, starts WebDAV, and mounts WebDAV at /tmp/s3files. To deploy in another region, just change the -c region= flag.

How It Works

Why Docker?

macOS only supports NFSv4.0. S3 Files requires NFSv4.2. There's no way to upgrade the macOS NFS client — it's baked into the kernel. Docker runs a Linux VM with full NFSv4.2 support.

Why efs-proxy?

S3 Files rejects all unencrypted NFS connections. It requires TLS + IAM authentication on every mount. The efs-proxy binary (part of amazon-efs-utils) handles this:

  1. Listens on localhost (e.g., port 20049)
  2. Connects to mount target on port 2049 with TLS
  3. Performs EFS RPC Bind with AWS credentials
  4. Forwards NFS traffic over the authenticated connection

mount -t s3files invokes efs-proxy automatically. Raw mount -t nfs4 gets "access denied" because it skips the auth layer.

Why WebDAV?

We benchmarked three protocols for re-exporting the NFS mount to macOS:

Operation Docker (NFS direct) Mac (WebDAV) Mac (SMB)
List directory 0.09s 0.08s 4.3s
Read small file 0.13s 0.05s 0.49s
Write + read back 0.27s 0.53s 1.7s
Throughput Docker (NFS) WebDAV SMB
10MB write 1.2s 1.4s 11.0s
10MB read 0.10s 0.03s 0.42s
100MB write 6.8s 9.3s 87.0s
Write throughput ~15 MB/s ~11 MB/s ~1.1 MB/s
Read throughput ~830 MB/s ~400 MB/s ~24 MB/s

WebDAV is 10–54x faster than SMB on macOS. Apple's SMB implementation is notoriously chatty — signing, metadata fetches, and delayed ACKs add massive overhead. WebDAV is lightweight HTTP with none of that baggage.

Why NLB?

Docker Desktop has no IPv6 connectivity. The mount target's private IPv4 (10.0.0.x) is only reachable from inside the VPC. An internet-facing NLB bridges the gap — Docker connects to the NLB's public IP, which forwards TCP 2049 to the mount target.

efs-proxy ReadBypass Incompatibility

The efs-proxy ReadBypass module (direct S3 reads for large files) crashes in our non-standard Docker + NLB setup:

ERROR efs_proxy::nfs::nfs_reader Error handling parsing error SendError { .. }

Fix: nodirects3read mount option disables ReadBypass. This is set automatically in our entrypoint. On a standard EC2 instance, ReadBypass likely works as intended.

efs-utils Config for Docker

Since Docker isn't an EC2 instance, there's no metadata service. The Dockerfile patches /etc/amazon/efs/s3files-utils.conf:

  • Removes {az_id} from dns_name_format (no AZ metadata available)
  • Region is set at runtime via the AWS_REGION environment variable

CDK Infrastructure

The infra/ directory provisions everything in one command:

  • VPC with public subnet + Internet Gateway (PoC only — for production, use private subnets with a Gateway endpoint for S3, which is free and routes S3 traffic through the AWS network, bypassing NAT Gateway costs)
  • S3 bucket (versioning enabled, required by S3 Files)
  • IAM role for S3 Files (trust: elasticfilesystem.amazonaws.com)
  • S3 Files file system + mount target (via AWS::S3Files::FileSystem / AWS::S3Files::MountTarget)
  • NLB (TCP 2049) with target group pointing at the mount target
  • Security groups: NLB SG (TCP 2049 inbound) → mount target SG (NLB SG only)
cd infra && npm install && npx cdk deploy -c region=ca-central-1

Outputs: NlbDns, BucketName, FileSystemId.

Region Matters

Pick the region closest to you — latency is dominated by internet RTT:

Operation (Docker NFS) us-east-2 ca-central-1
Read small file 0.13s 0.06s
Write + read back 0.27s 0.16s
10MB write 1.2s 1.0s

From Canada, ca-central-1 is ~40% faster for interactive operations.

What Didn't Work (and Why)

Approach Result Root Cause
Native macOS NFS mount 💀 Kernel panic (5x) macOS NFSv4.0 ≠ S3 Files NFSv4.2
Raw mount -t nfs4 (no efs-proxy) ❌ "access denied" S3 Files requires EFS RPC Bind auth
NLB + efs-proxy without TLS ❌ "access denied" S3 Files requires TLS on all connections
efs-proxy with ReadBypass enabled ❌ Proxy crash loop ReadBypass incompatible with Docker + NLB setup
NLB + efs-proxy + TLS + nodirects3read Works All requirements met

NFS Mount Options

These are set automatically by mount -t s3files (via efs-utils). From the S3 Files mounting docs:

Option Value Why
nfsvers=4.2 Required S3 Files requires NFSv4.2
hard Retries forever (AWS recommended)
timeo=600 60s AWS recommended for EFS/S3 Files
retrans=2 AWS recommended
noresvport Standard for Linux NFS clients
nodirects3read Avoids ReadBypass crash in efs-proxy

TLS and IAM auth are handled automatically by efs-proxy.

Performance Tuning

For faster SMB (if using Samba instead of WebDAV), tune the macOS SMB client:

# Disable delayed ACK (biggest win)
sudo sysctl net.inet.tcp.delayed_ack=0

# Disable .DS_Store on network shares
defaults write com.apple.desktopservices DSDontWriteNetworkStores -bool TRUE

/etc/nsmb.conf:

[default]
signing_required=no
notify_off=yes
port445=no_netbios
dir_cache_max_cnt=4096
protocol_vers_map=4
validate_neg_off=yes
streams=yes
soft=yes
mc_prefer_wired=yes

⚠️ Security Note

The NLB is internet-facing to allow connections from your Mac. This is a PoC configuration — acceptable because:

  • S3 Files enforces TLS encryption + IAM authentication on all connections
  • NLB security group restricts inbound to TCP 2049 only
  • Mount target security group only allows traffic from the NLB security group

For production:

IAM Role Setup

If not using the CDK stack, S3 Files needs an IAM role with:

Trust policy (elasticfilesystem.amazonaws.com):

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "Service": "elasticfilesystem.amazonaws.com" },
    "Action": "sts:AssumeRole"
  }]
}

Permissions (S3 + EventBridge):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject*", "s3:PutObject*", "s3:DeleteObject*", "s3:ListBucket", "s3:ListBucketVersions", "s3:AbortMultipartUpload"],
      "Resource": ["arn:aws:s3:::my-bucket", "arn:aws:s3:::my-bucket/*"]
    },
    {
      "Effect": "Allow",
      "Action": ["events:PutRule", "events:PutTargets", "events:DeleteRule", "events:DisableRule", "events:EnableRule", "events:RemoveTargets"],
      "Resource": ["arn:aws:events:*:*:rule/DO-NOT-DELETE-S3-Files*"],
      "Condition": { "StringEquals": { "events:ManagedBy": "elasticfilesystem.amazonaws.com" } }
    }
  ]
}

Your IAM user also needs s3files:ClientMount and s3files:ClientWrite permissions.

Project Structure

├── docker/
│   ├── Dockerfile          # Amazon Linux + efs-utils + WebDAV
│   ├── docker-compose.yml
│   ├── docker-mount.sh     # One command: up/down/shell/ls/cp
│   └── entrypoint.sh       # Mounts S3 Files + starts WebDAV
├── infra/                  # CDK stack (VPC, bucket, IAM, S3 Files, NLB)
├── README.md
└── .gitignore

Windows / WSL2

On Windows, WSL2 runs a real Linux kernel (5.15+) with full NFSv4.2 support. You can install amazon-efs-utils directly in WSL2 and mount S3 Files without Docker — no WebDAV re-export needed. The WSL2 mount appears as a native Linux path accessible from Windows Explorer via \\wsl$\.

This is untested but should work since WSL2's kernel supports all the NFS features S3 Files requires. You'd still need the NLB (or VPN) for connectivity.

S3 Files vs Mountpoint for Amazon S3

Mountpoint for Amazon S3 is a FUSE-based client that mounts S3 as a local filesystem. It's different from S3 Files:

S3 Files Mountpoint for S3
Protocol NFS (NFSv4.2) FUSE
Read/Write Full read/write Read-heavy (limited writes)
Latency Sub-millisecond Milliseconds
Sync Bidirectional (S3 ↔ FS) One-way (S3 → FS)
Requires Mount target in VPC Just IAM credentials
Platform Linux only (EC2, ECS, EKS) Linux, macOS

S3 Files is a managed NFS layer over S3 with full read/write and bidirectional sync. Mountpoint is a lightweight FUSE client optimized for reading large datasets.

License

MIT

About

Mount S3 buckets locally on macOS via Amazon S3 Files — Docker + efs-proxy + NLB + WebDAV. Two commands, native Finder folder.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors