Skip to content

feat: add Amazon S3 Files support (NFS file system control plane) #707

@fdelbrayelle

Description

@fdelbrayelle

Context

AWS launched Amazon S3 Files (GA: April 7, 2026) — a service that makes S3 buckets mountable as NFS v4.1+ file systems with ~1ms latency, enabling POSIX-compatible access from EC2, Lambda, ECS, EKS, and Fargate. Reference: https://aws.amazon.com/blogs/aws/launching-s3-files-making-s3-buckets-accessible-as-file-systems/

The feature has a dedicated control-plane REST API (s3files.{region}.amazonaws.com) for managing file systems, mount targets, and access points. The Java SDK module (software.amazon.awssdk:s3files) exists on GitHub but is not yet published to Maven Central, so the implementation will use the AWS SDK v2's built-in SigV4 signing infrastructure (already present as a transitive dependency of software.amazon.awssdk:s3).


Proposed Implementation

New package: io.kestra.plugin.aws.s3.files

src/main/java/io/kestra/plugin/aws/s3/files/
├── AbstractS3Files.java        ← base class: credentials + SigV4 HTTP client factory
├── S3FilesService.java         ← shared HTTP request/response utility
├── CreateFileSystem.java       ← Task
├── GetFileSystem.java          ← Task
├── ListFileSystems.java        ← Task
├── DeleteFileSystem.java       ← Task
├── CreateMountTarget.java      ← Task
├── ListMountTargets.java       ← Task
├── DeleteMountTarget.java      ← Task
└── models/
    ├── FileSystem.java
    └── MountTarget.java

API operations → HTTP mapping

Task Method Path
CreateFileSystem POST /filesystems
GetFileSystem GET /filesystems/{fileSystemId}
ListFileSystems GET /filesystems[?bucket=ARN&maxResults=N&nextToken=T]
DeleteFileSystem DELETE /filesystems/{fileSystemId}
CreateMountTarget POST /filesystems/{fileSystemId}/mounttargets
ListMountTargets GET /filesystems/{fileSystemId}/mounttargets
DeleteMountTarget DELETE /mounttargets/{mountTargetId}

Task inputs/outputs

CreateFileSystem

  • Required: bucket (S3 bucket ARN), roleArn (IAM role ARN for S3 Files service access)
  • Optional: prefix, kmsKeyId, clientToken, tags
  • Output: fileSystemId, fileSystemArn, status, creationTime

GetFileSystem

  • Required: fileSystemId
  • Output: full FileSystem model

ListFileSystems

  • Optional: bucket (filter by bucket ARN), maxResults, nextToken
  • Output: List<FileSystem>, nextToken

DeleteFileSystem

  • Required: fileSystemId

CreateMountTarget

  • Required: fileSystemId, subnetId
  • Optional: ipAddress, securityGroups
  • Output: mountTargetId, ipAddress, status

ListMountTargets

  • Required: fileSystemId
  • Optional: maxResults, nextToken
  • Output: List<MountTarget>, nextToken

DeleteMountTarget

  • Required: mountTargetId

Technical Approach

  • AbstractS3Files extends AbstractConnection (reuses accessKeyId, secretKeyId, region, STS support) and provides a factory method for a signed HTTP client
  • Signing: Aws4Signer from software.amazon.awssdk:auth with service name "s3files" — already on the classpath transitively
  • HTTP transport: ApacheHttpClient from software.amazon.awssdk:apache-client (already in build.gradle)
  • No new build.gradle dependencies required for the initial implementation
  • When software.amazon.awssdk:s3files is eventually published to Maven Central, S3FilesService can be refactored to use the generated client; task APIs remain unchanged

Testing Strategy

LocalStack 3.4.0 likely does not yet support S3 Files. Two-tier approach:

  1. Unit tests (CI): Mock S3FilesService HTTP calls with Mockito to test request construction and response parsing
  2. Manual integration tests: @Disabled until LocalStack adds S3 Files support, then re-enable following the existing AbstractLocalStackTest pattern

References

Metadata

Metadata

Assignees

Labels

area/pluginPlugin-related issue or feature request
No fields configured for Feature.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions