Skip to content

HDDS-15958. Return InvalidURI for unreadable S3 object keys on GetObject#10860

Open
Gargi-jais11 wants to merge 1 commit into
apache:masterfrom
Gargi-jais11:HDDS-15958
Open

HDDS-15958. Return InvalidURI for unreadable S3 object keys on GetObject#10860
Gargi-jais11 wants to merge 1 commit into
apache:masterfrom
Gargi-jais11:HDDS-15958

Conversation

@Gargi-jais11

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

S3 Gateway returns 404 NoSuchKey for GetObject requests with unreadable/invalid object keys, instead of 400 InvalidURI as AWS S3 does.

Ozone already defines the correct error in S3ErrorTable.INVALID_URI but never uses it:
INVALID_URI("InvalidURI", "Couldn't parse the specified URI.", HTTP_BAD_REQUEST)

The key \xae\x8a- contains non-printable high bytes (\x80–\xff) that Ozone cannot store and that AWS treats as an invalid URI. S3G should reject it before OM lookup.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15958

How was this patch tested?

Add Unit and Integration Tests.

@Gargi-jais11 Gargi-jais11 added the s3 S3 Gateway label Jul 24, 2026
@Gargi-jais11
Gargi-jais11 marked this pull request as ready for review July 24, 2026 09:09

@ayushtkn ayushtkn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that this logic was likely added based on the AWS S3 docs mentioning 'Non-printable ASCII characters (128–255 decimal characters)' under the 'Characters to avoid' section.

However, there are two issues with enforcing it this way:
AWS S3 doesn't actually block these: The AWS docs list this under 'Characters to avoid' as a recommendation for clients to prevent application-side parsing issues.

AWS S3 itself still accepts valid UTF-8 characters that resolve to this range; it does not throw an InvalidURI error.

This blocks valid international characters: Because I believe keyPath is a Java String (UTF-16), checking c >= 0x80 && c <= 0xFF targets the Unicode Latin-1 Supplement block.
This will hard-reject perfectly valid, printable characters like é (U+00E9 / 233), ñ (U+00F1 / 241), or £ (U+00A3 / 163).

A user uploading a file named café.txt or piñata.jpg will get an unexpected InvalidURI error.

The Ceph test_object_read_unreadable test is likely sending raw, unencoded invalid bytes over the wire to trigger a parsing failure. We should rely on the \uFFFD check to catch malformed UTF-8, and if we want to block actual control characters, we should use Character.isISOControl(c) instead of blocking the entire 128-255 block.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html#object-key-guidelines

Or Am I missing somethign here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

s3 S3 Gateway

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants