Support paginated folder metadata for files

[kryachkow]

Name and Version

latest

What is the problem this feature will solve?

files.get_metadata() and metadata.get() should support folder listing pagination parameters and expose nextToken for file metadata responses.

What is the feature you are proposing to solve the problem?

Motivation

StatGPT Backend lists files in DIAL storage folders and follows pagination using:

GET /v1/metadata/files/{bucket}/{path}?limit=100&token=...

The SDK currently exposes basic metadata lookup:

metadata = await async_client.files.get_metadata(url)

However, there is no public way to pass limit or token, and FileMetadata does not expose next_token.

Proposed API

For example:

metadata = await async_client.files.get_metadata(
    url,
    limit=100,
    token=next_token,
)

with:

metadata.next_token
metadata.items

Use Case

Consumers need to list all files in a folder without issuing raw HTTP requests outside the SDK.

Expected Behavior

• Folder metadata requests should accept limit and token.
• File folder metadata responses should expose next_token.
• Existing single-file metadata behavior should remain unchanged.

What alternatives have you considered?

No response