From 41cd0e8f1248731af6969b6c121c8e677b4e7627 Mon Sep 17 00:00:00 2001 From: deodharaditi Date: Sun, 21 Jun 2026 16:25:18 -0400 Subject: [PATCH 1/2] docs: document authentication for private remote stores Add an "Authentication" subsection to the storage user guide explaining how to pass credentials (key/secret/token/endpoint_url) through storage_options for private S3 / MinIO buckets, and clarify that credentials are top-level keys, not client_kwargs entries (nesting them there triggers the cryptic aiohttp 'unexpected keyword argument secret' error). Adds a towncrier news fragment. Refs #2995 --- changes/2995.doc.md | 1 + docs/user-guide/storage.md | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) create mode 100644 changes/2995.doc.md diff --git a/changes/2995.doc.md b/changes/2995.doc.md new file mode 100644 index 0000000000..2655a7ae30 --- /dev/null +++ b/changes/2995.doc.md @@ -0,0 +1 @@ +Document how to authenticate to private remote stores (e.g. S3 / MinIO) by passing credentials through `storage_options`, and clarify that credentials such as `key` and `secret` are top-level keys rather than `client_kwargs` entries. diff --git a/docs/user-guide/storage.md b/docs/user-guide/storage.md index d5f840ab4b..626725962e 100644 --- a/docs/user-guide/storage.md +++ b/docs/user-guide/storage.md @@ -151,6 +151,43 @@ store = zarr.storage.FsspecStore(fs) print(store) ``` +#### Authentication + +The examples above use anonymous access (`anon=True`) to a public bucket. To read from or write +to a **private** bucket — for example a private AWS S3 bucket or a self-hosted, S3-compatible +service such as [MinIO](https://min.io/) — pass your credentials through `storage_options`. These +options are forwarded to the underlying fsspec filesystem (for S3 that is +[`s3fs`](https://s3fs.readthedocs.io/)), so the accepted keys are the ones that filesystem +understands; for S3 the most common are `key`, `secret`, `token`, and `endpoint_url`: + +```python +import zarr + +store = zarr.storage.FsspecStore.from_url( + 's3://my-private-bucket/path', + storage_options={ + 'key': '', + 'secret': '', + # 'token': '', # optional, for temporary credentials + # 'endpoint_url': 'https://my-minio.example.com', # for S3-compatible services like MinIO + }, +) +group = zarr.open_group(store=store, mode='r') +``` + +!!! note + Credentials are **top-level** keys in `storage_options`; they map directly to + [`s3fs.S3FileSystem`](https://s3fs.readthedocs.io/en/latest/api.html) arguments. Do not nest + them inside `client_kwargs` — that dictionary is forwarded to the underlying + `aiobotocore`/`aiohttp` client session, which does not accept `key`/`secret` and will raise an + error such as `TypeError: ClientSession._request() got an unexpected keyword argument 'secret'`. + Use `client_kwargs` only for additional client-session settings such as `region_name`. + +If you prefer not to pass credentials explicitly, `s3fs`/`botocore` will also automatically discover +the standard AWS credential sources — environment variables (`AWS_ACCESS_KEY_ID`, +`AWS_SECRET_ACCESS_KEY`), the shared credentials file (`~/.aws/credentials`), or an instance/role +profile — in which case the credential keys in `storage_options` can be omitted. + ### Memory Store From 9be3d5480431729ad87bd6fd0833494f1f5bffc1 Mon Sep 17 00:00:00 2001 From: deodharaditi Date: Mon, 29 Jun 2026 08:00:14 -0400 Subject: [PATCH 2/2] docs: annotate private-credentials example with exec=false --- docs/user-guide/storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/storage.md b/docs/user-guide/storage.md index 626725962e..1b32899640 100644 --- a/docs/user-guide/storage.md +++ b/docs/user-guide/storage.md @@ -160,7 +160,7 @@ options are forwarded to the underlying fsspec filesystem (for S3 that is [`s3fs`](https://s3fs.readthedocs.io/)), so the accepted keys are the ones that filesystem understands; for S3 the most common are `key`, `secret`, `token`, and `endpoint_url`: -```python +```python exec="false" reason="requires private S3 credentials not available in the docs build environment" import zarr store = zarr.storage.FsspecStore.from_url(