Skip to content

feat(gcp): Add Hierarchical Namespace (HNS) support for GCS bucket #3996

Open
varunarya002 wants to merge 2 commits intoapache:mainfrom
varunarya002:gcp_hns_bucket
Open

feat(gcp): Add Hierarchical Namespace (HNS) support for GCS bucket #3996
varunarya002 wants to merge 2 commits intoapache:mainfrom
varunarya002:gcp_hns_bucket

Conversation

@varunarya002
Copy link

@varunarya002 varunarya002 commented Mar 13, 2026

Issue:
GCS buckets with HNS enabled require explicit folder creation and storage.folderAdmin permissions for managed folder operations. Without this, Iceberg table creation and Spark ingestion fail on HNS buckets.

Changes:

  • Add hierarchicalNamespace boolean field to GcpStorageConfigInfo (OpenAPI spec)
  • Add @nullable Boolean isHierarchicalNamespace() to GcpStorageConfigurationInfo
  • Generate storage.folderAdmin access boundary rule scoped to managedFolders/{path} for write locations when HNS is enabled
  • Plumb HNS flag through CatalogEntity model<->entity conversion
  • Add backward-compatible 3-param overload defaulting HNS to false

Tests:

  • HNS enabled single bucket (read + write + folder rules)
  • HNS with multiple buckets and partial writes
  • HNS without writes (no folder rules)
  • HNS with separate metadata and data buckets
  • HNS same bucket with separate metadata/data paths
  • Non-HNS vs HNS rule count comparison
  • Config serialization round-trip
  • CatalogEntity round-trip with HNS flag

Checklist

  • 🛡️ Don't disclose security issues! (contact security@apache.org)
  • 🔗 Clearly explained why the changes are needed, or linked related issues: Fixes #
  • 🧪 Added/updated tests with good coverage, or manually tested (and explained how)
  • 💡 Added comments for complex logic
  • 🧾 Updated CHANGELOG.md (if needed)
  • 📚 Updated documentation in site/content/in-dev/unreleased (if needed)

…nding

Co-Authored-By: claude-flow <ruv@ruv.net>
@varunarya002 varunarya002 changed the title feat(gcp): Add Hierarchical Namespace (HNS) support for GCS bucket cr… feat(gcp): Add Hierarchical Namespace (HNS) support for GCS bucket Mar 13, 2026
Copy link
Contributor

@dimas-b dimas-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution, @varunarya002 !

Since this PR affects REST API parameters, please open a corresponding discussion on the dev ML, which is a usual practice in Polaris.

(this is not a complete review from my side 😅 ... just noting a couple of points for a start)

gcsServiceAccount:
type: string
description: a Google cloud storage service account
hierarchicalNamespace:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: a similar field in Azure is called simply hierarchical

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants