Add Azure Blob Storage Gen2 (ADLS HNS) support to BlobFileStore by Skrypt · Pull Request #19014 · OrchardCMS/OrchardCore

Skrypt · 2026-03-16T17:34:16Z

Make BlobFileStore adaptive: it now auto-detects whether the storage account has Hierarchical Namespace (HNS) enabled and uses native DataLake APIs for atomic moves, real directories, and efficient listing when available. Falls back to standard flat-namespace blob operations otherwise. HNS detection can be overridden via configuration.

Add IFileStoreCapabilities interface and FileStoreCapabilities impl
Add GetFilesAsync, GetDirectoriesAsync, Capabilities to IFileStore
Add UseHierarchicalNamespace option to BlobStorageOptions
Add Azure.Storage.Files.DataLake package dependency
Add MediaCreatedFileAsync and MediaCopiedFileAsync event hooks

Storage Capabilities in the Vue 3 Media App

The Vue app fetches capabilities — but only for display

The frontend calls a dedicated GetCapabilities API endpoint exposed by MediaApiController:

[HttpGet]
[Route("GetCapabilities")]
public async Task<ActionResult<FileStoreCapabilitiesDto>> GetCapabilities()
{
        return Ok(new FileStoreCapabilitiesDto
        {
            StorageName = _mediaFileStore.StorageName,
            HasHierarchicalNamespace = _mediaFileStore.Capabilities.HasHierarchicalNamespace,
            SupportsAtomicMove = _mediaFileStore.Capabilities.SupportsAtomicMove,
        });
}

The Vue app retrieves this via FileDataService.getCapabilities() and a useStoragePopover composable shows it in a UI popover (storage provider name, quota, capabilities). However, it does not use the flags to conditionally change UX behavior — there is no "if supportsAtomicMove then show rename, else show copy" logic.

All operations go through separate, unified API endpoints that adapt server-side

The Vue app calls the same endpoints regardless of storage backend:

Vue Operation	API Endpoint	Server-side adaptation
Rename file	`POST /api/media/MoveMedia`	Backend calls atomic `RenameAsync` or copy-then-delete
Move file	`POST /api/media/MoveMedia`	Same — rename is treated as a move
Copy file	`POST /api/media/CopyMedia`	Backend uses blob copy or `File.Copy`
Bulk move	`POST /api/media/MoveMediaList`	Backend loops, each adapts per storage
Create folder	`POST /api/media/CreateFolder`	Backend creates real dir or marker file
Delete folder	`POST /api/media/DeleteFolder`	Backend does recursive delete or enumerate-and-delete
Delete file	`POST /api/media/DeleteMedia`	Unified
Bulk delete	`POST /api/media/DeleteMediaList`	Unified

Conclusion

The Vue app uses a single set of API endpoints — it does not branch its UX based on capabilities. All the adaptation happens server-side in DefaultMediaFileStore, which delegates to the underlying IFileStore implementation. The capabilities are surfaced to the user purely as informational (shown in a storage popover), not to drive conditional frontend behavior.

This is the right pattern — the frontend stays simple and the storage abstraction handles the differences transparently.

TODO

Implement a set of integration tests against BlobFileStore (Gen 1 and Gen2) with Azurite
Azurite needs to support ADSL; see: ADSL Gen 2 Azure/Azurite#2635

Make BlobFileStore adaptive: it now auto-detects whether the storage account has Hierarchical Namespace (HNS) enabled and uses native DataLake APIs for atomic moves, real directories, and efficient listing when available. Falls back to standard flat-namespace blob operations otherwise. HNS detection can be overridden via configuration. - Add IFileStoreCapabilities interface and FileStoreCapabilities impl - Add GetFilesAsync, GetDirectoriesAsync, Capabilities to IFileStore - Add UseHierarchicalNamespace option to BlobStorageOptions - Add Azure.Storage.Files.DataLake package dependency - Add MediaCreatedFileAsync and MediaCopiedFileAsync event hooks - Wire up EnsureCapabilitiesAsync at startup in Media.Azure module

Add integration tests that run against Azurite to verify BlobFileStore behavior in both flat-namespace (Gen1) and hierarchical-namespace (Gen2) modes. Tests cover capabilities detection, file CRUD, directory operations, move/copy, and directory content listing. Gen2 tests that use DataLake APIs will fail until Azurite supports the DFS endpoint (--dfsPort flag). This is intentional to track when a newer Azurite version enables full Gen2 testing. - Add abstract BlobFileStoreTestsBase with 26 test methods - Add BlobFileStoreGen1Tests and BlobFileStoreGen2Tests subclasses - Add AzuriteFactAttribute to skip tests when Azurite is unavailable - Add OrchardCore.FileStorage.AzureBlob reference to test project - Start Azurite in main_ci and pr_ci workflows on ubuntu runners

Add DfsEndpoint option to BlobStorageOptions for local emulators where the DFS endpoint runs on a separate port. Parse storage credentials from the connection string to construct the DataLakeServiceClient when an explicit DFS endpoint is configured. Fix Gen2 code paths to handle 404 RequestFailedException from DataLakePathClient.ExistsAsync, which can throw instead of returning false when the path does not exist.

Reverted both workflows back to docker run steps (instead of services) since that lets us pass --skipApiVersionCheck directly. The command now uses your image with the correct flags:

Piedone · 2026-03-21T02:32:01Z

I'll try to review this over the weekend.

Piedone

Will #14256 use the new APIs for better UX?

Piedone · 2026-03-24T23:17:54Z

    <ProjectReference Include="..\..\src\OrchardCore.Modules\OrchardCore.DynamicCache\OrchardCore.DynamicCache.csproj" />
    <ProjectReference Include="..\..\src\OrchardCore\OrchardCore.Apis.GraphQL.Abstractions\OrchardCore.Apis.GraphQL.Abstractions.csproj" />
    <ProjectReference Include="..\..\src\OrchardCore\OrchardCore.Apis.GraphQL.Client\OrchardCore.Apis.GraphQL.Client.csproj" />
+    <ProjectReference Include="..\..\src\OrchardCore\OrchardCore.FileStorage.AzureBlob\OrchardCore.FileStorage.AzureBlob.csproj" />


Same concerns for this being in OrchardCore.Tests as under #19015 (comment).

Piedone · 2026-03-24T23:34:51Z

+    }
+
+    [AzuriteFact]
+    public async Task CreateDirectory_AlreadyExists_ReturnsFalse()


This and DeleteDirectory_WithNestedSubdirectories_DeletesAll fails for me in the Gen1 tests with an actual storage account in Azure.

That would mean that Azurite is not emulating properly for Gen 1. I'll take a look.

We should not test this with Gen 1 storage simply.

Piedone · 2026-03-24T23:56:14Z

    {
        options.RemoveContainer = rawOptions.RemoveContainer;
        options.RemoveFilesFromBasePath = rawOptions.RemoveFilesFromBasePath;
+        options.UseHierarchicalNamespace = rawOptions.UseHierarchicalNamespace;


Add a sample for this into the Web project's appsettings and here, with an explanation on when would you set what (my understanding is that usually, you wouldn't need to set this): https://docs.orchardcore.net/en/latest/reference/modules/Media.Azure/#configuration

It's an escape hatch for two edge cases where auto-detection doesn't work:

SAS tokens — GetAccountInfoAsync() requires storage account-level permissions. A SAS token scoped to a container won't have them, so the detection call fails and falls back to flat-namespace. If the account is actually Gen2/HNS, the user needs to set UseHierarchicalNamespace: true to force the correct behavior.

Local emulators (Azurite) — Azurite doesn't implement the GetAccountInfo endpoint, so detection always fails. Combined with the separate DfsEndpoint option, this lets dev environments opt-in to HNS simulation without a real Azure account.

Without the override, there'd be no way to use HNS-dependent features (atomic moves, real directory ops) in either of those scenarios.

OK then, please document this.

Piedone · 2026-03-25T00:02:17Z

-                var fileStore = new BlobFileStore(blobStorageOptions, clock, contentTypeProvider);
+                var blobLogger = serviceProvider.GetRequiredService<ILogger<BlobFileStore>>();
+                var fileStore = new BlobFileStore(blobStorageOptions, clock, contentTypeProvider, blobLogger);
+                fileStore.EnsureCapabilitiesAsync().GetAwaiter().GetResult();


Sync-over-async is not the best, can this be done better?

Piedone · 2026-03-25T00:19:24Z


    private readonly string _basePrefix;

+    public IFileStoreCapabilities Capabilities => _capabilities ?? FileStoreCapabilities.Default;


Do this in the other file stores too, at least to set StorageProvider. And FileSystemStore has HasHierarchicalNamespace and SupportsAtomicMove too.

No. AwsFileStore and FileSystemStore don't override Capabilities at all — they inherit the interface default (FileStoreCapabilities.Default), which returns all-false. That means:

FileSystemStore — the local filesystem doesn't need HNS detection; it just uses regular directory semantics, so Default (flat, no atomic move) is correct.
AwsFileStore — S3 is flat-namespace only (no HNS concept), so Default is also correct.
The EnsureCapabilitiesAsync pattern is specific to BlobFileStore because Azure Blob Storage has two distinct modes (Gen1 flat vs Gen2 HNS/ADLS) that need to be detected at runtime by calling the Azure API. The other stores don't have that ambiguity — their capabilities are statically known.

Now you removed StorageProvider so that's not needed but I don't see why you wouldn't want to set the other two for FileSystemStore. You added this to describe capabilities of the provider to consumers, so they can e.g. optimize the UX, right? (In addition to this being not just nice to have, but necessary to distinguish between Gen1 and 2 Blob Storage). The local file system definitely has both a hierarchical namespace and supports atomic moves (two things that Gen2 mimics from a standard file system), so it shouldn't use FileStoreCapabilities.Default.

Now, if you actually meant IFileStoreCapabilities to be specific to BlobFileStore, then it shouldn't be a type in the general OrchardCore.FileStorage, nor should it be exposed via the general IFileStore interface.

Also, I'd ask you to please check AI answers instead of copying them to replies. I'm conversing with you, if I want to ask Copilot, I'll do that directly.

I will take a look at if we can do this better. There is an assumption you are doing that the UX will be different on the Media Gallery that you should not do. I can explain in details by showing you on a call because it is not changing the way the new Media Gallery will behave for different reasons. I see that you are trying to see the whole things altogether here.

I will rethink how this part can be done here if we can make it more self explanatory through code. Though the code should only affect the server side behaviors.

As for the Copilot/AI comment, I'm simply just trying to save time. The answer was accurate as this is how the current server side code works. If we want to support atomic move on local filestorage then we need to change the way the server side code works and thus changing another provider. Then this would be a different PR.

Thanks!

Based on the documentation on IFileStoreCapabilities about how these properties are meant to be used, both look like they apply to FileSystemStore.

/// Gets a value indicating whether the store has a true hierarchical namespace /// (i.e. directories are first-class objects, not simulated via path prefixes).

Directories are not simulated via path prefixes but are first-class objects e.g. in NTFS and Ext4.

/// Gets a value indicating whether the store supports atomic (server-side) move / rename /// that does not require a copy-then-delete round-trip.

Same, you can tell a file system to move or rename a file and it won't be a copy.

Piedone · 2026-03-25T00:20:32Z

+    /// <summary>
+    /// Gets the capabilities supported by this file store.
+    /// </summary>
+    IFileStoreCapabilities Capabilities => FileStoreCapabilities.Default;


I don't think this should have a default. Every provider should set at least StorageProvider.

I simplified this. No need to set a StorageProvider but a StorageName now.

Piedone · 2026-03-25T00:20:52Z

+    /// <summary>
+    /// Gets a human-readable name identifying the storage provider (e.g. "Local", "Azure Blob").
+    /// </summary>
+    string StorageProvider => "Unknown";


Where do we need this? It's not currently used.

agreed, removed

…dia startup Move BlobFileStore.EnsureCapabilitiesAsync() from the DI singleton factory (where it blocked via GetAwaiter().GetResult()) into MediaBlobContainerTenantEvents.ActivatingAsync(), which already runs asynchronously at tenant startup. BlobFileStore is now registered as its own singleton so it can be injected. Also remove the unused StorageProvider member from IFileStoreCapabilities and FileStoreCapabilities, and remove the default implementation of IFileStore.Capabilities so every provider must declare capabilities explicitly. FileSystemStore and AwsFileStore updated accordingly.

…s cleanup

Skrypt · 2026-03-29T16:16:34Z

    private readonly ILogger<FileSystemStore> _logger;
    private readonly string _fileSystemPath;

+    public IFileStoreCapabilities Capabilities { get; } = new FileStoreCapabilities(hasHierarchicalNamespace: false, supportsAtomicMove: false);


I'm changing this configuration to report that the local FileStorage uses HierarchicalNamespace and supports atomic move. It technically should not break any custom implementations.

…lesystem behavior FileSystemStore was configured with hasHierarchicalNamespace: false and supportsAtomicMove: false, which does not match its actual implementation. The local filesystem uses real Directory.* APIs for first-class directory operations and File.Move() which maps to an atomic OS rename syscall. Updated both flags to true to align with the implementation and be consistent with how BlobFileStore Gen2 reports the same native behaviors.

…tification Add a StorageName default interface member to IFileStore that returns a human-readable name for the underlying storage backend. Implementations: FileSystemStore ("Local"), BlobFileStore ("Azure Blob (Gen1)"/"Azure Blob (Gen2)" based on detected HNS), AwsFileStore ("Amazon S3"). DefaultMediaFileStore delegates to the inner store. This enables the media UI to display which storage provider is active at runtime, including distinguishing Azure Gen1 from Gen2.

…tegration project Extract BlobFileStore Gen1/Gen2 tests from OrchardCore.Tests into a new OrchardCore.Tests.Integration project. This separates integration tests that require external services (Azurite) from unit tests, avoiding unnecessary Docker container startup and extra execution time on every PR commit. The new project is built and run by a dedicated integration_tests.yml workflow that only triggers on review submission, push to main/release, or manual dispatch — matching the same pattern as functional_all_db.yml. Removed Azurite setup from pr_ci.yml and main_ci.yml.

github-actions · 2026-04-02T06:30:33Z

This pull request has merge conflicts. Please resolve those before requesting a review.

Skrypt · 2026-04-02T06:31:08Z

@Piedone Same here, moved everything to a new project and also new GH workflow.

github-actions · 2026-04-07T21:33:17Z

This pull request has merge conflicts. Please resolve those before requesting a review.

# Conflicts: # .github/workflows/integration_tests.yml # test/OrchardCore.Tests.Integration/OrchardCore.Tests.Integration.csproj

hishamco · 2026-04-17T12:18:31Z

As we discussed in the latest steering meeting to utilize TestContainers, this closed in favor of #19162

Skrypt marked this pull request as draft March 16, 2026 17:54

Skrypt added 2 commits March 16, 2026 15:07

Skrypt mentioned this pull request Mar 17, 2026

ADSL Gen 2 Azure/Azurite#2635

Open

Skrypt added 3 commits March 18, 2026 04:11

Update unit tests after updating Azurite

7be25b8

Update GH workflows with temp Azurite package repo

71ced65

skipApiVersionCheck

256835f

Reverted both workflows back to docker run steps (instead of services) since that lets us pass --skipApiVersionCheck directly. The command now uses your image with the correct flags:

Skrypt requested a review from Piedone March 19, 2026 20:46

Merge branch 'main' into skrypt/media-azure-gen2

121557d

Piedone requested changes Mar 25, 2026

View reviewed changes

Skrypt added 3 commits March 28, 2026 01:34

Merge branch 'main' into skrypt/media-azure-gen2

03c0980

fix: Remove StorageProvider from test broken by IFileStoreCapabilitie…

7d16f4f

…s cleanup

Skrypt commented Mar 29, 2026

View reviewed changes

Skrypt added 5 commits March 29, 2026 12:19

fix build

aa477e6

add documentation on added configuration

56b294a

github-actions bot added the merge conflict label Apr 2, 2026

Merge branch 'main' into skrypt/media-azure-gen2

7c01e5c

github-actions bot added merge conflict and removed merge conflict labels Apr 2, 2026

Merge branch 'main' into skrypt/media-azure-gen2

3a44b79

# Conflicts: # .github/workflows/integration_tests.yml # test/OrchardCore.Tests.Integration/OrchardCore.Tests.Integration.csproj

github-actions bot removed the merge conflict label Apr 14, 2026

hishamco mentioned this pull request Apr 15, 2026

Use TestContainers for AWS Local Stack #19157

Open

hishamco closed this Apr 17, 2026

Skrypt reopened this Apr 17, 2026


		private readonly string _basePrefix;

		public IFileStoreCapabilities Capabilities => _capabilities ?? FileStoreCapabilities.Default;

Conversation

Skrypt commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Storage Capabilities in the Vue 3 Media App

The Vue app fetches capabilities — but only for display

All operations go through separate, unified API endpoints that adapt server-side

Conclusion

TODO

Uh oh!

Piedone commented Mar 21, 2026

Uh oh!

Piedone left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Skrypt Mar 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Skrypt Mar 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Skrypt Mar 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

Skrypt commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 7, 2026

Uh oh!

hishamco commented Apr 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Skrypt commented Mar 16, 2026 •

edited

Loading

Skrypt Mar 29, 2026 •

edited

Loading

Skrypt Mar 28, 2026 •

edited

Loading

Skrypt Mar 29, 2026 •

edited

Loading