From b668a9db7385da12b7e2ef21118fd02cfaaa9226 Mon Sep 17 00:00:00 2001 From: "dmitry.kuleshov" Date: Wed, 20 May 2026 21:05:40 +0300 Subject: [PATCH 1/2] docs(server): document local Azurite setup and fix seed-script auth (#435) server/README.md now has a complete local-dev path against Azurite: - Docker / Podman invocation including --skipApiVersionCheck so Azurite accepts newer x-ms-version headers from current Azure SDKs. - A working appsettings.Development.json BlobStorage snippet using the short UseDevelopmentStorage=true form, plus a warning against the long ;BlobEndpoint=.../devstoreaccount1 form that some SDK versions mis-sign against (Azurite then returns 403 AuthorizationFailure, which Azure CLI surfaces as the misleading "may be blocked by network rules" error). - Pointer to scripts/Seed-AzuriteContainers.ps1 for one-shot container creation (answers, conversation-references). - A standalone callout for Auth:SeedAdministrators, including dev@localhost so DevelopmentAuthMiddleware-injected requests pass the admin check, plus the az storage blob delete command to reset the seed when the list is changed. server/scripts/Seed-AzuriteContainers.ps1 detects Azurite by the well-known account key (a public constant in Azurite's source; real storage accounts generate random keys) and substitutes the short UseDevelopmentStorage=true connection string plus --auth-mode key before invoking az storage container create. This avoids the canonical-resource signing mismatch that fails on the long connection-string form. Non-Azurite connection strings pass through unchanged. Closes #435 Co-Authored-By: Claude Opus 4.7 (1M context) --- server/README.md | 82 +++++++++++++++++++++-- server/scripts/Seed-AzuriteContainers.ps1 | 34 ++++++++-- 2 files changed, 108 insertions(+), 8 deletions(-) diff --git a/server/README.md b/server/README.md index f4ebf014..89e63e87 100644 --- a/server/README.md +++ b/server/README.md @@ -44,6 +44,7 @@ The server exposes `POST /api/notify` and delivers to any enabled channels. Team - Azure CLI (`az`) - Terraform >= 1.6 - Azure subscription (APPS_EU_TEST) +- Docker or Podman (local dev only - runs the Azurite blob storage emulator) ## Setup @@ -72,10 +73,32 @@ This creates: Resource Group, Entra ID App, App Service Plan, App Service, Bot S After `terraform apply`, create `src/Dotbot.Server/appsettings.Development.json` (gitignored). Start from `appsettings.Example.json` (Serilog + minimal `Auth` block) and add the keys below — these are required at startup and the server will throw `InvalidOperationException` without them: - `MicrosoftAppTenantId`, `MicrosoftAppId`, `MicrosoftAppPassword` — top-level keys read by the Agents SDK (`Program.cs` / Bot Service auth) -- `BlobStorage:AccountUri` **or** `BlobStorage:ConnectionString` — one is required (`Program.cs:92`) +- `BlobStorage:AccountUri` **or** `BlobStorage:ConnectionString` - one is required (`Program.cs:92`). For local dev, set `ConnectionString` to the Azurite emulator (see step 3); production uses `AccountUri` with managed identity. - `ApiSecurity:ApiKey` — shared secret for the `X-Api-Key` header (`ApiKeyMiddleware.cs`) - `TokenValidation:{Audiences,TenantId}` + `Connections:ServiceConnection:Settings:{ClientId,ClientSecret,TenantId}` — mirror the structure in the committed `appsettings.json` (use `{{MicrosoftAppId}}`-style tokens there, or paste the real values into `appsettings.Development.json`) - `DeliveryChannels:{Email,Jira,Slack}` — only needed if you enable that channel; schema lives in `Models/DeliveryChannelSettings.cs` +- `Auth:SeedAdministrators` — see the admin-seed callout below; **must include `"dev@localhost"` for local login to work**. + +> **Local admin seed (required for dashboard login).** +> In `Development`, the `DevelopmentAuthMiddleware` injects `dev@localhost` as the current user. The server's `AdministratorService` checks that email against the `Auth:SeedAdministrators` list, which is persisted to the `answers/dev/config/administrators.json` blob on first startup. If `dev@localhost` is not in that list, every dashboard request fails with `Access denied: dev@localhost is not an administrator`. +> +> In `appsettings.Development.json`: +> +> ```json +> "Auth": { +> "SeedAdministrators": [ "dev@localhost" ] +> } +> ``` +> +> The blob is only seeded when absent. If you change the list after the blob has been written, delete the blob and restart the server so the seed re-runs: +> +> ```powershell +> az storage blob delete ` +> --container-name answers ` +> --name dev/config/administrators.json ` +> --connection-string "UseDevelopmentStorage=true" ` +> --auth-mode key +> ``` Get the bot credentials from Terraform: @@ -86,7 +109,58 @@ terraform output -raw azuread_app_password User-secrets (`dotnet user-secrets set "ApiSecurity:ApiKey" "…"`) work too and keep secrets off disk. -### 3. Run Locally +### 3. Start Azurite (local blob storage) + +Local dev uses [Azurite](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite), the official Microsoft emulator for Azure Blob / Queue / Table storage. The server reads from two blob containers on startup (`answers`, `conversation-references`) and does not call `CreateIfNotExists`, so a fresh Azurite returns `404 ContainerNotFound` until the containers are seeded once. + +Default Azurite ports: `10000` (blob), `10001` (queue), `10002` (table). Use the short emulator connection string in `appsettings.Development.json`: + +```json +"BlobStorage": { + "AccountUri": "", + "ConnectionString": "UseDevelopmentStorage=true", + "Backend": "AzureBlob", + "MaxAttachmentSizeMb": 15 +} +``` + +`UseDevelopmentStorage=true` is the standard Azurite shortcut understood by both the .NET `BlobServiceClient` and `az storage`. Avoid the long `DefaultEndpointsProtocol=http;...;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;...` form - the trailing `/devstoreaccount1` path in `BlobEndpoint` makes some SDK versions compute a different canonical resource than Azurite expects, producing `AuthorizationFailure` (403) that surfaces as the misleading "request may be blocked by network rules" message from Azure CLI. + +**Run Azurite in a container.** Pick the runtime you have installed. + +`--skipApiVersionCheck` is included so Azurite accepts newer `x-ms-version` headers from current Azure SDKs / Azure CLI. Without it, requests fail with `The API version YYYY-MM-DD is not supported by Azurite` whenever the SDK is newer than the Azurite image. + +Docker: + +```powershell +docker run -d --name azurite ` + -p 10000:10000 -p 10001:10001 -p 10002:10002 ` + -v azurite-data:/data ` + mcr.microsoft.com/azure-storage/azurite ` + azurite --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 --location /data --skipApiVersionCheck +``` + +Podman: + +```powershell +podman run -d --name azurite ` + -p 10000:10000 -p 10001:10001 -p 10002:10002 ` + -v azurite-data:/data ` + mcr.microsoft.com/azure-storage/azurite ` + azurite --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 --location /data --skipApiVersionCheck +``` + +The named volume (`azurite-data`) keeps blobs across container restarts. Subsequent runs: `docker start azurite` / `podman start azurite`. + +**Seed the containers.** Run once per fresh Azurite data volume: + +```powershell +.\scripts\Seed-AzuriteContainers.ps1 +``` + +`Seed-AzuriteContainers.ps1` reads `BlobStorage:ConnectionString` from `appsettings.Development.json`, pings the blob endpoint, and creates both `answers` and `conversation-references` containers via `az storage container create` (idempotent - safe to re-run). + +### 4. Run Locally ```powershell dotnet run --project src/Dotbot.Server @@ -94,13 +168,13 @@ dotnet run --project src/Dotbot.Server Use [dev tunnels](https://learn.microsoft.com/en-us/azure/developer/dev-tunnels/) or ngrok to expose `http://localhost:5048` to the internet, then update the Bot Service messaging endpoint. -### 4. Deploy to Azure +### 5. Deploy to Azure ```powershell .\scripts\Deploy.ps1 ``` -### 5. Test +### 6. Test - Open Teams → chat with "Dotbot" → send any message → receive question card → pick answer - Proactive: `.\Send-DotbotQuestion.ps1 -User -Question "Pick one" -Options @(@{ key='A'; label='Option A' }, @{ key='B'; label='Option B' })` (see `SampleQuestions.json` for full payloads) diff --git a/server/scripts/Seed-AzuriteContainers.ps1 b/server/scripts/Seed-AzuriteContainers.ps1 index 6dcb9ef3..3a6c77c3 100644 --- a/server/scripts/Seed-AzuriteContainers.ps1 +++ b/server/scripts/Seed-AzuriteContainers.ps1 @@ -72,6 +72,27 @@ if ($blobEndpoint) { } } +# --- pick auth flags --------------------------------------------------------- +# When the connection string targets Azurite, Azure CLI 2.85+ mis-signs requests +# for the full DefaultEndpointsProtocol=...;AccountName=...;AccountKey=...;BlobEndpoint=...; +# form and returns "AuthorizationFailure" (surfaced as the misleading +# "request may be blocked by network rules" message). The short +# "UseDevelopmentStorage=true" form uses Azure CLI's built-in Azurite preset +# which signs correctly. Substitute it transparently when we detect Azurite. +# +# Detection uses the well-known Azurite account key (a public constant baked +# into Azurite's source). Real Azure storage accounts generate random 512-bit +# keys at creation time, so this value never appears outside the emulator. +$AzuriteWellKnownKey = 'Eby8vdM02xNOcqFlqUwJPLlmEu9Fyuwz2LNgaXcHMRvXv3lGbPjhImDs4Wqg6zY/JxGgZANyEDOmXpKXHVRjkw==' + +if ($connectionString -like "*$AzuriteWellKnownKey*" -or $connectionString -match '^\s*UseDevelopmentStorage\s*=\s*true\s*;?\s*$') { + $effectiveConnectionString = 'UseDevelopmentStorage=true' + $authMode = 'key' +} else { + $effectiveConnectionString = $connectionString + $authMode = $null +} + # --- create containers -------------------------------------------------------- $containers = @('answers', 'conversation-references') @@ -79,10 +100,15 @@ foreach ($name in $containers) { Write-Host "Ensuring container '$name' ... " -NoNewline -ForegroundColor Gray # Note: omitting --fail-on-exist makes the command idempotent. # Passing it (even with no value) flips it on and would error when the container exists. - $result = az storage container create ` - --name $name ` - --connection-string $connectionString ` - --output json 2>&1 + $azArgs = @( + 'storage', 'container', 'create', + '--name', $name, + '--connection-string', $effectiveConnectionString, + '--output', 'json' + ) + if ($authMode) { $azArgs += @('--auth-mode', $authMode) } + + $result = az @azArgs 2>&1 if ($LASTEXITCODE -ne 0) { Write-Host "FAIL" -ForegroundColor Red From 7fdb2b182619a7142db87817fffe9d86907486c6 Mon Sep 17 00:00:00 2001 From: "dmitry.kuleshov" Date: Wed, 20 May 2026 21:17:06 +0300 Subject: [PATCH 2/2] docs(server): address Copilot review on Azurite local-dev PR server/scripts/Seed-AzuriteContainers.ps1 - Move the "doesn't look like Azurite" warning behind the new detection (well-known key OR UseDevelopmentStorage=true) so it no longer fires on the recommended short connection string. - When the conn string is the short form and no BlobEndpoint= is parseable, fall back to Azurite's default endpoint (http://127.0.0.1:10000/devstoreaccount1) so the reachability ping still runs in both connection-string styles. server/README.md - Split Prerequisites into Local development vs Production deploy so Terraform and an Azure subscription are no longer presented as unconditional requirements. - Bind the Docker and Podman -p mappings to 127.0.0.1 so the Azurite emulator is not exposed on the host LAN, and explain why in the prose above the snippets. Co-Authored-By: Claude Opus 4.7 (1M context) --- server/README.md | 16 ++++--- server/scripts/Seed-AzuriteContainers.ps1 | 54 +++++++++++++---------- 2 files changed, 41 insertions(+), 29 deletions(-) diff --git a/server/README.md b/server/README.md index 89e63e87..c9102983 100644 --- a/server/README.md +++ b/server/README.md @@ -40,11 +40,17 @@ The server exposes `POST /api/notify` and delivers to any enabled channels. Team ## Prerequisites +**Local development** (run the server against Azurite, no real Azure resources): + - .NET 9 SDK -- Azure CLI (`az`) +- Azure CLI (`az`) - used by `Seed-AzuriteContainers.ps1` to create the local blob containers +- Docker or Podman - runs the Azurite blob storage emulator + +**Production deploy** (everything above, plus): + - Terraform >= 1.6 - Azure subscription (APPS_EU_TEST) -- Docker or Podman (local dev only - runs the Azurite blob storage emulator) +- Permission to create Entra ID app registrations and Bot Service resources ## Setup @@ -128,13 +134,13 @@ Default Azurite ports: `10000` (blob), `10001` (queue), `10002` (table). Use the **Run Azurite in a container.** Pick the runtime you have installed. -`--skipApiVersionCheck` is included so Azurite accepts newer `x-ms-version` headers from current Azure SDKs / Azure CLI. Without it, requests fail with `The API version YYYY-MM-DD is not supported by Azurite` whenever the SDK is newer than the Azurite image. +The published ports below are explicitly bound to `127.0.0.1` so the emulator (which serves unauthenticated traffic with a well-known key) is not exposed to the LAN. `--skipApiVersionCheck` is included so Azurite accepts newer `x-ms-version` headers from current Azure SDKs / Azure CLI - without it, requests fail with `The API version YYYY-MM-DD is not supported by Azurite` whenever the SDK is newer than the Azurite image. Docker: ```powershell docker run -d --name azurite ` - -p 10000:10000 -p 10001:10001 -p 10002:10002 ` + -p 127.0.0.1:10000:10000 -p 127.0.0.1:10001:10001 -p 127.0.0.1:10002:10002 ` -v azurite-data:/data ` mcr.microsoft.com/azure-storage/azurite ` azurite --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 --location /data --skipApiVersionCheck @@ -144,7 +150,7 @@ Podman: ```powershell podman run -d --name azurite ` - -p 10000:10000 -p 10001:10001 -p 10002:10002 ` + -p 127.0.0.1:10000:10000 -p 127.0.0.1:10001:10001 -p 127.0.0.1:10002:10002 ` -v azurite-data:/data ` mcr.microsoft.com/azure-storage/azurite ` azurite --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 --location /data --skipApiVersionCheck diff --git a/server/scripts/Seed-AzuriteContainers.ps1 b/server/scripts/Seed-AzuriteContainers.ps1 index 3a6c77c3..6804784b 100644 --- a/server/scripts/Seed-AzuriteContainers.ps1 +++ b/server/scripts/Seed-AzuriteContainers.ps1 @@ -50,29 +50,7 @@ if (-not $connectionString) { throw "BlobStorage.ConnectionString is empty in $AppSettingsPath. Use AccountUri+managed-identity in production; this script is for the local Azurite dev path only." } -if ($connectionString -notmatch 'devstoreaccount1') { - Write-Warning "Connection string does not look like Azurite (no 'devstoreaccount1' marker). Continuing anyway." -} - -# --- check Azurite is reachable ---------------------------------------------- -# Parse BlobEndpoint from the conn string for a quick sanity ping. -$blobEndpoint = ($connectionString -split ';' | - Where-Object { $_ -match '^BlobEndpoint=' } | - ForEach-Object { ($_ -split '=', 2)[1] }) | Select-Object -First 1 - -if ($blobEndpoint) { - try { - $null = Invoke-WebRequest -Uri $blobEndpoint -Method Get -TimeoutSec 3 -UseBasicParsing -ErrorAction Stop - } catch { - # Azurite returns 400 on bare-endpoint GET, which is fine — we just - # want to confirm something is listening there. - if (-not ($_.Exception.Response -and [int]$_.Exception.Response.StatusCode -ge 400)) { - throw "Azurite does not appear to be running at $blobEndpoint. Start it (e.g. 'azurite --silent') and re-run." - } - } -} - -# --- pick auth flags --------------------------------------------------------- +# --- detect Azurite, pick auth flags, derive blob endpoint ------------------- # When the connection string targets Azurite, Azure CLI 2.85+ mis-signs requests # for the full DefaultEndpointsProtocol=...;AccountName=...;AccountKey=...;BlobEndpoint=...; # form and returns "AuthorizationFailure" (surfaced as the misleading @@ -84,13 +62,41 @@ if ($blobEndpoint) { # into Azurite's source). Real Azure storage accounts generate random 512-bit # keys at creation time, so this value never appears outside the emulator. $AzuriteWellKnownKey = 'Eby8vdM02xNOcqFlqUwJPLlmEu9Fyuwz2LNgaXcHMRvXv3lGbPjhImDs4Wqg6zY/JxGgZANyEDOmXpKXHVRjkw==' +$AzuriteDefaultBlobEndpoint = 'http://127.0.0.1:10000/devstoreaccount1' -if ($connectionString -like "*$AzuriteWellKnownKey*" -or $connectionString -match '^\s*UseDevelopmentStorage\s*=\s*true\s*;?\s*$') { +$isShortForm = $connectionString -match '^\s*UseDevelopmentStorage\s*=\s*true\s*;?\s*$' +$isAzurite = $isShortForm -or ($connectionString -like "*$AzuriteWellKnownKey*") + +if ($isAzurite) { $effectiveConnectionString = 'UseDevelopmentStorage=true' $authMode = 'key' } else { $effectiveConnectionString = $connectionString $authMode = $null + Write-Warning "Connection string does not look like Azurite (well-known key not present and not 'UseDevelopmentStorage=true'). Continuing anyway." +} + +# Use the explicit BlobEndpoint when present; otherwise (e.g. the short form) +# fall back to Azurite's default. This lets the reachability ping work for +# either connection-string style. +$blobEndpoint = ($connectionString -split ';' | + Where-Object { $_ -match '^BlobEndpoint=' } | + ForEach-Object { ($_ -split '=', 2)[1] }) | Select-Object -First 1 +if (-not $blobEndpoint -and $isAzurite) { + $blobEndpoint = $AzuriteDefaultBlobEndpoint +} + +# --- check Azurite is reachable ---------------------------------------------- +if ($blobEndpoint) { + try { + $null = Invoke-WebRequest -Uri $blobEndpoint -Method Get -TimeoutSec 3 -UseBasicParsing -ErrorAction Stop + } catch { + # Azurite returns 400 on bare-endpoint GET, which is fine - we just + # want to confirm something is listening there. + if (-not ($_.Exception.Response -and [int]$_.Exception.Response.StatusCode -ge 400)) { + throw "Azurite does not appear to be running at $blobEndpoint. Start it (e.g. 'azurite --silent') and re-run." + } + } } # --- create containers --------------------------------------------------------