Skip to content

az sql db export fails in ubuntu version of PS but not windows #28359

@trumsey2005

Description

@trumsey2005

Describe the bug

Steps to reproduce
Below is a copy of my ADO pipeline to export sql dbs to a blob storage bacpac file on a regular basis... when it is run from an ubuntu builder (or from ubuntu using wsl + powershell) it will fail and say that the blob already exists. Since this is creating a blob based on a timestamped file name, this is not possible. When it is run from a windows ado builder or windows with powershell 7 it works as expected.

pool:
vmImage: 'windows-latest'

steps:

  • task: AzureCLI@2
    displayName: Create Sql Backups
    inputs:
    azureSubscription: "AzureDevelopment"
    scriptType: 'pscore'
    scriptLocation: 'inlineScript'
    inlineScript: |
    $currentDate = Get-Date
    $timestamp = $currentDate.ToUniversalTime().ToString("yyyyMMddHHmmss")
    $expiryDate = $currentDate.ToUniversalTime().AddHours(24).ToString("yyyy-MM-ddTHH:mmZ")
    $storageAccount = "empactk8sqlbackups4af"
    $containerExists = az storage container exists --account-name $storageAccount --name $(resourceGroup) --auth-mode login

    if ($containerExists.exists -eq $True) {
      Write-Output "The container exists"
    } else {
      Write-Output "Creating Container"
      az storage container create --account-name $storageAccount --name $(resourceGroup) --auth-mode login
    }
    
    $sasToken = az storage container generate-sas --account-name $storageAccount --name $(resourceGroup) --permissions lacrw --expiry $expiryDate --auth-mode login --as-user
    $databaseList = az sql db list --server $(resourceGroup) --resource-group $(resourceGroup) | ConvertFrom-Json
    
    foreach($database in ($databaseList | Where-Object {$_.name -ne "master"})){
      $blobName = "${timestamp}/$($database.name)_${timestamp}.bacpac"
      Write-Output $blobName
      $storageUri = "https://$storageAccount.blob.core.windows.net/$(resourceGroup)/${blobName}"
      Write-Output $storageUri
      az sql db export --name $database.name --resource-group $(resourceGroup) --server $(resourceGroup) --admin-user $(sql-username) --admin-password $(sql-password) --storage-key $sasToken --storage-key-type SharedAccessKey --storage-uri $storageUri --no-wait
    }
    

Related command

az sql db export

Errors

(BlobAlreadyExist) The ImportExport operation failed because the storage blob already exists.
Code: BlobAlreadyExist

Issue script & Debug output

DEBUG: cli.knack.cli: init debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7fc746c71440>, <function OutputProducer.on_global_arguments at 0x7fc746c09e40>, <function CLIQuery.on_global_arguments at 0x7fc746c43920>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Modules found from index for 'sql': ['azure.cli.command_modules.sql', 'azure.cli.command_modules.sqlvm']
DEBUG: cli.azure.cli.core: Loading command modules:
DEBUG: cli.azure.cli.core: Name Load Time Groups Commands
DEBUG: cli.azure.cli.core: sql 0.136 56 215
DEBUG: cli.azure.cli.core: sqlvm 0.067 4 20
DEBUG: cli.azure.cli.core: Total (2) 0.203 60 235
DEBUG: cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
DEBUG: cli.azure.cli.core: Loading extensions:
DEBUG: cli.azure.cli.core: Name Load Time Groups Commands Directory
DEBUG: cli.azure.cli.core: Total (0) 0.000 0 0
DEBUG: cli.azure.cli.core: Loaded 59 groups, 235 commands.
DEBUG: cli.azure.cli.core: Found a match in the command table.
DEBUG: cli.azure.cli.core: Raw command : sql db export
DEBUG: cli.azure.cli.core: Command table: sql db export
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7fc745b30fe0>]
DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/home/vsts/work/_temp/.azclitask/commands/2024-02-12.17-29-45.sql_db_export.1850.log'.
INFO: az_command_data_logger: command args: sql db export --name {} --resource-group {} --server {} --admin-user {} --admin-password {} --storage-key {} --storage-key-type {} --storage-uri {} --no-wait --debug
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x7fc745b474c0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x7fc745b9d1c0>, <function register_cache_arguments..add_cache_arguments at 0x7fc745b9d300>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7fc746c09ee0>, <function CLIQuery.handle_query_parameter at 0x7fc746c439c0>, <function register_ids_argument..parse_ids_arguments at 0x7fc745b9d260>]
DEBUG: cli.azure.cli.core.commands.client_factory: Getting management service client client_type=SqlManagementClient
DEBUG: cli.azure.cli.core.auth.persistence: build_persistence: location='/home/vsts/work/_temp/.azclitask/service_principal_entries.json', encrypt=False
DEBUG: cli.azure.cli.core.auth.persistence: build_persistence: location='/home/vsts/work/_temp/.azclitask/msal_token_cache.json', encrypt=False
DEBUG: cli.azure.cli.core.auth.binary_cache: load: /home/vsts/work/_temp/.azclitask/msal_http_cache.bin
DEBUG: urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
DEBUG: msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com/5b28166c-1b07-4c94-a114-4b907d61f0ad/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/5b28166c-1b07-4c94-a114-4b907d61f0ad/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/5b28166c-1b07-4c94-a114-4b907d61f0ad/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/5b28166c-1b07-4c94-a114-4b907d61f0ad/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/5b28166c-1b07-4c94-a114-4b907d61f0ad/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/5b28166c-1b07-4c94-a114-4b907d61f0ad/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/5b28166c-1b07-4c94-a114-4b907d61f0ad/kerberos', 'tenant_region_scope': 'NA', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'}
DEBUG: msal.application: Broker enabled? False
DEBUG: msal.application: Region to be used: None
DEBUG: cli.azure.cli.core.auth.credential_adaptor: CredentialAdaptor.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={}
DEBUG: cli.azure.cli.core.auth.msal_authentication: ServicePrincipalCredential.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={}
DEBUG: msal.application: Cache hit an AT
DEBUG: msal.telemetry: Generate or reuse correlation_id: 486191cc-6c81-4570-abe1-cc7d0c473d75
DEBUG: cli.azure.cli.core.sdk.policies: Request URL: 'https://management.azure.com/subscriptions/d2b947c2-fcab-4ee5-8d17-04752d378c9d/resourceGroups/k8-internal-dev/providers/Microsoft.Sql/servers/k8-internal-dev/databases/EMPACT.Thumbnails/export?api-version=2023-02-01-preview'
DEBUG: cli.azure.cli.core.sdk.policies: Request method: 'POST'
DEBUG: cli.azure.cli.core.sdk.policies: Request headers:
DEBUG: cli.azure.cli.core.sdk.policies: 'Content-Type': 'application/json'
DEBUG: cli.azure.cli.core.sdk.policies: 'Content-Length': '573'
DEBUG: cli.azure.cli.core.sdk.policies: 'Accept': 'application/json'
DEBUG: cli.azure.cli.core.sdk.policies: 'x-ms-client-request-id': '5013156b-c9cc-11ee-b3cb-55e0398569ae'
DEBUG: cli.azure.cli.core.sdk.policies: 'CommandName': 'sql db export'
DEBUG: cli.azure.cli.core.sdk.policies: 'ParameterSetName': '--name --resource-group --server --admin-user --admin-password --storage-key --storage-key-type --storage-uri --no-wait --debug'
DEBUG: cli.azure.cli.core.sdk.policies: 'User-Agent': 'AZURECLI/2.56.0 (DEB) azsdk-python-mgmt-sql/4.0.0b13 Python/3.11.5 (Linux-6.2.0-1019-azure-x86_64-with-glibc2.35) VSTS_c993a862-4fb8-4bbf-8a7b-2a74611c4f13_build_701_0'
DEBUG: cli.azure.cli.core.sdk.policies: 'Authorization': '*****'
DEBUG: cli.azure.cli.core.sdk.policies: Request body:
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): management.azure.com:443
DEBUG: urllib3.connectionpool: https://management.azure.com:443 "POST /subscriptions/d2b947c2-fcab-4ee5-8d17-04752d378c9d/resourceGroups/k8-internal-dev/providers/Microsoft.Sql/servers/k8-internal-dev/databases/EMPACT.Thumbnails/export?api-version=2023-02-01-preview HTTP/1.1" 400 124
DEBUG: cli.azure.cli.core.sdk.policies: Response status: 400
DEBUG: cli.azure.cli.core.sdk.policies: Response headers:
DEBUG: cli.azure.cli.core.sdk.policies: 'Cache-Control': 'no-cache'
DEBUG: cli.azure.cli.core.sdk.policies: 'Pragma': 'no-cache'
DEBUG: cli.azure.cli.core.sdk.policies: 'Content-Length': '124'
DEBUG: cli.azure.cli.core.sdk.policies: 'Content-Type': 'application/json; charset=utf-8'
DEBUG: cli.azure.cli.core.sdk.policies: 'Expires': '-1'
DEBUG: cli.azure.cli.core.sdk.policies: 'x-ms-request-id': '64159c90-056b-44cb-9209-b94164f350d6'

Expected behavior

az command run in powershell should work the same across operating systems

Environment Summary

Working:
azure-cli 2.56.0 *

core 2.56.0 *
telemetry 1.1.0

Extensions:
k8s-extension 1.6.0

Dependencies:
msal 1.24.0b2
azure-mgmt-resource 23.1.0b2

Python location 'C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\Tyler.azure\cliextensions'

Python (Windows) 3.11.5 (tags/v3.11.5:cce6ba9, Aug 24 2023, 14:21:31) [MSC v.1936 32 bit (Intel)]

Legal docs and information: aka.ms/AzureCliLegal

Not-Working:
azure-cli 2.57.0

core 2.57.0
telemetry 1.1.0

Extensions:
k8s-extension 1.6.0

Dependencies:
msal 1.26.0
azure-mgmt-resource 23.1.0b2

Python location '/opt/az/bin/python3'
Extensions directory '/home/tyler/.azure/cliextensions'

Python (Linux) 3.11.7 (main, Jan 31 2024, 05:29:49) [GCC 11.4.0]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Auto-AssignAuto assign by botSQLaz sqlService AttentionThis issue is responsible by Azure service team.act-codegen-extensibility-squadcustomer-reportedIssues that are reported by GitHub users external to the Azure organization.questionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions