VM.osProfile.customData always returns null

[antimatter15]

Describe the bug

This is essentially a repost of #5866 with a little bit of additional information.

To Reproduce

Provision a new Azure VM running Ubuntu 20.04 LTS with a defined cloud-init customData.

Log into the newly provisioned VM to verify that the custom data has been run and has had an effect.

Run az vm show --resource-group groupName --name vmName and observe that customData is null.

Expected behavior

Running az vm show --resource-group groupName --name vmName should include an osProfile.customData field containing the base64 encoded custom data used to initialize the VM.

I can confirm that the custom data is run, and that the base64-encoded data is accessible from /var/lib/waagent/ovf-env.xml within the VM itself.

Environment Summary

Linux-5.11.0-1022-azure-x86_64-with-glibc2.29
Python 3.8.10
Shell: bash

azure-cli 2.0.81

Extensions:
azure-devops 0.17.0

Additional context

This is almost certainly a service related issue, as it is also visible from the Azure Resource Explorer (https://resources.azure.com/) and when issuing raw requests to the Azure Management API using cURL.

From the Azure Resource Explorer, when viewing the JSON contents of a VM, the osProfile.customData key is reliably missing, even though it is included in the documentation tab:

Specifies a base-64 encoded string of custom data. The base-64 encoded string is decoded to a binary array that is saved as a file on the Virtual Machine. The maximum length of the binary array is 65535 bytes.

For using cloud-init for your VM, see Using cloud-init to customize a Linux VM during creation

Likewise the osProfile.customData field is also missing when using cURL against the API directly:

curl 'https://management.azure.com/subscriptions/SUBSCRIPTION_ID/resourceGroups/GROUP_ID/providers/Microsoft.Compute/virtualMachines/VM_NAME?api-version=2021-03-01'

Apologies if this isn't the right place to report such an issue— if so I'd appreciate pointers on where it might be more appropriate.

Thanks and happy new year!

[yonzhan]

Compute

[harshavmb]

Hi @antimatter15 ,

Late to the party! I also hit with this issue today but we do have a workaround/fix for customData.

Following userData documentation, MS says userData is the newer version of custom data & it has below advantages.

• User data can be retrieved from Azure Instance Metadata Service(IMDS) after provision.
• User data is persistent. It will be available during the lifetime of the VM.
• User data can be updated from outside the VM, without stopping or rebooting the VM.
• User data can be queried via GET VM/VMSS API with $expand option.

I do see the userData in az-cli response ::

$az vm show --ids <vm-id> -u

...blah-blah "timeCreated": "2023-07-03T09:25:18.658200+00:00", "type": "Microsoft.Compute/virtualMachines", "userData": "IyEvYmluL2Jhc2gKCnRvdWNoIC90bXAvdGVzdGN1c3RvbWRhdGE=", "virtualMachineScaleSet": null, "vmId": "xxxx-yyyy-zzzzz", "zones": null

Here -u is an abbreviation of --include-user-data equivalent to $expand query argument as per restapi specs here

[microsoft-github-policy-service]

🔔 Routing this issue to @act-observability-squad.