MEGE TEST - TRE Upgrade v0.25.0 Custom and upstream changes

[JaimieWi]

Upgrade to AzureTRE v0.25.0

TRE run id: a2b2910e

tvstre-prod-main-v0.23.1 -> tvstre-prod-main-v0.25.0

Summary

This PR upgrades the TVS SDE fork to incorporate upstream changes from v0.24.0 and v0.25.0, and includes a set of local customisations covering new Gen2 VM images for both Linux and Windows, updated Guacamole user resource templates, and streamlined VM deployment scripts.

---

Upstream Changes

v0.25.0

⚠️ Important: If core deployment is updated prior to this release, an upstream issue with the Azure RM Terraform provider will cause the App Gateway and its certificates to be destroyed. Review the upstream release notes and plan your deployment order carefully before applying to production.

Enhancements

• Allow ENABLE_DNS_POLICY and ALLOWED_DNS variables to be set via CI/CD

Bug fixes

• App Gateway destroyed on second and subsequent deploys
• Workspace deletion fails due to storage account lock

---

v0.24.0

Enhancements

• Add revoke functionality and confirmation dialogs for Airlock requests
• Migrate Azure Firewall and Route Tables to Core Terraform
• Display VM owner information on resource cards with clickable mailto links
• Add sort/filter options to Workspace list
• Added backup vault to base workspace and updated Azure RM provider to match core

Bug fixes

• "Log analytics workspaces not found" error when deploying Databricks workspace service
• CostTag calling Costs API for every tag
• Reduced frequency of queue reader logging to improve log readability
• Resource lock indicator persisting when switching between resources
• Renew Letsencrypt GitHub action failing to access storage account
• Deployment pipeline failures due to KeyVault network policies preventing public access
• Storage account network rules not properly cleaned up on script failure
• UI display issue when workspace is deploying and user management is enabled
• Error details display when workspace deletion fails with deleted Entra app
• Cloud-init failure on VMSS startup caused by outdated gnupg2 package on Ubuntu Jammy
• Terraform dependency issue introduced by Azure Firewall and Route Tables migration

---

TVS SDE Custom Changes

New Gen2 VM Images

Both the Linux and Windows custom VM images have been rebuilt from scratch as Gen2 images, replacing the previous builds. All Nexus proxy configuration, package tooling, and desktop environment setup is now baked into the image at build time, significantly reducing the work performed by the VM deployment scripts at workspace provisioning time.

Linux — Ubuntu 22.04 Gen2 (imgdef-linux-vm-custom-gen2)

• XFCE desktop, xrdp, Anaconda, R 4.5.2, RStudio, VS Code with 16 extensions, Docker, Azure CLI, Azure Storage Explorer, Chrome
• pip, conda, R repo, and Docker daemon config all baked in pointing to internal Nexus proxy
• Keyring prompts eliminated for VS Code, Storage Explorer, and Chrome

Windows — Windows Server 2019 Gen2 (imgdef-windows-vm-custom-gen2)

• Python 3.10 and 3.13, Anaconda, VS Code with 11 extensions, R 4.5.2, RTools 4.5, RStudio, Git, LibreOffice, Azure Storage Explorer
• pip, conda, and R repo config baked in pointing to internal Nexus proxy
• Anaconda PATH set at Machine scope (C:\tools\Anaconda3)
• R configured with wininet download method to resolve SSL certificate trust issues with the internal Nexus CA
• Docker Desktop excluded — incompatible with Windows Server 2019 (no WSL2/nested virtualisation support)

---

Guacamole Linux VM Template (guacamole-linuxvm-ouh2)

• Updated source_image_name to imgdef-linux-vm-custom-gen2
• secure_boot_enabled and vtpm_enabled set to false — the build VM used Standard security type so the image definition does not carry SecurityType=TrustedLaunchSupported; to be revisited when images are next rebuilt from a Trusted Launch build VM

---

Guacamole Windows VM Template (guacamole-windowsvm-ouh2)

• Updated source_image_name from OUHWindowsImage to imgdef-windows-vm-custom-gen2
• Updated default os_image to OUH Server 2019 Data Science VM
• secure_boot_enabled and vtpm_enabled set to false — same reason as Linux above
• Added enable_shutdown_schedule, shutdown_time, and shutdown_timezone parameters, bringing the Windows template to parity with the Linux template
• Version bumped to 1.1.6

---

VM Deployment Scripts

vm_config.sh (Linux)

Removed all configuration now baked into the image:

• apt source configuration
• Base package installs
• Desktop environment (XFCE, xorg)
• pip, conda, R, and Docker proxy configuration
• Azure Storage Explorer install
• Chrome install and screensaver configuration

Retained (per-VM, per-workspace):

• Shared storage autofs/CIFS mount with per-workspace credentials
• Per-user xrdp session setup (~/.xsession, screen timeout settings)
• xrdp service enable and restart
• Docker service restart (daemon config is image-baked; service needs starting per VM)

vm_config.ps1 (Windows)

Removed all configuration now baked into the image:

• pip proxy configuration (pip.ini)
• conda channel configuration
• R repo and SSL configuration (Rprofile.site)
• Docker daemon configuration
• Anaconda PATH setup

Retained (per-VM, per-workspace):

• C:\AzureData cleanup
• Shared storage startup script (net use mapped to Z: at logon)

---

Testing

• Dev environment — deploy Linux VM from new image and verify RDP, desktop, and applications
• Dev environment — deploy Windows VM from new image and verify RDP and applications
• Verify shared storage mounts correctly on both VM types
• Verify Nexus package access on both VM types (pip install, conda install, R package install)
• Confirm upstream upgrade does not affect App Gateway (see v0.25.0 warning above)

[JaimieWi]

Merging — 1 April 2026

What was completed

All code changes are in place and the PR is ready to merge. The upstream v0.24.0 and v0.25.0 changes have been integrated, all custom bundle and script changes are included, and both the Linux and Windows Gen2 VM images have been built and captured to the gallery (imgdef-linux-vm-custom-gen2 and imgdef-windows-vm-custom-gen2).

This upgrade required significantly more work than anticipated. Several issues not identified at assessment time were only discovered during DEV testing, requiring multiple patch releases (v0.25.1 through v0.25.12) to resolve before the branch reached a deployable state:

• az login --identity --username deprecated across all Porter bundles, not just the Resource Processor — required fixes across commands.py, test_commands.py, cloud-config.yaml, and six bundle porter.yaml files
• gnupg2 pinned package version unavailable in Ubuntu 22.04 repos — caused Docker and Azure CLI to silently fail to install on VMSS cloud-init, meaning the Resource Processor never started; only visible on VMSS reimage
• Devcontainer base image dropped yarn independently of TRE version — pipeline UI build step failed regardless of TRE code version; required a deployment repo workaround

Pointing to release v0.25.12 https://github.com/OxBRCInformatics/AzureTRE/releases/tag/tvstre-prod-main-v0.25.12

Testing completed

• DEV TRE deployed and validated against the patched branch
• Linux VM deployed from new Gen2 image — RDP, desktop, and applications verified
• Windows VM deployed from new Gen2 image — RDP and applications verified, kernel discovery confirmed working
• Nexus proxy access verified on both VM types
• Shared storage mount verification
• Full workspace test suite against a prod-like environment — not completed (see risk below)

⚠️ Process Risk

Our standard process at this stage involves deploying a second DEV TRE from the feature branch — a clone of production containing pre-existing workspaces, workspace services, and user resources — and running the full workspace test suite against it. This validates that the upgraded TRE can manage resources that were created under a previous version, not just that the code merges cleanly.

Due to the volume of errors encountered during this upgrade, we were unable to get the feature branch into a deployable state early enough to stand up that second DEV TRE. We are therefore merging into production without that validation step. Any issues caused by the upgraded TRE interacting with existing production workspaces and resources will not have been caught pre-deployment and will only surface when researchers interact with those resources.

Production should be monitored closely following this merge and any researcher-reported issues prioritised accordingly.

[GamblingFool]

Gone through the merge process with Jaimie.
Tested merge & deploy with existing TRE prod 'clone' in DEV
All changes good.

[GamblingFool]

Gone through the merge process with Jaimie.
Tested merge & deploy with existing TRE prod 'clone' in DEV
All changes good.