fix: wpb-22439 offline-vm-setup script to manage VMs wiab-staging by mohitrajain · Pull Request #845 · wireapp/wire-server-deploy

mohitrajain · 2025-12-23T16:19:30Z

Summary of changes :

Removed command-line argument parsing - Script no longer supports --deploy-vm, --verbose, or -h flags; now only deploys the full 7-VM standard infrastructure.
Switched from static to dynamic DHCP IPs - VMs now acquire IPs automatically via DHCP instead of using pre-configured static IPs (192.168.122.10-33).
Replaced Autoinstall with Cloud-Init - Simplified provisioning from complex Ubuntu Installer to lightweight cloud-init with cloud-localds seed ISOs. VMs creation was taking 8 mins on physical hypervisor(KVM) enabled nodes and VM were in shut-off state once created, manual start was required. Now it takes 1.5 mins for 6 VMs.
Centralized SSH keys to deploy_dir/ssh/ - SSH keypairs now stored locally in the project directory instead of user's home .ssh/ folder with automatic permission validation.
Added .vm-env environment file export - Script now generates and exports all VM IPs as environment variables for downstream automation and inventory generation.
IP management was static - now it is DHCP based - better way to avoid collisions and in virtual network native way
If VMs are ready - old way was simple running check - modified to perform a ssh check

This ticket is part of cascading PRs:
#846 - ansible prepare host for vm provisioning
#847 - minor fixes for bin/offline-deploy.sh and cd.sh terraform inventory
#848 - add terraform resources for wiab-staging
#849 - add cd_staging.sh script
#850 - GitHub workflow wiab staging - All the changes are being verified here
#851 - docs for wiab-staging are here
#852 - clean old wiab staging script and workflows
#853 - Testing wiab-staging solution for wire-server-5.14
#856 - postgresql repmgr node config

No labels:
The changes in the PR - can't be verified using Github workflows - will add an asciinema.

Change type

Fix
Feature
Documentation
Security / Upgrade

Basic information

THIS CHANGE REQUIRES A DEPLOYMENT PACKAGE RELEASE
THIS CHANGE REQUIRES A WIRE-DOCS RELEASE

Testing

I ran/applied the changes myself, in a test environment.
The CI job attached to this repo will test it for me.

Offline Build CI (label-based)

Add one or more labels to trigger offline builds:

build-default - Full production build (ansible, terraform, all packages)
build-demo - Demo/WIAB build
build-min - Minimal build (fastest, essential charts only)
build-all - Run all three builds

Note: No builds run by default. Add a label to trigger CI.

Tracking

I added a new entry in an appropriate subdirectory of changelog.d
I mentioned this PR in Jira, OR I mentioned the Jira ticket in this PR.
I mentioned this PR in one of the issues attached to one of our repositories.

Knowledge Transfer

An Asciinema session is attached to the Jira ticket.

Motivation

Objective

Reason

Use case

bin/offline-vm-setup.sh

julialongtin · 2026-01-27T14:12:00Z

bin/offline-vm-setup.sh

+BASE_IMAGE="$BASE_IMAGE_DIR/ubuntu-22.04-base.qcow2"
+IMAGE_URL="https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img"

 if [ ! -d "$NOCLOUD_DIR" ]; then


remove the if block. -p is unconditional, and always succeeds.

julialongtin · 2026-01-27T14:12:34Z

bin/offline-vm-setup.sh

-  VM_VCPU=(2 6 6 6 4 4 4)
-  VM_RAM=(4096 8192 8192 8192 8192 8192 8192)
-  VM_DISK=(100 100 100 100 100 100 100)
+if [ ! -d "$BASE_IMAGE_DIR" ]; then


same here. do not mkdir -p in a if'd off block, unless you are doing something else with the if.

julialongtin · 2026-01-27T14:14:54Z

bin/offline-vm-setup.sh

+# Download base Ubuntu cloud image if not present
+if [ ! -f "$BASE_IMAGE" ]; then
+  msg "Downloading Ubuntu 22.04 cloud image to $BASE_IMAGE ..."
+  wget -q "$IMAGE_URL" -O "$BASE_IMAGE" || die "Failed to download Ubuntu cloud image"


wget vs curl here. if you're specifying the output file name, curl is same as wget, only saner to debug.

julialongtin · 2026-01-27T14:15:18Z

bin/offline-vm-setup.sh

-  ssh-keygen -t ed25519 -q -N '' -f "$HOME"/.ssh/id_ed25519
-  SSHKEY_DEMO=$(cat "$HOME"/.ssh/id_ed25519.pub)
+SSH_DIR="$DEPLOY_DIR/ssh"
+if [ ! -d "$SSH_DIR" ]; then


mkdir -p here.

changelog.d/3-deploy-builds/wiab-staging

…al seed iso and VM verification process

* fix: wpb-21356 rename ansible templates for wiab-staging to make them generic * build: wpb-21356 wiab-staging VMs ansible inventory template to be populated by wiab-staging-provision.yml playbook * build: wpb-21356 ansible playbook for wiab-staging VM provisioning * build: wpb-21356 add ansible inventory for playbook wiab-staging-provision.yml * fix: wpb-21356 rebase from wpb-21356-update-vm-management * fix: wpb-21356 remove vars specific to Hetzner CI process * fix: wpb-22439 remove screen package from wiab-staging-provision.yml ansible playbook * fix: wpb-22439 simplify the wiab-staging inventory to use ansible children instead of creating unique nodes * fix: wpb-22439 ansible playbook wiab-staging-provision.yml to consider the inventory changes * fix: wpb-22439 postgresql_rw and ro group inventory * fix: wpb-22439 fix wiab-staging inventory elasticsearch group name * fix: wpb-22439 fix wiab-staging inventory elasticsearch group names * fix: wpb-22439 remove sshd config template as it is not functionally required * fix: wpb-22439 moved artifact hash to wiab-staging inventory * fix: wpb-22439 fix secondary inventory creation, organize playbook using tags and separate out wiab-staging-nftables playbook * fix: wpb-22439 allow for load balancing for ingress-controller, variable usage to define each k8s node and calling node and add comments * fix: wpb-22439 add wiab-staging-nftables.yaml playbook to manage nftables rules * fix: wpb-22439 allow rabbitmq ansible inventory for wiab-staging * wpb-22439 minor fixes for bin/offline-deploy.sh and cd.sh terraform inventory (#847) * fix: wpb-21356 rebase from wpb-21356-ansible-VM-provisioning * fix: wpb-21356 offline-deploy.sh for SSH_AUTH_SOCK handling * fix: wpb-21356 remove defunct passwords for postgresql * fix: wpb-22439 add terraform resources for wiab-staging (#848) * fix: wpb-21356 fix node types for assethost and adminhost * build: wpb-21356 add changelog for terraform wiab-staging resources * fix: wpb-22439 fix wiab-staging hetzner setup_nodes playbook to handle connection issue * build: wpb-22439 add cd_staging script (#849) * build: wpb-21356 add cd_staging script * build: wpb-21356 add changelog for cd_staging script * build: wpb-21356 enable creating secondary inventory for the VMs * fix: wpb-22439 cd_staging script as per wiab-staging inventory * fix: wpb-22439 fix secondary-inventory geenration logic to keep the original hostnames and bring SSH_OPTS * fix: wpb-22439 create a separate group for rabbitmq-hosts and keep other groups same * fix: wpb-22439 remove old children group for rmq-cluster before adding new one * fix: wpb-22439 fix postgresql_network_interface variable for postgresql * fix: wpb-22439 cd_staging.sh script to handle terrafrom nodenames for all the iventory groups * wpb-22439 6 GitHub workflow wiab staging (#850) * build: wpb-21356 changed the flow of offline.yml - introduced wiab-staging build and split bundle processing with default-build * build: wpb-21356 update pull_request_template and add new label for build-wiab-staging * fix: wpb-21356 offline/cd.sh for demo verification process as github workflow has been updated * fix: wpb-21356 add host key verification flags for scp commands * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * fix: wpb-22846 remove debug statements from cd.sh * fix: wpb-22439 remove new-line in cd_staging.sh * fix: wpb-22439 standardize SSH_OPTS in cd.sh * fix: wpb-22439 remove branch limitation for workflow * fix: wpb-22439 harden the ansible inventory params for default profile * Wpb 22439 postgresql repmgr node config (#856) * fix: wpb-22439 remove repmgr_node_config from group_vars * fix: wpb-22439 add repmgr_node_config to node specific inventory * fix: wpb-22439 add node specific repmgr_node_config to wiab-staging inventory * fix: wpb-22439 add changelog * fix: wpb-22439 add comment on hostname requirement directly for rmq-cluster group * wpb-22439 7 docs wiab staging (#851) * fix: wpb-21356 fix coturn port ranges * build: wpb-21356 add documentation for wiab-staging and clean old single_hetzner_machine_installation * build: wpb-21356 add changelog and small fix for offline-vm-setup.sh * fix: wpb-22439 memory requirements for VMs in wiab-staging.md doc * fix: wpb-22846 update wiab-staging documentation for hairpin networking * fix: wpb-22439 fix documentation typo * fix: wpb-22439 fix documentation suggestions * fix: wpb-22439 update wiab-staging documentation * wpb-22439 8 clean old wiab staging script and workflows (#852) * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * refactor: wpb-21356 remove old deploy-wiab workflow as it has already been incorporated in offline.yml * refactor: wpb-21356 remove old autodeploy.sh as offline/cd_staging.sh will replace it * refactor: wpb-21356 remove references of old autodeploy.sh, wiab-hetzner playbook and documentation * fix: wpb-22439 ansible/wiab-staging-provision.yml for ssh_private_key update * fix: wpb-22439 update cert-manager & hairpin NAT documentation

sonarqubecloud · 2026-02-19T10:20:51Z

Quality Gate passed

Issues
12 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

* fix: wpb-22439 enable rabbitmq deployment via ansible * fix: wpb-22439 fix static inventory for rabbitmq to remove nodename dependency * fix: wpb-22439 enable deploying rabbitmq-external and stop deploying rabbitmq chart * fix: wpb-22439 enable creating dedicated nodes for rabbitmq VMs * fix: wpb-22439 fix rmq-cluster inventory group name for rabbitmq terraform created nodes * fix: wpb-22439 include rmq-cluster nodes for networking fix * fix: wpb-22439 refactor node names in hetzner terraform for rabbitmq * fix: wpb-22439 offline-vm-setup script to manage VMs wiab-staging (#845) * fix: wpb-21356 offline-vm-setup script to use ubuntu cloud image, local seed iso and VM verification process * fix: wpb-21356 linting issues and push changelog * fix: wpb-22439 memory requirements for VMs in offline-vm-setup.sh * fix: wpb-22439 fix kubenode disk size * Update changelog.d/3-deploy-builds/wiab-staging * fix: wpb-22439 fix offline-vm-setup.sh * fix: wpb-22439 BASE_IMAGE_DIR value * wpb-22439 wiab-staging: ansible prepare host for vm provisioning (#846) * fix: wpb-21356 rename ansible templates for wiab-staging to make them generic * build: wpb-21356 wiab-staging VMs ansible inventory template to be populated by wiab-staging-provision.yml playbook * build: wpb-21356 ansible playbook for wiab-staging VM provisioning * build: wpb-21356 add ansible inventory for playbook wiab-staging-provision.yml * fix: wpb-21356 rebase from wpb-21356-update-vm-management * fix: wpb-21356 remove vars specific to Hetzner CI process * fix: wpb-22439 remove screen package from wiab-staging-provision.yml ansible playbook * fix: wpb-22439 simplify the wiab-staging inventory to use ansible children instead of creating unique nodes * fix: wpb-22439 ansible playbook wiab-staging-provision.yml to consider the inventory changes * fix: wpb-22439 postgresql_rw and ro group inventory * fix: wpb-22439 fix wiab-staging inventory elasticsearch group name * fix: wpb-22439 fix wiab-staging inventory elasticsearch group names * fix: wpb-22439 remove sshd config template as it is not functionally required * fix: wpb-22439 moved artifact hash to wiab-staging inventory * fix: wpb-22439 fix secondary inventory creation, organize playbook using tags and separate out wiab-staging-nftables playbook * fix: wpb-22439 allow for load balancing for ingress-controller, variable usage to define each k8s node and calling node and add comments * fix: wpb-22439 add wiab-staging-nftables.yaml playbook to manage nftables rules * fix: wpb-22439 allow rabbitmq ansible inventory for wiab-staging * wpb-22439 minor fixes for bin/offline-deploy.sh and cd.sh terraform inventory (#847) * fix: wpb-21356 rebase from wpb-21356-ansible-VM-provisioning * fix: wpb-21356 offline-deploy.sh for SSH_AUTH_SOCK handling * fix: wpb-21356 remove defunct passwords for postgresql * fix: wpb-22439 add terraform resources for wiab-staging (#848) * fix: wpb-21356 fix node types for assethost and adminhost * build: wpb-21356 add changelog for terraform wiab-staging resources * fix: wpb-22439 fix wiab-staging hetzner setup_nodes playbook to handle connection issue * build: wpb-22439 add cd_staging script (#849) * build: wpb-21356 add cd_staging script * build: wpb-21356 add changelog for cd_staging script * build: wpb-21356 enable creating secondary inventory for the VMs * fix: wpb-22439 cd_staging script as per wiab-staging inventory * fix: wpb-22439 fix secondary-inventory geenration logic to keep the original hostnames and bring SSH_OPTS * fix: wpb-22439 create a separate group for rabbitmq-hosts and keep other groups same * fix: wpb-22439 remove old children group for rmq-cluster before adding new one * fix: wpb-22439 fix postgresql_network_interface variable for postgresql * fix: wpb-22439 cd_staging.sh script to handle terrafrom nodenames for all the iventory groups * wpb-22439 6 GitHub workflow wiab staging (#850) * build: wpb-21356 changed the flow of offline.yml - introduced wiab-staging build and split bundle processing with default-build * build: wpb-21356 update pull_request_template and add new label for build-wiab-staging * fix: wpb-21356 offline/cd.sh for demo verification process as github workflow has been updated * fix: wpb-21356 add host key verification flags for scp commands * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * fix: wpb-22846 remove debug statements from cd.sh * fix: wpb-22439 remove new-line in cd_staging.sh * fix: wpb-22439 standardize SSH_OPTS in cd.sh * fix: wpb-22439 remove branch limitation for workflow * fix: wpb-22439 harden the ansible inventory params for default profile * Wpb 22439 postgresql repmgr node config (#856) * fix: wpb-22439 remove repmgr_node_config from group_vars * fix: wpb-22439 add repmgr_node_config to node specific inventory * fix: wpb-22439 add node specific repmgr_node_config to wiab-staging inventory * fix: wpb-22439 add changelog * fix: wpb-22439 add comment on hostname requirement directly for rmq-cluster group * wpb-22439 7 docs wiab staging (#851) * fix: wpb-21356 fix coturn port ranges * build: wpb-21356 add documentation for wiab-staging and clean old single_hetzner_machine_installation * build: wpb-21356 add changelog and small fix for offline-vm-setup.sh * fix: wpb-22439 memory requirements for VMs in wiab-staging.md doc * fix: wpb-22846 update wiab-staging documentation for hairpin networking * fix: wpb-22439 fix documentation typo * fix: wpb-22439 fix documentation suggestions * fix: wpb-22439 update wiab-staging documentation * wpb-22439 8 clean old wiab staging script and workflows (#852) * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * refactor: wpb-21356 remove old deploy-wiab workflow as it has already been incorporated in offline.yml * refactor: wpb-21356 remove old autodeploy.sh as offline/cd_staging.sh will replace it * refactor: wpb-21356 remove references of old autodeploy.sh, wiab-hetzner playbook and documentation * fix: wpb-22439 ansible/wiab-staging-provision.yml for ssh_private_key update * fix: wpb-22439 update cert-manager & hairpin NAT documentation --------- Co-authored-by: Julia Longtin <julia.longtin@wire.com> * fix: wpb-23462: remove unused vars from cd.sh and cd_staging.sh * fix: wpb-23462: fix linting issues for cd_staging.sh * fix: wpb-22439 SSH_OPTS var in cd_staging.sh and ignore linting in shellcheck.sh * fix: wpb-22439 update the wiab-staging.yml with the latest artifact hash --------- Co-authored-by: Julia Longtin <julia.longtin@wire.com>

mohitrajain requested review from a team and julialongtin as code owners December 23, 2025 16:19

mohitrajain changed the title ~~fix: wpb-21356 offline-vm-setup script to manage VMs~~ fix: wpb-21356 offline-vm-setup script to manage VMs wiab-staging Dec 28, 2025

mohitrajain changed the title ~~fix: wpb-21356 offline-vm-setup script to manage VMs wiab-staging~~ fix: wpb-22439 offline-vm-setup script to manage VMs wiab-staging Jan 5, 2026

Veki301 requested changes Jan 7, 2026

View reviewed changes

bin/offline-vm-setup.sh Outdated Show resolved Hide resolved

bin/offline-vm-setup.sh Outdated Show resolved Hide resolved

bin/offline-vm-setup.sh Outdated Show resolved Hide resolved

bin/offline-vm-setup.sh Show resolved Hide resolved

bin/offline-vm-setup.sh Show resolved Hide resolved

Veki301 approved these changes Jan 8, 2026

View reviewed changes

julialongtin reviewed Jan 27, 2026

View reviewed changes

bin/offline-vm-setup.sh Show resolved Hide resolved

julialongtin reviewed Jan 27, 2026

View reviewed changes

bin/offline-vm-setup.sh Show resolved Hide resolved

julialongtin reviewed Jan 27, 2026

View reviewed changes

changelog.d/3-deploy-builds/wiab-staging Outdated Show resolved Hide resolved

mohitrajain force-pushed the wpb-21356-update-vm-management branch from 5441047 to 71c155e Compare January 29, 2026 11:12

mohitrajain changed the base branch from master to wpb-22439-0-enable-rabbitmq-ansible January 29, 2026 11:12

mohitrajain and others added 7 commits January 29, 2026 16:25

fix: wpb-21356 offline-vm-setup script to use ubuntu cloud image, loc…

56a0e81

…al seed iso and VM verification process

fix: wpb-21356 linting issues and push changelog

0d50d36

fix: wpb-22439 memory requirements for VMs in offline-vm-setup.sh

b2622b2

fix: wpb-22439 fix kubenode disk size

b23cc86

Update changelog.d/3-deploy-builds/wiab-staging

48a842a

fix: wpb-22439 fix offline-vm-setup.sh

2629656

fix: wpb-22439 BASE_IMAGE_DIR value

7b9a97f

mohitrajain force-pushed the wpb-21356-update-vm-management branch from 1f58fc3 to 7b9a97f Compare January 29, 2026 15:25

mohitrajain merged commit f727881 into wpb-22439-0-enable-rabbitmq-ansible Feb 19, 2026
15 checks passed

mohitrajain deleted the wpb-21356-update-vm-management branch February 19, 2026 10:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

fix: wpb-22439 offline-vm-setup script to manage VMs wiab-staging#845

fix: wpb-22439 offline-vm-setup script to manage VMs wiab-staging#845
mohitrajain merged 8 commits intowpb-22439-0-enable-rabbitmq-ansiblefrom
wpb-21356-update-vm-management

mohitrajain commented Dec 23, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

julialongtin Jan 27, 2026

Uh oh!

julialongtin Jan 27, 2026

Uh oh!

julialongtin Jan 27, 2026

Uh oh!

julialongtin Jan 27, 2026

Uh oh!

Uh oh!

sonarqubecloud bot commented Feb 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

Conversation

mohitrajain commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change type

Basic information

Testing

Offline Build CI (label-based)

Tracking

Knowledge Transfer

Motivation

Objective

Reason

Use case

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

julialongtin Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

julialongtin Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

julialongtin Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

julialongtin Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sonarqubecloud bot commented Feb 19, 2026

Quality Gate passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mohitrajain commented Dec 23, 2025 •

edited

Loading