wpb-22439 7 docs wiab staging#851
wpb-22439 7 docs wiab staging#851mohitrajain merged 9 commits intowpb-22439-postgresql-repmgr_node_configfrom
Conversation
594b922 to
28b9f70
Compare
ea69e50 to
6dea77e
Compare
7b39bdd to
9e1ee11
Compare
6dea77e to
be20a6c
Compare
21281f9 to
a9032df
Compare
be20a6c to
6ded955
Compare
a9032df to
7f9ddf0
Compare
668a27c to
7c8f95f
Compare
| Added: terraform resources for wiab-staging | ||
| Added: cd_staging script to verify the default build bundle | ||
| Changed: changed the flow of offline.yml - introduced wiab-staging build and split bundle processing with default-build | ||
| Added: wiab-staging documetation to wire-server-deploy and fix coturn port ranges |
offline/wiab-staging.md
Outdated
| - One physical machine with hypervisor support: | ||
| - **Memory:** 55 GiB RAM | ||
| - **Compute:** 29 vCPUs | ||
| - **Storage:** 550 GB disk space (thin-provisioned) |
| | assethost | Asset/Storage Server | 4 GiB | 2 | 100 GB | | ||
| | kubenode1 | Kubernetes Node 1 | 9 GiB | 5 | 150 GB | | ||
| | kubenode2 | Kubernetes Node 2 | 9 GiB | 5 | 150 GB | | ||
| | kubenode3 | Kubernetes Node 3 | 9 GiB | 5 | 150 GB | |
| - Source the `bin/offline-env.sh` shell script to set up a `d` alias that runs commands inside a Docker container with all necessary tools for offline deployment. | ||
|
|
||
| - **[Generating secrets](docs_ubuntu_22.04.md#generating-secrets)** | ||
| - Run `./bin/offline-secrets.sh` to generate fresh secrets for Minio and coturn services. This creates two secret files: `ansible/inventory/group_vars/all/secrets.yaml` and `values/wire-server/secrets.yaml`. |
There was a problem hiding this comment.
values/wire-server/secrets.yaml does not get generated anymore, now its prod-secrets.example.yaml, which also brings the question, why ?
There was a problem hiding this comment.
#858 - it is generated here and this PR came post creating this documentation, and the idea is to create a copy first from prod-secrets.example.yaml to secrets.yaml for each chart and make changes directly to this secrets.yaml and values.yaml, so that an admin can understand what changes has been done by our scripts. As of now, the changes are going to prod-secrets.example.yaml in #858 I will patch it to make offline-secrets run post creating the copies.
| - **Deploying Helm charts** | ||
| - **[Deploying stateless services and other dependencies](docs_ubuntu_22.04.md#deploying-stateless-dependencies)** | ||
| - Deploy cassandra-external, elasticsearch-external, minio-external, and databases-ephemeral helm charts to set up connections to external data services and stateless database dependencies. | ||
|
|
There was a problem hiding this comment.
you are missing rabbitmq here, which causes the deployment to fail
There was a problem hiding this comment.
rabbitmq is coming from the helm chart, In the recent failure, i was trying to enable it again. As of this PR, rabbitmq is getting deployed using helm chart.
https://github.com/wireapp/wire-server-deploy/blob/master/bin/offline-cluster.sh#L55
offline/wiab-staging.md
Outdated
|
|
||
| - **Deploying Helm charts** | ||
| - **[Deploying stateless services and other dependencies](docs_ubuntu_22.04.md#deploying-stateless-dependencies)** | ||
| - Deploy cassandra-external, elasticsearch-external, minio-external, and databases-ephemeral helm charts to set up connections to external data services and stateless database dependencies. |
There was a problem hiding this comment.
reaper does not deploy
Failed to pull image "docker.io/bitnamilegacy/kubectl:1.32.9": rpc error: code = NotFound desc = failed to pull and unpack image "docker.io/bitnamilegacy/kubectl:1.32.9": failed to resolve reference "docker.io/bitnamilegacy/kubectl:1.32.9": docker.io/bitnamilegacy/kubectl:1.32.9: not found
There was a problem hiding this comment.
Where is it happening, can you please share the run?
| ### Network & Security | ||
|
|
||
| - **[Enabling emails for wire](smtp.md)** | ||
| - Configure SMTP for user onboarding via email. Deploy either a temporary SMTP service included in the bundle or integrate with your existing SMTP relay, and ensure proper network configuration for email delivery. |
There was a problem hiding this comment.
you can remove this, I dont see why anybody would be bothering with setting up DKIM for a test environment
There was a problem hiding this comment.
smtp can be deployed without setting up DKIM and we would need some kind of mail service to enable user creation and testing
Veki301
left a comment
There was a problem hiding this comment.
documentation incomplete
| @@ -0,0 +1,220 @@ | |||
| # Scope | |||
|
|
|||
There was a problem hiding this comment.
documentation is incomplete, as we have discussed in our meeting
d11d557 to
d56c938
Compare
7c8f95f to
a9e1137
Compare
fdb2c12 to
0254dc9
Compare
18d8d0c to
6350fce
Compare
31d4a93 to
de86aa0
Compare
6350fce to
bddee18
Compare
de86aa0 to
425c518
Compare
bddee18 to
bafba56
Compare
425c518 to
7728269
Compare
bafba56 to
e63a276
Compare
7728269 to
84e6103
Compare
…gle_hetzner_machine_installation
9927886 to
a7b6ab6
Compare
* fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * refactor: wpb-21356 remove old deploy-wiab workflow as it has already been incorporated in offline.yml * refactor: wpb-21356 remove old autodeploy.sh as offline/cd_staging.sh will replace it * refactor: wpb-21356 remove references of old autodeploy.sh, wiab-hetzner playbook and documentation
|
c304abf
into
wpb-22439-postgresql-repmgr_node_config
* fix: wpb-22439 remove repmgr_node_config from group_vars * fix: wpb-22439 add repmgr_node_config to node specific inventory * fix: wpb-22439 add node specific repmgr_node_config to wiab-staging inventory * fix: wpb-22439 add changelog * fix: wpb-22439 add comment on hostname requirement directly for rmq-cluster group * wpb-22439 7 docs wiab staging (#851) * fix: wpb-21356 fix coturn port ranges * build: wpb-21356 add documentation for wiab-staging and clean old single_hetzner_machine_installation * build: wpb-21356 add changelog and small fix for offline-vm-setup.sh * fix: wpb-22439 memory requirements for VMs in wiab-staging.md doc * fix: wpb-22846 update wiab-staging documentation for hairpin networking * fix: wpb-22439 fix documentation typo * fix: wpb-22439 fix documentation suggestions * fix: wpb-22439 update wiab-staging documentation * wpb-22439 8 clean old wiab staging script and workflows (#852) * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * refactor: wpb-21356 remove old deploy-wiab workflow as it has already been incorporated in offline.yml * refactor: wpb-21356 remove old autodeploy.sh as offline/cd_staging.sh will replace it * refactor: wpb-21356 remove references of old autodeploy.sh, wiab-hetzner playbook and documentation * fix: wpb-22439 ansible/wiab-staging-provision.yml for ssh_private_key update * fix: wpb-22439 update cert-manager & hairpin NAT documentation
* build: wpb-21356 changed the flow of offline.yml - introduced wiab-staging build and split bundle processing with default-build * build: wpb-21356 update pull_request_template and add new label for build-wiab-staging * fix: wpb-21356 offline/cd.sh for demo verification process as github workflow has been updated * fix: wpb-21356 add host key verification flags for scp commands * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * fix: wpb-22846 remove debug statements from cd.sh * fix: wpb-22439 remove new-line in cd_staging.sh * fix: wpb-22439 standardize SSH_OPTS in cd.sh * fix: wpb-22439 remove branch limitation for workflow * fix: wpb-22439 harden the ansible inventory params for default profile * Wpb 22439 postgresql repmgr node config (#856) * fix: wpb-22439 remove repmgr_node_config from group_vars * fix: wpb-22439 add repmgr_node_config to node specific inventory * fix: wpb-22439 add node specific repmgr_node_config to wiab-staging inventory * fix: wpb-22439 add changelog * fix: wpb-22439 add comment on hostname requirement directly for rmq-cluster group * wpb-22439 7 docs wiab staging (#851) * fix: wpb-21356 fix coturn port ranges * build: wpb-21356 add documentation for wiab-staging and clean old single_hetzner_machine_installation * build: wpb-21356 add changelog and small fix for offline-vm-setup.sh * fix: wpb-22439 memory requirements for VMs in wiab-staging.md doc * fix: wpb-22846 update wiab-staging documentation for hairpin networking * fix: wpb-22439 fix documentation typo * fix: wpb-22439 fix documentation suggestions * fix: wpb-22439 update wiab-staging documentation * wpb-22439 8 clean old wiab staging script and workflows (#852) * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * refactor: wpb-21356 remove old deploy-wiab workflow as it has already been incorporated in offline.yml * refactor: wpb-21356 remove old autodeploy.sh as offline/cd_staging.sh will replace it * refactor: wpb-21356 remove references of old autodeploy.sh, wiab-hetzner playbook and documentation * fix: wpb-22439 ansible/wiab-staging-provision.yml for ssh_private_key update * fix: wpb-22439 update cert-manager & hairpin NAT documentation
* build: wpb-21356 add cd_staging script * build: wpb-21356 add changelog for cd_staging script * build: wpb-21356 enable creating secondary inventory for the VMs * fix: wpb-22439 cd_staging script as per wiab-staging inventory * fix: wpb-22439 fix secondary-inventory geenration logic to keep the original hostnames and bring SSH_OPTS * fix: wpb-22439 create a separate group for rabbitmq-hosts and keep other groups same * fix: wpb-22439 remove old children group for rmq-cluster before adding new one * fix: wpb-22439 fix postgresql_network_interface variable for postgresql * fix: wpb-22439 cd_staging.sh script to handle terrafrom nodenames for all the iventory groups * wpb-22439 6 GitHub workflow wiab staging (#850) * build: wpb-21356 changed the flow of offline.yml - introduced wiab-staging build and split bundle processing with default-build * build: wpb-21356 update pull_request_template and add new label for build-wiab-staging * fix: wpb-21356 offline/cd.sh for demo verification process as github workflow has been updated * fix: wpb-21356 add host key verification flags for scp commands * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * fix: wpb-22846 remove debug statements from cd.sh * fix: wpb-22439 remove new-line in cd_staging.sh * fix: wpb-22439 standardize SSH_OPTS in cd.sh * fix: wpb-22439 remove branch limitation for workflow * fix: wpb-22439 harden the ansible inventory params for default profile * Wpb 22439 postgresql repmgr node config (#856) * fix: wpb-22439 remove repmgr_node_config from group_vars * fix: wpb-22439 add repmgr_node_config to node specific inventory * fix: wpb-22439 add node specific repmgr_node_config to wiab-staging inventory * fix: wpb-22439 add changelog * fix: wpb-22439 add comment on hostname requirement directly for rmq-cluster group * wpb-22439 7 docs wiab staging (#851) * fix: wpb-21356 fix coturn port ranges * build: wpb-21356 add documentation for wiab-staging and clean old single_hetzner_machine_installation * build: wpb-21356 add changelog and small fix for offline-vm-setup.sh * fix: wpb-22439 memory requirements for VMs in wiab-staging.md doc * fix: wpb-22846 update wiab-staging documentation for hairpin networking * fix: wpb-22439 fix documentation typo * fix: wpb-22439 fix documentation suggestions * fix: wpb-22439 update wiab-staging documentation * wpb-22439 8 clean old wiab staging script and workflows (#852) * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * refactor: wpb-21356 remove old deploy-wiab workflow as it has already been incorporated in offline.yml * refactor: wpb-21356 remove old autodeploy.sh as offline/cd_staging.sh will replace it * refactor: wpb-21356 remove references of old autodeploy.sh, wiab-hetzner playbook and documentation * fix: wpb-22439 ansible/wiab-staging-provision.yml for ssh_private_key update * fix: wpb-22439 update cert-manager & hairpin NAT documentation
* fix: wpb-21356 fix node types for assethost and adminhost * build: wpb-21356 add changelog for terraform wiab-staging resources * fix: wpb-22439 fix wiab-staging hetzner setup_nodes playbook to handle connection issue * build: wpb-22439 add cd_staging script (#849) * build: wpb-21356 add cd_staging script * build: wpb-21356 add changelog for cd_staging script * build: wpb-21356 enable creating secondary inventory for the VMs * fix: wpb-22439 cd_staging script as per wiab-staging inventory * fix: wpb-22439 fix secondary-inventory geenration logic to keep the original hostnames and bring SSH_OPTS * fix: wpb-22439 create a separate group for rabbitmq-hosts and keep other groups same * fix: wpb-22439 remove old children group for rmq-cluster before adding new one * fix: wpb-22439 fix postgresql_network_interface variable for postgresql * fix: wpb-22439 cd_staging.sh script to handle terrafrom nodenames for all the iventory groups * wpb-22439 6 GitHub workflow wiab staging (#850) * build: wpb-21356 changed the flow of offline.yml - introduced wiab-staging build and split bundle processing with default-build * build: wpb-21356 update pull_request_template and add new label for build-wiab-staging * fix: wpb-21356 offline/cd.sh for demo verification process as github workflow has been updated * fix: wpb-21356 add host key verification flags for scp commands * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * fix: wpb-22846 remove debug statements from cd.sh * fix: wpb-22439 remove new-line in cd_staging.sh * fix: wpb-22439 standardize SSH_OPTS in cd.sh * fix: wpb-22439 remove branch limitation for workflow * fix: wpb-22439 harden the ansible inventory params for default profile * Wpb 22439 postgresql repmgr node config (#856) * fix: wpb-22439 remove repmgr_node_config from group_vars * fix: wpb-22439 add repmgr_node_config to node specific inventory * fix: wpb-22439 add node specific repmgr_node_config to wiab-staging inventory * fix: wpb-22439 add changelog * fix: wpb-22439 add comment on hostname requirement directly for rmq-cluster group * wpb-22439 7 docs wiab staging (#851) * fix: wpb-21356 fix coturn port ranges * build: wpb-21356 add documentation for wiab-staging and clean old single_hetzner_machine_installation * build: wpb-21356 add changelog and small fix for offline-vm-setup.sh * fix: wpb-22439 memory requirements for VMs in wiab-staging.md doc * fix: wpb-22846 update wiab-staging documentation for hairpin networking * fix: wpb-22439 fix documentation typo * fix: wpb-22439 fix documentation suggestions * fix: wpb-22439 update wiab-staging documentation * wpb-22439 8 clean old wiab staging script and workflows (#852) * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * refactor: wpb-21356 remove old deploy-wiab workflow as it has already been incorporated in offline.yml * refactor: wpb-21356 remove old autodeploy.sh as offline/cd_staging.sh will replace it * refactor: wpb-21356 remove references of old autodeploy.sh, wiab-hetzner playbook and documentation * fix: wpb-22439 ansible/wiab-staging-provision.yml for ssh_private_key update * fix: wpb-22439 update cert-manager & hairpin NAT documentation
…nventory (#847) * fix: wpb-21356 rebase from wpb-21356-ansible-VM-provisioning * fix: wpb-21356 offline-deploy.sh for SSH_AUTH_SOCK handling * fix: wpb-21356 remove defunct passwords for postgresql * fix: wpb-22439 add terraform resources for wiab-staging (#848) * fix: wpb-21356 fix node types for assethost and adminhost * build: wpb-21356 add changelog for terraform wiab-staging resources * fix: wpb-22439 fix wiab-staging hetzner setup_nodes playbook to handle connection issue * build: wpb-22439 add cd_staging script (#849) * build: wpb-21356 add cd_staging script * build: wpb-21356 add changelog for cd_staging script * build: wpb-21356 enable creating secondary inventory for the VMs * fix: wpb-22439 cd_staging script as per wiab-staging inventory * fix: wpb-22439 fix secondary-inventory geenration logic to keep the original hostnames and bring SSH_OPTS * fix: wpb-22439 create a separate group for rabbitmq-hosts and keep other groups same * fix: wpb-22439 remove old children group for rmq-cluster before adding new one * fix: wpb-22439 fix postgresql_network_interface variable for postgresql * fix: wpb-22439 cd_staging.sh script to handle terrafrom nodenames for all the iventory groups * wpb-22439 6 GitHub workflow wiab staging (#850) * build: wpb-21356 changed the flow of offline.yml - introduced wiab-staging build and split bundle processing with default-build * build: wpb-21356 update pull_request_template and add new label for build-wiab-staging * fix: wpb-21356 offline/cd.sh for demo verification process as github workflow has been updated * fix: wpb-21356 add host key verification flags for scp commands * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * fix: wpb-22846 remove debug statements from cd.sh * fix: wpb-22439 remove new-line in cd_staging.sh * fix: wpb-22439 standardize SSH_OPTS in cd.sh * fix: wpb-22439 remove branch limitation for workflow * fix: wpb-22439 harden the ansible inventory params for default profile * Wpb 22439 postgresql repmgr node config (#856) * fix: wpb-22439 remove repmgr_node_config from group_vars * fix: wpb-22439 add repmgr_node_config to node specific inventory * fix: wpb-22439 add node specific repmgr_node_config to wiab-staging inventory * fix: wpb-22439 add changelog * fix: wpb-22439 add comment on hostname requirement directly for rmq-cluster group * wpb-22439 7 docs wiab staging (#851) * fix: wpb-21356 fix coturn port ranges * build: wpb-21356 add documentation for wiab-staging and clean old single_hetzner_machine_installation * build: wpb-21356 add changelog and small fix for offline-vm-setup.sh * fix: wpb-22439 memory requirements for VMs in wiab-staging.md doc * fix: wpb-22846 update wiab-staging documentation for hairpin networking * fix: wpb-22439 fix documentation typo * fix: wpb-22439 fix documentation suggestions * fix: wpb-22439 update wiab-staging documentation * wpb-22439 8 clean old wiab staging script and workflows (#852) * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * refactor: wpb-21356 remove old deploy-wiab workflow as it has already been incorporated in offline.yml * refactor: wpb-21356 remove old autodeploy.sh as offline/cd_staging.sh will replace it * refactor: wpb-21356 remove references of old autodeploy.sh, wiab-hetzner playbook and documentation * fix: wpb-22439 ansible/wiab-staging-provision.yml for ssh_private_key update * fix: wpb-22439 update cert-manager & hairpin NAT documentation
* fix: wpb-21356 rename ansible templates for wiab-staging to make them generic * build: wpb-21356 wiab-staging VMs ansible inventory template to be populated by wiab-staging-provision.yml playbook * build: wpb-21356 ansible playbook for wiab-staging VM provisioning * build: wpb-21356 add ansible inventory for playbook wiab-staging-provision.yml * fix: wpb-21356 rebase from wpb-21356-update-vm-management * fix: wpb-21356 remove vars specific to Hetzner CI process * fix: wpb-22439 remove screen package from wiab-staging-provision.yml ansible playbook * fix: wpb-22439 simplify the wiab-staging inventory to use ansible children instead of creating unique nodes * fix: wpb-22439 ansible playbook wiab-staging-provision.yml to consider the inventory changes * fix: wpb-22439 postgresql_rw and ro group inventory * fix: wpb-22439 fix wiab-staging inventory elasticsearch group name * fix: wpb-22439 fix wiab-staging inventory elasticsearch group names * fix: wpb-22439 remove sshd config template as it is not functionally required * fix: wpb-22439 moved artifact hash to wiab-staging inventory * fix: wpb-22439 fix secondary inventory creation, organize playbook using tags and separate out wiab-staging-nftables playbook * fix: wpb-22439 allow for load balancing for ingress-controller, variable usage to define each k8s node and calling node and add comments * fix: wpb-22439 add wiab-staging-nftables.yaml playbook to manage nftables rules * fix: wpb-22439 allow rabbitmq ansible inventory for wiab-staging * wpb-22439 minor fixes for bin/offline-deploy.sh and cd.sh terraform inventory (#847) * fix: wpb-21356 rebase from wpb-21356-ansible-VM-provisioning * fix: wpb-21356 offline-deploy.sh for SSH_AUTH_SOCK handling * fix: wpb-21356 remove defunct passwords for postgresql * fix: wpb-22439 add terraform resources for wiab-staging (#848) * fix: wpb-21356 fix node types for assethost and adminhost * build: wpb-21356 add changelog for terraform wiab-staging resources * fix: wpb-22439 fix wiab-staging hetzner setup_nodes playbook to handle connection issue * build: wpb-22439 add cd_staging script (#849) * build: wpb-21356 add cd_staging script * build: wpb-21356 add changelog for cd_staging script * build: wpb-21356 enable creating secondary inventory for the VMs * fix: wpb-22439 cd_staging script as per wiab-staging inventory * fix: wpb-22439 fix secondary-inventory geenration logic to keep the original hostnames and bring SSH_OPTS * fix: wpb-22439 create a separate group for rabbitmq-hosts and keep other groups same * fix: wpb-22439 remove old children group for rmq-cluster before adding new one * fix: wpb-22439 fix postgresql_network_interface variable for postgresql * fix: wpb-22439 cd_staging.sh script to handle terrafrom nodenames for all the iventory groups * wpb-22439 6 GitHub workflow wiab staging (#850) * build: wpb-21356 changed the flow of offline.yml - introduced wiab-staging build and split bundle processing with default-build * build: wpb-21356 update pull_request_template and add new label for build-wiab-staging * fix: wpb-21356 offline/cd.sh for demo verification process as github workflow has been updated * fix: wpb-21356 add host key verification flags for scp commands * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * fix: wpb-22846 remove debug statements from cd.sh * fix: wpb-22439 remove new-line in cd_staging.sh * fix: wpb-22439 standardize SSH_OPTS in cd.sh * fix: wpb-22439 remove branch limitation for workflow * fix: wpb-22439 harden the ansible inventory params for default profile * Wpb 22439 postgresql repmgr node config (#856) * fix: wpb-22439 remove repmgr_node_config from group_vars * fix: wpb-22439 add repmgr_node_config to node specific inventory * fix: wpb-22439 add node specific repmgr_node_config to wiab-staging inventory * fix: wpb-22439 add changelog * fix: wpb-22439 add comment on hostname requirement directly for rmq-cluster group * wpb-22439 7 docs wiab staging (#851) * fix: wpb-21356 fix coturn port ranges * build: wpb-21356 add documentation for wiab-staging and clean old single_hetzner_machine_installation * build: wpb-21356 add changelog and small fix for offline-vm-setup.sh * fix: wpb-22439 memory requirements for VMs in wiab-staging.md doc * fix: wpb-22846 update wiab-staging documentation for hairpin networking * fix: wpb-22439 fix documentation typo * fix: wpb-22439 fix documentation suggestions * fix: wpb-22439 update wiab-staging documentation * wpb-22439 8 clean old wiab staging script and workflows (#852) * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * refactor: wpb-21356 remove old deploy-wiab workflow as it has already been incorporated in offline.yml * refactor: wpb-21356 remove old autodeploy.sh as offline/cd_staging.sh will replace it * refactor: wpb-21356 remove references of old autodeploy.sh, wiab-hetzner playbook and documentation * fix: wpb-22439 ansible/wiab-staging-provision.yml for ssh_private_key update * fix: wpb-22439 update cert-manager & hairpin NAT documentation
* fix: wpb-21356 offline-vm-setup script to use ubuntu cloud image, local seed iso and VM verification process * fix: wpb-21356 linting issues and push changelog * fix: wpb-22439 memory requirements for VMs in offline-vm-setup.sh * fix: wpb-22439 fix kubenode disk size * Update changelog.d/3-deploy-builds/wiab-staging * fix: wpb-22439 fix offline-vm-setup.sh * fix: wpb-22439 BASE_IMAGE_DIR value * wpb-22439 wiab-staging: ansible prepare host for vm provisioning (#846) * fix: wpb-21356 rename ansible templates for wiab-staging to make them generic * build: wpb-21356 wiab-staging VMs ansible inventory template to be populated by wiab-staging-provision.yml playbook * build: wpb-21356 ansible playbook for wiab-staging VM provisioning * build: wpb-21356 add ansible inventory for playbook wiab-staging-provision.yml * fix: wpb-21356 rebase from wpb-21356-update-vm-management * fix: wpb-21356 remove vars specific to Hetzner CI process * fix: wpb-22439 remove screen package from wiab-staging-provision.yml ansible playbook * fix: wpb-22439 simplify the wiab-staging inventory to use ansible children instead of creating unique nodes * fix: wpb-22439 ansible playbook wiab-staging-provision.yml to consider the inventory changes * fix: wpb-22439 postgresql_rw and ro group inventory * fix: wpb-22439 fix wiab-staging inventory elasticsearch group name * fix: wpb-22439 fix wiab-staging inventory elasticsearch group names * fix: wpb-22439 remove sshd config template as it is not functionally required * fix: wpb-22439 moved artifact hash to wiab-staging inventory * fix: wpb-22439 fix secondary inventory creation, organize playbook using tags and separate out wiab-staging-nftables playbook * fix: wpb-22439 allow for load balancing for ingress-controller, variable usage to define each k8s node and calling node and add comments * fix: wpb-22439 add wiab-staging-nftables.yaml playbook to manage nftables rules * fix: wpb-22439 allow rabbitmq ansible inventory for wiab-staging * wpb-22439 minor fixes for bin/offline-deploy.sh and cd.sh terraform inventory (#847) * fix: wpb-21356 rebase from wpb-21356-ansible-VM-provisioning * fix: wpb-21356 offline-deploy.sh for SSH_AUTH_SOCK handling * fix: wpb-21356 remove defunct passwords for postgresql * fix: wpb-22439 add terraform resources for wiab-staging (#848) * fix: wpb-21356 fix node types for assethost and adminhost * build: wpb-21356 add changelog for terraform wiab-staging resources * fix: wpb-22439 fix wiab-staging hetzner setup_nodes playbook to handle connection issue * build: wpb-22439 add cd_staging script (#849) * build: wpb-21356 add cd_staging script * build: wpb-21356 add changelog for cd_staging script * build: wpb-21356 enable creating secondary inventory for the VMs * fix: wpb-22439 cd_staging script as per wiab-staging inventory * fix: wpb-22439 fix secondary-inventory geenration logic to keep the original hostnames and bring SSH_OPTS * fix: wpb-22439 create a separate group for rabbitmq-hosts and keep other groups same * fix: wpb-22439 remove old children group for rmq-cluster before adding new one * fix: wpb-22439 fix postgresql_network_interface variable for postgresql * fix: wpb-22439 cd_staging.sh script to handle terrafrom nodenames for all the iventory groups * wpb-22439 6 GitHub workflow wiab staging (#850) * build: wpb-21356 changed the flow of offline.yml - introduced wiab-staging build and split bundle processing with default-build * build: wpb-21356 update pull_request_template and add new label for build-wiab-staging * fix: wpb-21356 offline/cd.sh for demo verification process as github workflow has been updated * fix: wpb-21356 add host key verification flags for scp commands * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * fix: wpb-22846 remove debug statements from cd.sh * fix: wpb-22439 remove new-line in cd_staging.sh * fix: wpb-22439 standardize SSH_OPTS in cd.sh * fix: wpb-22439 remove branch limitation for workflow * fix: wpb-22439 harden the ansible inventory params for default profile * Wpb 22439 postgresql repmgr node config (#856) * fix: wpb-22439 remove repmgr_node_config from group_vars * fix: wpb-22439 add repmgr_node_config to node specific inventory * fix: wpb-22439 add node specific repmgr_node_config to wiab-staging inventory * fix: wpb-22439 add changelog * fix: wpb-22439 add comment on hostname requirement directly for rmq-cluster group * wpb-22439 7 docs wiab staging (#851) * fix: wpb-21356 fix coturn port ranges * build: wpb-21356 add documentation for wiab-staging and clean old single_hetzner_machine_installation * build: wpb-21356 add changelog and small fix for offline-vm-setup.sh * fix: wpb-22439 memory requirements for VMs in wiab-staging.md doc * fix: wpb-22846 update wiab-staging documentation for hairpin networking * fix: wpb-22439 fix documentation typo * fix: wpb-22439 fix documentation suggestions * fix: wpb-22439 update wiab-staging documentation * wpb-22439 8 clean old wiab staging script and workflows (#852) * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * refactor: wpb-21356 remove old deploy-wiab workflow as it has already been incorporated in offline.yml * refactor: wpb-21356 remove old autodeploy.sh as offline/cd_staging.sh will replace it * refactor: wpb-21356 remove references of old autodeploy.sh, wiab-hetzner playbook and documentation * fix: wpb-22439 ansible/wiab-staging-provision.yml for ssh_private_key update * fix: wpb-22439 update cert-manager & hairpin NAT documentation --------- Co-authored-by: Julia Longtin <julia.longtin@wire.com>
* fix: wpb-22439 enable rabbitmq deployment via ansible * fix: wpb-22439 fix static inventory for rabbitmq to remove nodename dependency * fix: wpb-22439 enable deploying rabbitmq-external and stop deploying rabbitmq chart * fix: wpb-22439 enable creating dedicated nodes for rabbitmq VMs * fix: wpb-22439 fix rmq-cluster inventory group name for rabbitmq terraform created nodes * fix: wpb-22439 include rmq-cluster nodes for networking fix * fix: wpb-22439 refactor node names in hetzner terraform for rabbitmq * fix: wpb-22439 offline-vm-setup script to manage VMs wiab-staging (#845) * fix: wpb-21356 offline-vm-setup script to use ubuntu cloud image, local seed iso and VM verification process * fix: wpb-21356 linting issues and push changelog * fix: wpb-22439 memory requirements for VMs in offline-vm-setup.sh * fix: wpb-22439 fix kubenode disk size * Update changelog.d/3-deploy-builds/wiab-staging * fix: wpb-22439 fix offline-vm-setup.sh * fix: wpb-22439 BASE_IMAGE_DIR value * wpb-22439 wiab-staging: ansible prepare host for vm provisioning (#846) * fix: wpb-21356 rename ansible templates for wiab-staging to make them generic * build: wpb-21356 wiab-staging VMs ansible inventory template to be populated by wiab-staging-provision.yml playbook * build: wpb-21356 ansible playbook for wiab-staging VM provisioning * build: wpb-21356 add ansible inventory for playbook wiab-staging-provision.yml * fix: wpb-21356 rebase from wpb-21356-update-vm-management * fix: wpb-21356 remove vars specific to Hetzner CI process * fix: wpb-22439 remove screen package from wiab-staging-provision.yml ansible playbook * fix: wpb-22439 simplify the wiab-staging inventory to use ansible children instead of creating unique nodes * fix: wpb-22439 ansible playbook wiab-staging-provision.yml to consider the inventory changes * fix: wpb-22439 postgresql_rw and ro group inventory * fix: wpb-22439 fix wiab-staging inventory elasticsearch group name * fix: wpb-22439 fix wiab-staging inventory elasticsearch group names * fix: wpb-22439 remove sshd config template as it is not functionally required * fix: wpb-22439 moved artifact hash to wiab-staging inventory * fix: wpb-22439 fix secondary inventory creation, organize playbook using tags and separate out wiab-staging-nftables playbook * fix: wpb-22439 allow for load balancing for ingress-controller, variable usage to define each k8s node and calling node and add comments * fix: wpb-22439 add wiab-staging-nftables.yaml playbook to manage nftables rules * fix: wpb-22439 allow rabbitmq ansible inventory for wiab-staging * wpb-22439 minor fixes for bin/offline-deploy.sh and cd.sh terraform inventory (#847) * fix: wpb-21356 rebase from wpb-21356-ansible-VM-provisioning * fix: wpb-21356 offline-deploy.sh for SSH_AUTH_SOCK handling * fix: wpb-21356 remove defunct passwords for postgresql * fix: wpb-22439 add terraform resources for wiab-staging (#848) * fix: wpb-21356 fix node types for assethost and adminhost * build: wpb-21356 add changelog for terraform wiab-staging resources * fix: wpb-22439 fix wiab-staging hetzner setup_nodes playbook to handle connection issue * build: wpb-22439 add cd_staging script (#849) * build: wpb-21356 add cd_staging script * build: wpb-21356 add changelog for cd_staging script * build: wpb-21356 enable creating secondary inventory for the VMs * fix: wpb-22439 cd_staging script as per wiab-staging inventory * fix: wpb-22439 fix secondary-inventory geenration logic to keep the original hostnames and bring SSH_OPTS * fix: wpb-22439 create a separate group for rabbitmq-hosts and keep other groups same * fix: wpb-22439 remove old children group for rmq-cluster before adding new one * fix: wpb-22439 fix postgresql_network_interface variable for postgresql * fix: wpb-22439 cd_staging.sh script to handle terrafrom nodenames for all the iventory groups * wpb-22439 6 GitHub workflow wiab staging (#850) * build: wpb-21356 changed the flow of offline.yml - introduced wiab-staging build and split bundle processing with default-build * build: wpb-21356 update pull_request_template and add new label for build-wiab-staging * fix: wpb-21356 offline/cd.sh for demo verification process as github workflow has been updated * fix: wpb-21356 add host key verification flags for scp commands * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * fix: wpb-22846 remove debug statements from cd.sh * fix: wpb-22439 remove new-line in cd_staging.sh * fix: wpb-22439 standardize SSH_OPTS in cd.sh * fix: wpb-22439 remove branch limitation for workflow * fix: wpb-22439 harden the ansible inventory params for default profile * Wpb 22439 postgresql repmgr node config (#856) * fix: wpb-22439 remove repmgr_node_config from group_vars * fix: wpb-22439 add repmgr_node_config to node specific inventory * fix: wpb-22439 add node specific repmgr_node_config to wiab-staging inventory * fix: wpb-22439 add changelog * fix: wpb-22439 add comment on hostname requirement directly for rmq-cluster group * wpb-22439 7 docs wiab staging (#851) * fix: wpb-21356 fix coturn port ranges * build: wpb-21356 add documentation for wiab-staging and clean old single_hetzner_machine_installation * build: wpb-21356 add changelog and small fix for offline-vm-setup.sh * fix: wpb-22439 memory requirements for VMs in wiab-staging.md doc * fix: wpb-22846 update wiab-staging documentation for hairpin networking * fix: wpb-22439 fix documentation typo * fix: wpb-22439 fix documentation suggestions * fix: wpb-22439 update wiab-staging documentation * wpb-22439 8 clean old wiab staging script and workflows (#852) * fix: wpb-21356 debugging ssh banner issues * fix: wpb-21356 disable flows on push and removing the need for the default-build for verify steps * fix: wpb-21356 remove static build hashes and enabled bundle build verification in offline.yml workflow * refactor: wpb-21356 remove old deploy-wiab workflow as it has already been incorporated in offline.yml * refactor: wpb-21356 remove old autodeploy.sh as offline/cd_staging.sh will replace it * refactor: wpb-21356 remove references of old autodeploy.sh, wiab-hetzner playbook and documentation * fix: wpb-22439 ansible/wiab-staging-provision.yml for ssh_private_key update * fix: wpb-22439 update cert-manager & hairpin NAT documentation --------- Co-authored-by: Julia Longtin <julia.longtin@wire.com> * fix: wpb-23462: remove unused vars from cd.sh and cd_staging.sh * fix: wpb-23462: fix linting issues for cd_staging.sh * fix: wpb-22439 SSH_OPTS var in cd_staging.sh and ignore linting in shellcheck.sh * fix: wpb-22439 update the wiab-staging.yml with the latest artifact hash --------- Co-authored-by: Julia Longtin <julia.longtin@wire.com>



Summary of changes :
This ticket is part of cascading PRs:
#852 - clean old wiab staging script and workflows
#853 - Testing wiab-staging solution for wire-server-5.14
No labels:
The changes in the PR will be verified in #853
Change type
Basic information
Testing
Offline Build CI (label-based)
Add one or more labels to trigger offline builds:
build-default- Full production build (ansible, terraform, all packages)build-demo- Demo/WIAB buildbuild-min- Minimal build (fastest, essential charts only)build-all- Run all three buildsNote: No builds run by default. Add a label to trigger CI.
Tracking
changelog.dKnowledge Transfer
Motivation
Objective
Reason
Use case