feat: etcd reunion procedure

[alexarefev]

Description

• Provide an automation procedure to remove and add etcd member. It might be useful in case of corruption one of etcd members.

Solution

• Add new KubeMarine procedure
• Implement particular steps to remove and add corrupted etcd member
• Add an article into maintenance guide

Test Cases

TestCase 1
Check if the reunion procedure is working (all members are up and running)

Test Configuration:

• Hardware: 4CPU/8GB
• OS: Ubuntu 22.04
• Inventory: miniHA

Steps:

1. Install K8S
2. Run the kubemarine reunion procedure

Results:

Before	After
Not applicable	The etcd cluster is healthy and all of k8s cluster nodes are ready

TestCase 2
Check if the reunion procedure is working (one member is corrupted)

Test Configuration:

• Hardware: 4CPU/8GB
• OS: Ubuntu 22.04
• Inventory: miniHA

Steps:

1. Install K8S
2. Run the following commands on one of the control plane nodes:

$ mv /etc/kubernetes/manifests/etcd.yaml /etc/kubernetes/tmp/
$ dd if=/dev/urandom of=/var/lib/etcd/member/snap/db bs=1k count=100 seek=100 conv=notrunc
$ mv /etc/kubernetes/tmp/etcd /etc/kubernetes/manifests/etcd.yaml

3. Set that node as corrupted in procedure.yaml
4. Run the kubemarine reunion procedure

Results:

Before	After
Not applicable	The etcd cluster is healthy and all of k8s cluster nodes are ready

Checklist

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• Integration CI passed
• There is no merge conflicts

[disa1217]

Hi @alexarefev ,
reviewed.