High Availability and Autoscaling

[type-checker]

Hi @biersoeckli , great work! I have some queries related to QuickStack:

• How to setup High Availability K3s cluster which provisions three control plane?
• How does Autoscaling works for both K3s control plane and worker node for apps and services? Do we have any Horizontal Pod Autoscaling (HPA) and Cluster Autoscaler setup for this?
• And are we setting up and load balancer?
• Can we do the deployment using HELM?

[biersoeckli]

Hi @type-checker

Thanks for reaching out.

- How to setup High Availability K3s cluster which provisions three control plane?
To setup a HA cluster you need to setup the first Node with the QuickStack installation script and then follow the setup steps on the official k3s website. There are two options available (https://docs.k3s.io/architecture#high-availability-k3s):
--> Datastore Embedded DB (https://docs.k3s.io/datastore/ha-embedded)
--> Datastore External DB (https://docs.k3s.io/datastore/ha)

Use one of these guides to setup a HA cluster and also install nfs-common on every node (for ReadWriteMany PVCs). If you use the setup script for worker nodes from QuickStack you won't configure a HA cluster.

- How does Autoscaling works for both K3s control plane and worker node for apps and services? Do we have any Horizontal Pod Autoscaling (HPA) and Cluster Autoscaler setup for this?
--> Horizontal Pod Autoscaling (HPA): I'm not sure, but I believe that Pod Autoscaling is not available with k3s. Therefore, it is not supported by QuickStack to configure it.
--> Cluster Autoscaler: QuickStack itself doesn't support Cluster autoscaling. To do this you could take a look at the Rancher Autoscaler: https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/manage-clusters/install-cluster-autoscaler

- And are we setting up and load balancer?
The default Loadbalancer (ServiceLB) from k3s is configured when you install QuickStack (https://docs.k3s.io/networking/networking-services#service-load-balancer). You could extend k3s with any other load balancer like Metallb if you want, but i haven't testet this yet.

- Can we do the deployment using HELM?
Currently its not possible to install QuickStack using HELM in a Kubernetes cluster. If you look at the install script (https://github.com/biersoeckli/QuickStack/blob/main/setup/setup.sh) currently just k3s, cert-manager, nfs-common and longhorn is installed and after that the QuickStack pod is started using the env variable START_MODE=setup. In theory it is possible to run QuickStack on any Kubernetes Cluster, where Traefik and longhorn is configured.

I hope this helps :)

[type-checker]

Hi @biersoeckli , Thank you for responding to my queries!

• Where does QuickStack saves it's state, does it uses any DB or it syncs the states in s3? And if I've to update K3s or longhorn or any defaults from setup.sh, is there any single script or interface available for the same?
• Can services auto-scale, or do we need to set up our own solution like a KEDA scaler?
• I was asking about deploying services through Helm after setting up QuickStack, specifically whether CloudNativePG needs to be deployed via Helm.
• I'm a bit confused about the load balancer. I was asking whether we configure a cloud load balancer for ingress, like we typically do, or if we can also use Cloudflare for load balancing (not ideal, but I’ve experimented with it). Do we have any setup for that?

I am not familiar with this (https://docs.k3s.io/networking/networking-services#service-load-balancer), doe this work same as cloud load balancer?

• Is NGINX ingress support available?

[biersoeckli]

Hi @type-checker

Where does QuickStack saves it's state, does it uses any DB or it syncs the states in s3?

• QuickStack itself is a Next.JS applications wich runs within the Kubernetes Cluster. It uses a SQLite Database wich is stored on a Persisten Volume Claim (PVC). The database contains configuration stuff like the URL to a Git Repo or Docker image of an app, Environment Variables of an app, Information about wich Volumes have to be created for an app and so on as well as the users in QuickStack. All status information and logs are consumed directly through the Kubernetes API.

And if I've to update K3s or longhorn or any defaults from setup.sh, is there any single script or interface available for the same?

• Currently the following versions are "hardcoded" in the setup script:
  --> k3s: v1.31.3+k3s1
  --> lognhorn: v1.7.2

In the future the idea is to update k3s and longhorn on a running QuickStack instance using the Kubernetes API through QuickStack. So when you update QuickStack it automatically checks the version of the installed k3s and longhorn and then initializes an upgrade. Both tools offer automated upgrades (k3s: https://docs.k3s.io/upgrades/automated#install-the-system-upgrade-controller, longhorn: https://longhorn.io/docs/1.8.1/deploy/upgrade/longhorn-manager/#upgrade-with-kubectl)

Upgrades for longhorn or k3s wich are initiated by QuickStack are not implemented yet and I haven't testet it until now.

To update the default values for the setup script, updates to the setup script and a pull request is needed.

Can services auto-scale, or do we need to set up our own solution like a KEDA scaler?

• Because k3s runs under the hood, a tool like KEDA is needed to approach this. Other Kubernetes distributions may include an auto-scale functionality.

I was asking about deploying services through Helm after setting up QuickStack, specifically whether CloudNativePG needs to be deployed via Helm.

• Currently only deployments of "single container images" are possible. Helm Deployments are currently not supported through the QuickStack UI, but i think this would also be interesting. Feel free to contribute your idea to QuickStack by creating an issue and/or pull request on this topic :)

I'm a bit confused about the load balancer. I was asking whether we configure a cloud load balancer for ingress, like we typically do, or if we can also use Cloudflare for load balancing (not ideal, but I’ve experimented with it). Do we have any setup for that?

• The naming of the ServiceLB is a bit confusing. ServiceLB just routes the nodeports 443 and 80 traffic to the kubernetes ingress controller. So currently if you install QuickStack on a VPS all traffic on the nodeport 80 and 443 will be forwarded to the internal traefik ingress controller and then to the specified app/service. In a multi-node cluster, traffic can enter through any node (port 80 or 443), and it will still be correctly routed to the designated application within the cluster. For example if you use hetzner as provider, you create a private network where all your nodes are in and then use a hetzner loadbalancer to distribute the traffic over all nodes. Currently there is no guide in the docs for this specific setup. I haven't used Cloudflare load balancing yet, but I think it could also be used.

In this image the "Internet" can be replaced with a load balancer.

Is NGINX ingress support available?

• Using NGINX instead of Traefik as Ingress Controller is not possible, because QuickStack depends on Traefik when creating Ingress configurations for apps in QuickStack.

I hope that this answers your questions and if not, feel free to ask :)

[type-checker]

Hi @biersoeckli

Hi, could you please help me with a guide for setting up a high-availability server installation with WireGuard configured using QuickStack? It would be great if this could be made an optional step during the QuickStack installation process.

Additionally, I’ve noticed that workloads are currently being distributed randomly between the server node and the worker node. If possible, it would be helpful to have an option to restrict workloads to only be scheduled on worker nodes.

Could you also share how to view QuickStack logs, and how to access dashboards for Longhorn and metrics? And how to take backup of QuickStack DB in object storage (I'm using GCP).

Lastly, it would be great to have domain support for the server, so that we can connect to the cluster from any device using kubeconfig.

PS: I attempted to set up a high-availability (HA) cluster, but I ran into an error loop during the process. Also, I noticed that Nginx appears to perform faster than Traefik, and I was curious to understand why Traefik is still being used as the default.

The Docker-based deployment works really well, but it would be super helpful to also have support for Helm-based deployment as an additional option.

[type-checker]

Hi @biersoeckli, when you get time, could you plz. check my above request.