Clarification on CloudStack VM Storage Architecture (NFS Primary Storage vs KVM Host Local Storage)

[arpanbht]

Hello everyone,

I have a query regarding how storage is managed for virtual machines in Apache CloudStack, specifically concerning the location of root and data disks for running VMs.

In our private CloudStack setup, we have followed the official documentation to deploy the infrastructure, which includes:

• A CloudStack Management Server
• Multiple KVM Host machines
• An NFS server configured as primary storage

When we create VMs on a KVM host, we observe that the local storage of the host machine is not being utilized. Below is a sample output from one of our KVM hosts:

compute1@compute1:~$ df -h
Filesystem                     Size  Used Avail Use% Mounted on
tmpfs                           13G  4.8M   13G   1% /run
/dev/sda2                      878G   15G  818G   2% /
tmpfs                           63G     0   63G   0% /dev/shm
tmpfs                          5.0M     0  5.0M   0% /run/lock
tmpfs                           63G     0   63G   0% /run/qemu
/dev/sda1                      1.1G  6.1M  1.1G   1% /boot/efi
172.16.17.202:/export/primary  3.5T  2.7T  701G  80% /mnt/81ecd9fd-9cdf-34cb-b99d-51d0e2cf3a0c
tmpfs                           13G  4.0K   13G   1% /run/user/1000

From this, we can see that:

• The NFS primary storage is mounted on the host (/mnt/...)
• The local disk of the host remains largely unused (~15 GB utilized)

Based on this observation, our understanding is that:

• The root and data disks of VMs are being stored entirely on the NFS-based primary storage
• The KVM host’s local storage is not used for VM disk storage

We would appreciate confirmation on whether this understanding is correct.

Additionally, we have a few concerns and questions:

1. Since all VM disk I/O appears to depend on the NFS primary storage, could this become a performance bottleneck at scale?
2. Is there a supported way in CloudStack to:

• Use local storage on KVM hosts for VM root/data disks?
• Or configure a hybrid approach (e.g., some VMs on NFS, others on local storage)?

3. If local storage can be used:

• What changes are required in storage configuration or zone setup?
• Are there any trade-offs (e.g., migration limitations, HA constraints)?

We are trying to evaluate whether our current architecture is optimal for long-term scalability and performance.

Any guidance, best practices, or references would be highly appreciated.

[weizhouapache]

@arpanbht

you can create a new service offering, and select the storage type: Shared or Local

refer to https://docs.cloudstack.apache.org/en/latest/adminguide/service_offerings.html#creating-a-new-compute-offering

Storage type is also available when create a disk offering