Nutanix CE Cluster Build

Wed, May 22, 2019 3-minute read

I was slowly purchasing used equipment to build out a storage cluster to replace an underpowered whitebox NAS. While the hardware was sitting around waiting for me to purchase hard drives for the storage cluster, I decided it would be fun to install Nutanix CE to get a better feel for the same tools we use at work.

Overall

Overall, Nutanix CE has performed quite well on these systems. They are, however, not the best choice for getting the best experience out of Nutanix. The HP Z600s are capped at 24 GBs of RAM, which highly limits the number of workloads the cluster can run, especially if Prism Central is installed.

Each CVM (controller virtual machine) requires a minimum of 12 GBs of RAM. I ended overridding the minimum down to 8 GBs to provide more RAM availablity; at 8 GBs of RAM for the CVMs, Prism Central will install, but will not start up, because it requires 16 GBs of RAM – which is more than a single node could provide. Not being able to run Prism Central reduces some of the features available, it specifically limits behind able to use terraform to manage the virtual machines.

Hardware

Below is a list of the hardware I ended up using for the running cluster – as of 2019-05-22:

I had a spare Dell T710 running a single L5640 with 20 GBs of RAM instead of one of the HP Z600s. The Dell was much better suited to the cluster and I had purchased an additional processor for the system, however the Dell turned out to be too noisy for use around my desk, and ultimately was swapped out for a quieter HP Z600.

Build Specific steps


I am not going to document where and how to put Nutanix CE on a USB 3 flash drive – there are plenty of guides to do similar things.
This first time I built the cluster I took the long way around to get the old network inferface names. The shorter version is much easier and quicker.

Set interface names
  1. Edit Grub - vi /etc/default/grub
  2. Add net.ifnames=0 biosdevname=0 to the line starting with GRUB_CMDLINE_LINUX.
  3. Regenerate grub - grub2-mkconfig -o /boot/grub2/grub.cfg
  4. Reboot
  5. Verify with ip addr
Reduce memory for Controller Virtual Machines (CVM)

It is best to reduce the CVM memory before creating a cluster. Otherwise the cluster will need to be shutdown, before any change to the CVM or host can be made.
I would not recommend less than 8 GBs of RAM for the CVM otherwise it will not start all the services.

  1. SSH into the Nutanix node (not the CVM).
  2. List the running VMs to get the VM name
virsh list --all
  1. Shutdown the CVM (This could take couple of minutes).
virsh shutdown VMNAME
  1. Set the memory (8G references 8 GBs of memory)
virsh setmem VMANME 8G --config
''`<br/>  
5. Set max memory 
```console
virsh setmaxmem VMNAME 8G --config
  1. Start the virtual machine
virsh start VMNAME
  1. Verify VM starts and memory changes take affect
virsh dominfo VMNAME