Monitoring host-swapping activity
Excessive memory demand can cause severe performance problems for one or more virtual machines on an ESXi host. When ESXi is actively swapping from the memory of a virtual machine to disk, the performance of that virtual machine will degrade. The overhead of swapping a virtual machine's memory to disk can also degrade the performance of other virtual machines, because the virtual machine expects to be writing to RAM (speeds measured in nanoseconds) but it is unknowingly writing to disk (speeds measured in milliseconds).
The metrics in the vSphere Client for monitoring swapping activity are the following:
- Memory Swap In rate: The rate at which memory is being swapped in from disk.
- Memory Swap Out rate: The rate at which memory is being swapped out to disk.
- Swapped: The total amount of data that is sitting inside the
.vswphypervisor-level swap file. However, this doesn't tell you anything about the current state of performance, nor about the current state of free pRAM. It just tells you that at some point in the past, there was low free pRAM. The only use of this metric is to see if there was a memory shortage in the past, and to see if there's a risk of Swap In (and bad performance) in the future. But it's hard to estimate how likely the Swap In will be, because sometimes there's "junk data" that will be rarely requested again (for example, pages holding rarely-accessed data, like an overly aggressive filesystem caching, or zeroed-out pages that were actually deemed free by the guest operating system).
High values for metric indicate a lack of memory and that performance is suffering as a result.
However, a high Swap Out rate means that the host is low on free pRAM, but essentially it does not indicate a current performance problem. A high Swap In rate indicates a current performance problem but not necessarily that the host is currently low on free pRAM (Swap In happens only on demand, which could be a week after the data was swapped out, and maybe the free-host-memory shortage has long since been resolved).
Getting ready
To step through this recipe, you will need a running ESXi Server, a couple of memory-hungry Virtual Machines, a vCenter Server, and a working installation of vSphere Client. No other prerequisites are required.
How to do it...
To monitor a host's swapping activity in vSphere Client, view the Memory screen. A couple of useful metrics are Swapped, Swap In, and Swap Out. This metric represents total swapping for a virtual machine on the host.
To monitor host-level swapping activity for all virtual machines, you need to select one single ESXi Host and then go to the Performance tab. There, you need to go to the Memory tab, which will show the entire VMs. After this is done, perform the following steps:
- Open vSphere Client.
- Log in to the vCenter Server.
- On your Home screen, select VMs and Templates.
- Choose the memory-hungry.
- Go to the Performance tab, and switch to the Memory screen.
- Click on the Advanced tab, and select Chart Options.
- Select the three values there, that is, Swapped, Swap in Rate in KBps, and Swap Out Rate in KBps.
- Click on OK. Now you should see something similar to the following image, if there are swapping activities on your VM:
In the preceding example, you can see that this VM has swapped 4372 Kilobytes, Swap out for 1012 Kilobytes, and Swap in for 40 Kilobytes.
Now to see the overall swapped situation for all VMs in a single ESXi server, you should perform the following steps:
- Open vSphere Client.
- Log in to the vCenter Server.
- On your Home screen, select Hosts and Clusters.
- Choose the poorly-performing ESXi host.
- Go to the Performance tab, and switch to the Memory screen.
- Click on the Advanced tab, and select Chart Options.
- Select the three values there, that is, Swap Used, Swap In, and Swap Out rate in KBps.
- Click on OK. Now you should see something similar to the following image if there are swapping activities on your ESXi host:
In this example, you can see that this ESXi host Swap in for 1657976 Kilobytes and Swap out for 4930940 Kilobytes.
How it works...
We talked about how it works in the previous recipe. Please refer to the Virtual memory reclamation technique recipe.
There's more...
The basic cause of host-level swapping is memory overcommitment from using memory intensive virtual machines whose combined configured memory is greater than the amount of host physical memory available.
To resolve this problem, you should consider the following factors:
- Reduce the level of memory overcommitment (for example, reduce oversized VM's, and add more memory where necessary, add more hosts to the cluster if necessary, and so on)
- Enable the balloon driver in all virtual machines by installing VMware tools
- Reduce memory Reservations. This is covered in Chapter 5, vSphere Cluster Design
- Use resource controls (Shares, Reservations, and Limits) with careful understanding and planning, to reduce memory contention on the most critical virtual machines
Note that this will not reduce the total memory contention on this host, but it allows you to mitigate the risk to the performance of your most important VMs.