Reclaiming VM Hard Disk Space from VirtualBox
Creating a virtual machine requires specifying the amount of space to allocate for the virtual hard disk as one of the first steps. Dreading the notorious "not enough disk space" message, one usually aims to be on the safe side and reserves a generous amount at this stage. If the assignment is done in a static fashion then the allocated space is immediately claimed from the underlying file system and one thus has to come up with a pretty good guess in order not to waste space. Thankfully most VM solutions allow dynamic allocations which can grow on demand limited only by an upper bound making this a very attractive choice.
A newly created VM with such an dynamic disk image indeed starts out nice and space saving but over time the image will eventually take up all the committed underlying disk space even though the VM reports to use only a subset of it. Reflecting on why this is the case leads us to an elegant setup reclaiming unused storage space on the host system.
Classic Hard Disks
Classic hard disks have a fixed capacity and thus always present the same amount of available space to the system. There is no concept of used or unused space. Every block can be written to and read from but there is no explicit "erase" operation. Erasing something on a hard disk usually means writing a fixed pattern like all zeros to a specific block. However, the hard disk itself cannot distinguish this operation from any other write operation. Indeed the original Parallel ATA standard ANSI X3.221-1994 talks mainly about "read" and "write" operations [1] .
So only the file system inside the operating system is keeping track whether a block on the hard disk is free or in active use. For ancient file systems like FAT, this would not be a big problem as they try to reuse blocks marked as erased, but modern file systems are usually journaling file systems using the underlying storage more as a circular log. More specifically, the Linux ext3 and ext4 file systems are log based just as Windows' NTFS is.
All this leads to the situation where all blocks of the hard disk will eventually be written to and even if a specific block is not in use, this is only recorded in by marking it in internal data structures.
So we now understand why dynamic disk images will eventually grow to the allowed size. Of course the guest OS will tell you that only a fraction of the space is in use, but the host OS will see the disk image files having their maximum size.
Luckily nowadays there is an elegant solution to this inefficient handling of precious disk space.
Solid State Drives (SSD)
Even though Solid State Drives were already conceived in 1978, they became commodity hardware only recently. Compared to classic hard disks, SSDs do not use rotating magnetic media but typically use flash memory to persistently store data. It quickly became apparent that it would be beneficial for the operating system to be able to tell the storage medium its intent to discard blocks. To that end the TRIM commands have been introduced in the lower layers and are used by SSD controllers to implement their wear leveling algorithm. These algorithms aim to "wear out" flash cells evenly by spreading erase/program cycles over all available memory. As flash storage guarantees only a finite amount of erase/program cycles (a few thousand nowadays) compared to the roughly infinite amount of rewrites possible in rotational magnetic hard disks, this handling is essential to efficiently use SSDs.
The rest of this post shows how all of this can be used very nicely to allow VirtualBox guests to compress the underlying dynamic volumes without using any external tools. Without this little trick it was common wisdom to do manual "garbage collection" cycles as described for example here. The intent of those instructions is to writes zeros to all hard disk blocks not in use from within the guest and then uses tools of the VM solution to find and remove those from the dynamic volumes.
Our solution pretends to the guest system that the storage medium is capable of discarding storage blocks through the TRIM commands described above. The guest file system then issues such calls and VirtualBox can then erase those blocks from the dynamic volume and thus effectively compress them. This works for both GNU/Linux and Windows guests.
VirtualBox trickery
Although VirtualBox allows the machine configuration to specify a storage medium to be a "Solid-state Drive", this is not enough for what we want. The guest OS will only generate TRIM commands when the volume also carries the "discard" attribute. Even though this cannot be set in the GUI, it is straight forward to enable this is the .vbox file. Here is an example diff for the required change:
$ diff -u Ubuntu\ GNOME\ 18.04.vbox-ORIG Ubuntu\ GNOME\ 18.04.vbox --- "Ubuntu GNOME 18.04.vbox-ORIG" 2019-08-22 15:13:35.362777400 +0200 +++ "Ubuntu GNOME 18.04.vbox" 2019-08-22 15:14:12.076301600 +0200 @@ -65,7 +65,7 @@ </AttachedDevice> </StorageController> <StorageController name="SATA" type="AHCI" PortCount="1" useHostIOCache="false" Bootable="true" IDE0MasterEmulationPort="0" IDE0SlaveEmulationPort="1" IDE1MasterEmulationPort="2" IDE1SlaveEmulationPort="3"> - <AttachedDevice nonrotational="true" type="HardDisk" hotpluggable="false" port="0" device="0"> + <AttachedDevice nonrotational="true" discard="true" type="HardDisk" hotpluggable="false" port="0" device="0"> <Image uuid="{1dd76dc5-ebf4-4401-b29f-7f5b62f8be1d}"/> </AttachedDevice> </StorageController>
Note that you need to shut down the VM and VirtualBox itself to be able to edit the .vbox file. Failing to do this may cancel the edit as VirtualBox seems to rewrite the .vbox file upon exit. To make sure your edit is effective, close the VM and VirtualBox and then make sure the addition is still in the file.
GNU/Linux Guest
In GNU/Linux the required property can be checked with "hdparm". The transcript shows the command to query the parameter after changing the vbox file and a diff against the saved output from before the change:
ubuntu@ubuntu-vm:~$ sudo hdparm -I /dev/sda > hdparm-sda-discard [sudo] password for ubuntu: ubuntu@ubuntu-vm:~$ diff -u hdparm-sda-nodiscard hdparm-sda-discard --- hdparm-sda-nodiscard 2019-08-22 15:08:42.020000000 +0200 +++ hdparm-sda-discard 2019-08-22 15:16:07.932449511 +0200 @@ -42,4 +42,5 @@ * FLUSH_CACHE_EXT * Gen2 signaling speed (3.0Gb/s) * Native Command Queueing (NCQ) + * Data Set Management TRIM supported (limit unknown) Checksum: correct ubuntu@ubuntu-vm:~$
Once the infrastructure is in place, we have two ways to actually use it. We can either enable the "discard" mount option for ext4/5 to continuously generate TRIM calls whenever needed or we can trigger a garbage collection manually by invoking "fstrim". Adjusting the mount options is left as an exercise to the reader, calling "fstrim" on the other hand is a no-brainer:
Windows 10 Guest
In a Windows guest machein one can check if the TRIM commands will be generated using "fsutil":
C:\Users\admin>fsutil behavior query DisableDeleteNotify NTFS DisableDeleteNotify = 0 (Deaktiviert) ReFS DisableDeleteNotify = 0 (Deaktiviert) C:\Users\admin>
I guess a clear thinking developer would have packaged the functionality into a "DeleteNotify" option that can be enabled, but for unknown reasons Microsoft developers decided we need to disable the "DisableDeleteNotify" option instead, ho hum. If not already disabled, the option can be changed in an (admin) command prompt like this:
C:\WINDOWS\system32>fsutil behavior set DisableDeleteNotify 0 NTFS DisableDeleteNotify = 0 (Deaktiviert) C:\WINDOWS\system32>
With everything setup correctly, the tool to optimize and de-fragment the drive will eventually trigger the TRIM commands to eventually shrink the underlying dynamic volume. The process can be seen in the following screen shot:
Summary
A simple XML change in the VBOX file of a VM allows us to elegantly handle the unused space in dynamically allocated virtual hard disks. For a GNU/Linux guest, a simple "fstrim" invocation triggers all the machinery. Compared to using "zerofree" and "VBoxManage" this saves a lot of time and hassle as "zerofree" only works on read-only mounted files systems and thus usually requires a recovery mode boot in order to work on the root file system.
Update 2020-01-22
Editing the .vbox file requires shutting down VirtualBox. The text has been amended to make this clear.
Footnotes
Comments
Comments powered by Disqus