Btrfs and Compression

2026-10-26 22:05

Detlev Zundel

Tags:

The btrfs filesystem sports some real cool features, like snapshots, automatic de-duplication and automatic compression. This post will look at the latter one, i.e. automatic file compression on file system level. This is a very cool feature, so let's just see a short console transcript of interacting with it.

In the test I use XML files from Garmin GPS devices, usually exported in an XML based TCX file. As these are text format files, they allow very high compression ratios to be achieved. So I created a fresh directory xml-files, copied some 100 MiB of data into it:

dzu@krikkit:~/fs-test/xml-files$ du -hs .
104M	.
dzu@krikkit:~/fs-test/xml-files$

Ok, so we really are looking at 104 MiB in our folder. Just for fun let's run a text utility against it, i.e. count the number of lines in them:

dzu@krikkit:~/fs-test/xml-files$ time wc -c * > /dev/null

real	0m0,002s
user	0m0,002s
sys	0m0,000s
dzu@krikkit:~/fs-test/xml-files$

Ok, those times are negligible. It can only show that the numbers will be in the same ball park, but not more. But hey, this is good enough for the quick little test.

So now we need to use a special filesystem tool to check for the real usage of the files. For btrfs, this is compsize:

dzu@krikkit:~/fs-test/xml-files$ sudo compsize *
Processed 37 files, 37 regular extents (37 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL      100%      103M         103M         103M       
none       100%      103M         103M         103M       
dzu@krikkit:~/fs-test/xml-files$

Ok, btrfs tells us, that Disk Usage = Uncompressed, i.e. there is no compression happening. We need another tool to set the flag on this directory. This tool is btrfs filseystem defrag -czstd (-verbose -recursive):

dzu@krikkit:~/fs-test/xml-files$ btrfs filesystem defrag -vr -czstd .
./2019-03-30_13-03-18.TCX
./2019-04-07_14-22-04.TCX
./2019-05-23_19-19-09.TCX
./2019-05-24_17-58-54.TCX
./2019-06-08_14-37-33.TCX
./2019-06-09_07-23-34.TCX
./2019-06-18_10-21-38.TCX
./2019-06-25_20-34-12.TCX
./2019-06-29_13-17-40.TCX
./2019-06-30_10-32-27.TCX
./2019-07-02_17-14-21.TCX
./2019-07-06_12-13-45.TCX
./2019-07-13_17-51-09.TCX
./2019-07-19_16-51-43.TCX
./2019-07-22_09-24-41.TCX
./2019-07-24_17-07-56.TCX
./2019-07-29_17-20-05.TCX
./2019-07-31_17-11-10.TCX
./2019-08-04_15-04-08.TCX
./2019-08-06_17-12-20.TCX
./2019-08-09_08-58-09.TCX
./2019-08-13_17-17-15.TCX
./2019-08-14_17-17-47.TCX
./2019-08-21_18-02-50.TCX
./2019-08-25_07-45-44.TCX
./2019-08-27_12-35-23.TCX
./2019-08-28_10-08-50.TCX
./2019-08-30_11-08-45.TCX
./2019-09-03_17-12-50.TCX
./2019-09-11_17-10-09.TCX
./2019-10-03_11-57-26.TCX
./2019-10-07_17-05-34.TCX
./2019-10-13_12-55-04.TCX
./2019-10-15_10-30-19.TCX
./2019-10-17_16-02-50.TCX
./2019-10-20_10-52-02.TCX
./2019-10-25_13-26-28.TCX
dzu@krikkit:~/fs-test/xml-files$

The files have been compressed now and the flag for future operations is set on the directory. Let's recheck the usage:

dzu@krikkit:~/fs-test/xml-files$ sudo compsize *
Processed 37 files, 851 regular extents (851 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL        6%      6.6M         103M         103M       
none       100%      8.0K         8.0K         8.0K       
zstd         6%      6.5M         103M         103M       
dzu@krikkit:~/fs-test/xml-files$

Wow, the whole directory has shrunk to 6.6 MiB in size. That's 6 percent of the original space. Quite impressive!

Just for fun, let's see if we see a speed difference when using the files as text files, i.e. they will be uncompresseed at the file system layer:

dzu@krikkit:~/fs-test/xml-files$ time wc -c * > /dev/null

real	0m0,002s
user	0m0,002s
sys	0m0,001s
dzu@krikkit:~/fs-test/xml-files$

No. We cannot really tell the difference. So we reclaim 94 percent of storage from a directory tree amenable to compression. All we need to do is to flip the switch once. Win.

Comments