Btrfs Best Practices

Technology
, , , ,

I came across two good Btrfs documentation when working with btrfs (in production), the docs and best practices are for those brave hearts who use Btrfs on a daily basis;-)

NOTE: Btrfs is experimental and under heavy development. Don’t use it to store critical data. DO backup your data (it is a good idea to place backup on an ext4 file system, don’t put all eggs in one basket) before trying it out. No one can save you without good backups;-)

  1. Btrfs fun - Funtoo Linux Wiki
  2. Chapter 5. The Btrfs File System- Oracle Linux 6 - Administrator’s Solutions Guide

Use tips based on practice

  1. Try **NOT **to use use Btrfs for /, especially when you ONLY have a single block device (partition)

NOTE: Metadata will by default be duplicated inside the single device (unless you explicitly specify -m single when creating the file system, dangerous!), BUT you ONLY have 1 copy of data.

  1. btrfs is recommend for LXC - (Linux Containers) as backingstore
    LXC works very well with Btrfs, it makes use of Btrfs snapshot feature to clone containers. When using lxc-create, the -B NEED NOT to be specified as it will be used automatically if the /var/lib/lxc filesystem is found to be Btrfs.
  2. If possible, use at least 2 block devices (can be 2 partitions or 2 HDDs), to mirror data using RAID 1 (RAID 10 is alsosupported)
    Example: mkfs.btrfs -m raid1 -d raid1 /dev/sdc /dev/sdd metadata is by default replicated on all devices so you don’t have to explicitly specify -m raid1
  3. Don’t trust df -hT output, use btrfs filesystem df /mount_point instead
  4. Use compression - LZO recommended to improve file system throughput
    Example: mount -o compress=lzo,subvol=subvol /dev/sdd /mnt/subvol

NOTE: compression can be enabled on a subvolume basis. In addition, it can be enabled any time after the file system is created. DO remember to run a btrfs filesystem defragment file or btrfs filesystem defrag directory to apply compression to existing data. Mounting the subvol without compression option, data is still accessible (NOT sure if mv cp or edit will change the compression state though, comment if you know).

  1. ONLY keep a minimum set of snapshots
    Think in Copy-on-Write, data blocks are NOT duplicated between a mounted subvolume and its snapshot unless changes are made to the files (a snapshot can occupy nearly zero extra space on the disk).  As time goes by, more and more data blocks will be changed and it silently eats up disk space.

NOTE: Btrfs snapshots are writable!

  1. **A drawback of having snapshots

**If in the original file system some files get deleted, the snapshot still contains them and the disk blocks CANNOT be claimed as free space. Remember to delete unwanted snapshots (just like deleting subvolumes) and keep a minimal set of them (tip #5).

  1. Turn OFF  system auto backup features (a snapshot will be created before system update, Oracle Linux yum-plugin-fs-snapshot plugin does this, Ubuntu and OpenSUSE do the same in a similar way). It eats up your disk space if you don’t keep an eye and don’t give a shit;-)
  2. Clone / snapshot single file using cp --reflink
    Btrfs supports creation of clones for individual files. Clones are lightweight copies (CoW) - only an inode is created, and it shares the same disk blocks as the original file.
  3. You can mount subvolumes
    Example: mount -t btrfs -o subvol=subvol_name device /mnt/subvol** **
  4. You can mount snapshots to restore data
    For full system recovery using snapshots, you’ll need to use Back To the Future, either 1. fiddle the default subvol number or 2. use the kernel command line parameters in the bootloader configuration files. Refer to the Funtoo wiki for details.
    Example: mount -t btrfs -o subvol=snapshot-id device /mnt/snapshot
  5. Use mount option mount -o space_cache to speed up boot and gain slight performance improvement
    When the Btrfs file system is mounted with the space_cache option, Btrfs is able to store the free space cache on the disk to make the caching of a block group much quicker. Without this support, Btrfs has to scan the entire tree each time looking for space that can be allocated.
  6. Last but not least, keep your kernel up-to-date, use the cutting edge stable kernel

BTW: Where the fuck is btrfsck ?

NOTE: I don’t think we’ll touch btrfsck on a regular basis, probably never (does it make sense to fsck a 3TB HDD?). Similar to ZFS, Btrfs is capable of self-healing and relies on snapshot backups to recover when shit happens. If a Btrfs file system is at the point where it is unable to heal itself and roll back to a good snapshot backup, it means game over => it is basically unrepairable.

The instructions on Funtoo wiki is out-of-date (upon finishing the article). Chris Mason has changed the branch name from donteveruse to dangerdonteveruse shit!

So the instructions:

# git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
# cd btrfs-progs/
# git checkout dangerdonteveruse
Switched to branch 'dangerdonteveruse'
# make

NOTE: I updated the Funto wiki page and it should be approved pretty soon.

NOTE: Chris Mason decided to release btrfs-progs at the same pace as the Linux kernel since Linux 3.12. btrfsck is NOW a part of the btrfs-progs package. NO need to build from source.

Recommended reading: How I Use The Advanced Capabilities of Btrfs