After finished first round of writing, I think it needs to be described in clearer way.
During first time setup, I got very confused by the document I was using, and didn't understand fully. After observation of the devices behavior, I think I will review this document later.
apt install bcache-tools
Bcache adds in one more layer between the actual filesystem and the block device (partition, raid, etc.) which filesystem located in. This is done by relocating filesystem header behind bcache header in block device, which offset the ordinary filesystem header and data 8KiB behind.
Note: Use partition as example
Ordinary filesystem partition = (Ordinary filesystem header + Ordinary filesystem data)
bcache partition = bcache header (8KiB) + (Ordinary filesystem header + Ordinary filesystem data)
In such case, the device (partition) is represented as a bcache partition, and bcache driver creates a new device called /dev/bcacheX without bcache header, then OS will detect /dev/bcacheX as ordinary filesystem.
This method is widely used in disk encryption as well, which allows encryption driver translates encrypted data device to OS in a newly created device.
Mount a filesystem bypass bcache header
By using following method, the ordinary filesystem in bcache partition can be mounted by ordinary filesystem driver without bcache driver.
- Create Loopback Device
losetup -o 8192 /dev/loop0 /dev/[BCACHE DEVICE]
- Mount Device
mount /dev/loop0 -o loop /mnt/[LOCATION]
Backing device is the actual device holding data, for example,
/dev/sdb1, it can be 10TB hard disk, disk raid, software raid, etc. It can be created by following command
make-bcache -B /dev/sdb1
It will create a bcache device as well, such as
Bcache device is the device created together with backing device (
/dev/sdb1), such as
/dev/bcache0, all ordinary filesystem operation will be operated on bcache device (
/dev/bcache0), such as mkfs, etc.
Caching device is the temporary device used as cache, for example
/dev/sdc1, it can be 128GB SSD. It can be created using following command
make-bcache -C /dev/sdc1
It has cset.uuid as below
# bcache-super-show /dev/sdc1 | grep cset
Before recreate caching device, use following command clean up the device header.
wipefs -a /dev/<device>
Attach action allows caching device starts working for backing device.
# echo <caching_device_uuid> > /sys/block/<bcache_device>/bcache/attach
echo f0e01318-f4fd-4fab-abbb-d76d870503ec > /sys/block/bcache0/bcache/attach
# or to the backing device
echo f0e01318-f4fd-4fab-abbb-d76d870503ec > /sys/block/sdb/sdb1/bcache/attach
Note: If following error occurred, and no output when ran
bcache-status command, then run
partprobe command to rescan partition tables
# echo f0e01318-f4fd-4fab-abbb-d76d870503ec > /sys/block/bcache0/bcache/attach
-bash: echo: write error: No such file or directory
bcache-status is a free bcache tool can be downloaded from github.
Operation on backing device and bcache device
The backing device and the bcache device have 1-to-1 relationship, because they are created at same time using one
make-bcache -C command.
In fact, both
bcache folders in both bcache and backing device are the same.
# ls -Hdi /sys/block/sdb/sdb1/bcache
# ls -Hdi /sys/block/bcache0/bcache
I think the correct thinking should be attaching to bcache device, but the bcache folder is created under backing device.
# readlink -f /sys/block/bcache0/bcache
I think this is because that bcache device is a virtual device created during backing device (real device) creation, so the actual device structure should assign to real device (backing device).
With or without caching device
Without caching device, bcache driver will directly translate ordinary filesystem driver read/write into bcache device read/write.
With cache device, bcache driver will utilize caching device before backing device operation.
ordinary filesystem operation => bcache driver => bcache filesystem operation => backing device
So, without caching device, bcache is still operating correctly.
Attach caching device to multiple bcache devices
One caching device can support multiple bcache devices as below
# echo f0e01318-f4fd-4fab-abbb-d76d870503ec > /sys/block/bcache0/bcache/attach
# echo 4b05ce02-19f4-4cc6-8ca0-1f765671ceda > /sys/block/bcache1/bcache/attach
# echo 4b05ce02-19f4-4cc6-8ca0-1f765671ceda > /sys/block/bcache2/bcache/attach
# echo 4b05ce02-19f4-4cc6-8ca0-1f765671ceda > /sys/block/bcache3/bcache/attach
# echo 75ff0598-7624-46f6-bcac-c27a3cf1a09f > /sys/block/bcache4/bcache/attach
A bcache device has only one caching device
The uuid of caching device which attached to bcache device can be found as below.
# ls -la /sys/block/<device>/bcache/cache
lrwxrwxrwx 1 root root 0 Jun 19 18:42 /sys/block/<device>/bcache/set -> ../../../../../../../../fs/bcache/<UUID>
To detach a caching device, needs to send
1 or cache-set-uuid to bcache device or backing device
Safely remove the caching device from bcache device
echo cache-set-uuid > /sys/block/bcache0/bcache/detach
echo cache-set-uuid > /sys/block/sdb/sdb1/bcache/detach
Detach the caching device from bcache device
echo 1 > /sys/block/bcache0/bcache/detach
echo 1 > /sys/block/sdb/sdb1/bcache/detach
Stop a bcache device, is the same as stop the backing device.
echo 1 > /sys/block/bcache0/bcache/stop
echo 1 > /sys/block/sdb/sdb1/bcache/stop
After stopped bcache/backing device,
/sys/block/bcache0virtual device disappers
- But no impact to caching device, and still registered in
Stop caching device, will impact all caching, bcache and backing devices
echo 1 > /sys/fs/bcache/cache-set-uuid/stop
After stopped caching device, all bcache setup related to that caching device disapper
/sys/block/sdb/sdb1/bcachefolder disappers if
/dev/sdb1is backing device attached
- The bcache devices
/dev/bcache0is the bcache device attached
- The caching device
The first way to resume whole setup, can be done by run
The second way is using register to resume device one by one.
In fact, registering is needed every bootup, but attaching only have to be done once.
Register is required if the caching or backing device missing during system start up or they are stopped manually.
Register the backing device as below
echo /dev/sdb1 > /sys/fs/bcache/register # backing device
After registered, the system will
- If didn't attach caching device before stopped or caching device had been registered, then creates
/sys/block/bcache0 is not created due to missing caching device,
/sys/block/bcache0device will be created after register missing caching device
- Or use following command to force to create
/sys/block/bcache0and start running
echo 1 > /sys/block/sdb/sdb1/bcache/running
Warning: If force start, all write cache in caching device will be lost, this can cause filesystem corruption
Register caching device
To register caching device, following command can be used
echo /dev/sdc1 > /sys/fs/bcache/register # caching device
It will create directory
If attached backing device had been registered, the
/sys/block/bcache0 will be created and running.
Ordinary filesystem operation
All ordinary filesystem operations will be operated on bcache device (
/dev/bcache0), for example
mount /dev/bcache0 /mnt
The caching state can be viewed using following command
- no cache: this means you have not attached a caching device to your backing bcache device
- clean: this means everything is ok. The cache is clean.
- dirty: this means everything is setup fine and that you have enabled writeback and that the cache is dirty.
- inconsistent: you are in trouble because the backing device is not in sync with the caching device
There are 4 caching modes,
echo writeback > /sys/block/bcache0/bcache/cache_mode
Show caching device info
echo 100 > /sys/block/bcache0/bcache/writeback_percent
How much data in cache has not written into backing device.
Flush cache to backing device
This might be required if filesystem maintenance needed.
Run following command to disable writeback mode
echo writethrough > /sys/block/bcache0/bcache/cache_mode
Wait until state reports "clean"
watch cat /sys/block/bcache0/bcache/state
Force flush of cache to backing device
echo 0 > /sys/block/bcache0/bcache/writeback_percent
/sys/fs/bcache/ folder does not exist
bcache module was not loaded.
sh: echo: write error: Invalid argument
If dmesg shows
bcache: bch_cached_dev_attach() Couldn't attach sdc: block size less than set's block size
Then the --block 4k parameter was not set on either device and defaults can mismatch.
Otherwise, the device might already be attached.
sh: echo: write error: No such file or directory
The UUID is not a valid cache.
Boot from bcache device
Grub2 does not offer support for bcache, but it is fully supported by UEFI. Check the following link for details