Convert Armbian to iSCSI btrfs root
In order to have bad SD card worry-free, I decided to convert Armbian root filesystem to btrfs and move to Synology iSCSI LUN.
Pros
- Backup can be done by iSCSI LUN snapshot
- Only very small SD card needs to be used
- iSCSI LUN can increase space easily
- SD card data can be recreated easily
- Faster even use slow SD card
- Harddisk is cheaper than SD card
Steps
Separate /boot
and root (/
) partition
Move root filesystem to new SD card
This step needs to have another SD card, which needs to be able to hold all the original SD card files.
-
Insert the new SD card via USB card reader
-
Identify the SD card device name using lsblk
command, normally should be /dev/sda
-
Format SD card to have two partitions, such as
/dev/sda1 2048 4196351 4194304 2G 83 Linux
/dev/sda2 4196352 33554431 29358080 14G 83 Linux
Note: There are some soft link in /boot
filesystem, if /dev/sda1
is vfat, those soft link files will not be copied. I'm not sure if any issue will be encountered.
Boot from SD - system on SATA, USB or NVMe
-
Select /dev/sda2
as destination to install system to /dev/sda2
-
Select btrfs as filesystem type, after that, the system will format /dev/sda2
and transfer all files into this new partition.
- Once completed, reboot the system
Verify system after reboot
Now, the system should have two filesystems
- root (
/
), which is in new USB drive
/boot
that binds to /media/mmcboot
, which point to SD card, /dev/mmcblk0p1
.
Copy all files from /boot
to /dev/sda1
We create the partition /dev/sda1
has same filesystem type as original partition, and maintain the same structure /boot as well.
mkfs.ext4 /dev/sda1
mount /dev/sda1 /mnt
cp -a /boot /mnt
Note: The boot partition files can be in subdirectory of /mnt/boot
, as well as in /mnt
. If they are in /mnt/, then the path in
/etc/fstab` needs to be changed. If failed to do this, the system is still bootable, just can not mount boot filesystem and taking time to scan filesystems as well.
Modify root partition UUID
Note: This should have been updated., because the root (/
) is already running on new SD card
- Find out the UUIDs for root (
/
) filesystem and /boot
filesystem.
blkid
- Update following line in
/mnt/boot/armbianEnv.txt
to root UUID if required.
rootdev=UUID=1c82450c-9013-43d9-9554-1049c264bfb8
Shutdown
- Shutdown the system
- Take out old SD card from SD card slot
- Remove new SD card USB device
- Insert new SD card into SD card slot
- Then power on system
Verify system
Now, the system should have /boot
and root (/
) filesystems on new SD card.
Move root to iSCSI
Install/configure iSCSI service
apt install open-iscsi
- Edit
/etc/iscsi/initiatorname.iscsi
, update following line
InitiatorName=<YOUR_INITIATOR_NAME>
Note: The YOUR_INITIATOR_NAME is the iSCSI client name
- Edit
/etc/iscsi/iscsid.conf
, update following lines
node.session.auth.authmethod = CHAP
node.session.auth.username = <YOUR_USERNAME>
node.session.auth.password = <YOUR_PASSWORD>
- Enable
iscsid
service and restart it
systemctl enable iscsid
systemctl restart iscsid
# iscsiadm --mode discovery --type sendtargets --portal <YOUR_TARGET_IP>
# iscsiadm --mode node --targetname <YOUR_TARGET_NAME> --portal <YOUR_TARGET_IP> --login
Note: If can not login, restart iscsid
and try again.
systemctl restart iscsid
Identify block device
Use lsblk
command to identify device file, normally should be /dev/sda
.
Partitioning
Creating two partitions using fdisk, the first partition is to prepare following for future used, such as
/dev/sda1 2048 2099199 2097152 1G 83 Linux
/dev/sda2 2099200 33554431 31455232 15G 83 Linux
Note: The first partition can be used for iSCSI boot or /boot
filesystem backup
Update initramfs
This is to enable kernel load iscsi driver during boot up
touch /etc/iscsi/iscsi.initramfs
update-initramfs -v -k $(uname -r) -c
Configure iSCSI kernel parameters
extraargs=ip=<ip_address>::<gateway>:<mask>:<host>:<interface_name>::<dns0>:<dns1>: ISCSI_INITIATOR=<ISCSI_INITIATOR> ISCSI_TARGET_NAME=<ISCSI_TARGET_NAME> ISCSI_TARGET_IP=<ISCSI_TARGET_IP> ISCSI_TARGET_PORT=3260 ISCSI_USERNAME=<YOUR_USERNAME> ISCSI_PASSWORD=<YOUR_PASSWORD> rw
Note: This includes IP configuration for fix IP. If use dhcp, just change to ip=dhcp
Reboot and verify
Make sure the iSCSI drive automatically loaded after reboot, and IP address assigned correctly.
This is to confirm that iSCSI is working during reboot.
Disable iscsi stop action
Disable iscsid service stop action is to prevent reboot hanging issue.
There is no need to logout as Allow multiple sessions is set to true in Synology iSCSI Target configuration, and the iSCSI Initiator used in both kernel and iscsid configure are the same.
systemctl edit --full open-iscsi.service
Comment out following line
#ExecStop=/lib/open-iscsi/logout-all.sh
Move root filesystem to iSCSI LUN
- Run nand-sata-install, select following option
Boot from SD - system on SATA, USB or NVMe
-
Select /dev/sda2
as destination to install system to /dev/sda2
-
Select btrfs as filesystem type, then the system will format /dev/sda2
, and transfer all files into this new partition.
-
Once completed, reboot the system
Verify system after reboot
Now, the system should have two filesystems
- root (
/
), which is in iSCSI drive
/boot
, binds to /media/mmcboot
, which point to SD card, mmcblk0p1.
Other consideration
Recreate /boot/boot.scr
To make sure /boot/boot.scr
is up to date, run following command is necessary especially if /boot/boot.cmd
was modified.
mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr
Backup /boot
to iSCSI
Create filesystems
- Create filesystem in iSCSI LUN
mkfs.ext4 /dev/sda1
- Copy files into
/dev/sda1
mount /dev/sda1 /mnt
cp -a /boot /mnt
umount /mnt
Recreate /boot
SD card
Assuming the new SD card is named as /dev/sdb
mkfs.ext4 /dev/sdb1
mount /dev/sdb1 /mnt
cp -a /boot /mnt
umount /mnt
-
Find out /dev/sdb1
UUID using command blkid
-
Modify /etc/fstab
Update /boot
filesystem UUID, which should be under /media/mmcboot
entry.
-
Change SD card to the new card and reboot
SD card as cache
Thinking of how to use rest of space in SD card. Maybe can use it as bcache caching device to reduce the network traffic.
References
quick way to create SD card with separate /boot and / partitions?
Mounting the root filesystem via NFS (nfsroot)
Setting a Static IP Address Using the Kernel Command Line
Shutdown hang on 16.04 with iscsi targets
mkimage - Generate image for U-Boot
How to install to eMMC, NAND, SATA & USB?
Diskless iSCSI boot with PXE HOWTO