Month: September 2021

KVM setup in Fedora

Posted on by Bian Xi Leave a Comment on KVM setup in Fedora

KVM setup in Fedora

Commands

virsh list --all
virsh start <vm>
virsh start <vm> --console
virsh stop <vm>

KVM vs XEN

KVM isn't kernel specific, XEN required special kernel, so XEN could have kernel upgrade issue.

Bridge Network

When creating bridging network, if grub is used to create network interface, then Network Manager should not be used to create same interface. If Network Manager used, same network interface will be appear in ifconfig -a command output twice, One is created by NetworkManager, another is created by grub. If bridge network interface created on top of grub created interface, the IP address will be still assigned to grub created interface.

In order to avoid above issue, following line in /etc/default/grub to create network interface with bridging network interface br0.

GRUB_CMDLINE_LINUX=" ... ip=192.168.1.9::192.168.1.254:255.255.255.0::br0:off nameserver=192.168.1.250 ifname=enp0s10:00:26:4a:18:82:c6 bridge=br0:enp0s10"

After br0 created, KVM manager can select bridging network for vm creation.

Update grub using following command

grub2-mkconfig -o /boot/grub2/grub.cfg

Download driver

Both Windows disk controller driver and ethernet driver can be downloaded from Fedora Website, https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.139-1/virtio-win-0.1.139.iso, and add additional CD-ROM to point to this iso.

Create VM

Using Virtual Machine Manager

Create VM requires add storage, if the storage file doesn't exist, need to select the storage location, and also input the size of disk which located above the location selection box.

Using command line

To create Ubuntu VM from local image,

virt-install \
--name ubuntu2104 \
--ram 3072 \
--vcpus 2 \
--disk path=/kvm/ubuntu2104.qcow2,size=20 \
--os-variant ubuntu20.04 \
--os-type linux \
--network bridge=br0 \
--graphics none \
--console pty,target_type=serial \
--cdrom /kvm/ubuntu-21.04-live-server-amd64.iso \
--boot kernel=casper/vmlinuz,initrd=casper/initrd,kernel_args="console=ttyS0"

To create Fedora VM from remote server

virt-install \
--name fed34 \
--ram 2048 \
--vcpus 2 \
--disk path=/kvm/fed34.img,size=20 \
--os-variant fedora34 \
--os-type linux \
--network bridge=virbr0 \
--graphics none \
--console pty,target_type=serial \
--location 'https://mirror.arizona.edu/fedora/linux/releases/34/Server/x86_64/os/' \
--extra-args 'console=ttyS0,115200n8 serial'

Create Windows 10 VM

virt-install \
   --ram=4096 \
   --name=windows10 \
   --os-type=win10 \
   --network network=default \
   --disk path=/kvm/kvm-windows10.img,size=100 \
   --cdrom=/kvm/virtio-win-0.1.139.iso \
   --graphics spice

Cons

  • Cannot select type of CPU or Passthru mode
  • Cannot select type of disk controller type to use virtual device driver.

References

10 Easy Steps To Install Windows 10 on Linux KVM – KVM Windows

Play music with DAC in MacOS

Posted on by Bian Xi Leave a Comment on Play music with DAC in MacOS

Play music with DAC in MacOS

Use music players in MacOS and output to DAC directly.

Colibri

Website: https://colibri-lossless.com/

Setup

  • Go to Preferences -> Output screen
  • Change Output Device
  • Change DSD mode to DoP if DAC supports DSD.
  • Change PCM mode from to the highest DSD format DAC can play
  • Enable Exclusive / Hog mode if don't expect OS sounds appear
  • Enable Audio Signal Bypass if want to disable OS sound functions
  • Auto-raise system volume to 100% for this device if the DAC has no problem such as reseting voice level. For example, Creative portable DAC can suddent raise to very high level when turning on.
  • Changing device requires Remember... setting to be off

Note: If using Exclusive mode, need to change system default sound device to other device in order to use the device selected for music playing.

Cons

  • Free
  • Can play many format
  • Changing options require player restart
  • Can not specify DAC unsupported formats to be software decoded

VLC

Setup

Forgot how to change them.

  • Go to **Preferences"
  • Click on Show All at bottom left corner
  • ...

Cons

  • Cannot play DFF format
  • Not easy to find out options
  • Mainly for video playing

JRiver Media Center

This is a library style player, but user interface isn't user friendly as iTune, and library isn't manging not well at all.

Pros

  • Not free and yearly upgrade
    • Master (for Windows, Mac, and Linux) License Upgrade $34.98
    • Single OS Upgrade $22.98

Cons

  • Support Windows, Mac, and Linux

Boot from small USB drive with iscsi root filesystem

Posted on by Bian Xi Leave a Comment on Boot from small USB drive with iscsi root filesystem

Boot from small USB drive with iscsi root filesystem

Boot from small size USB drive only holding boot partitions, rest of filesystems are on iscsi drives. Tested in EFI boot in Fedora 34.

Requirement

  • /boot partition can be 256M, can be very small, but better bigger
Filesystem                         Size  Used Avail Use% Mounted on
/dev/sdb2                          428M  190M  212M  48% /boot
  • /boot/efi is an almost static very small filesystem, can be very small
Filesystem                         Size  Used Avail Use% Mounted on
/dev/sdb1                          512M   31M  482M   6% /boot/efi

grub configuration

Define iscsi login info

GRUB_CMDLINE_LINUX="netroot=iscsi:<user>:<password>@<ip>::3260::<iqn> rd.iscsi.initiator=<client iqn> rhgb quiet ...

Define network interface with static ip 192.168.1.2, gateway 192.168.1.254, nameserver 192.168.1.1, interface enp0s10.

ip=192.168.1.2::192.168.1.254:255.255.255.0::enp0s10:off nameserver=192.168.1.1

Define network with bridge interface br0 on network interface enp0s10

ip=192.168.1.2::192.168.1.254:255.255.255.0::br0:off nameserver=192.168.1.1 ifname=enp0s10:xx:xx:xx:xx:xx:xx bridge=br0:enp0s10"

Update grub using following command

grub2-mkconfig -o /boot/grub2/grub.cfg

Shell command to remove `(1)` from filename

Posted on by Bian Xi Leave a Comment on Shell command to remove `(1)` from filename

Shell command to remove (1) from filename

To compare massive number of files with (1) in file name, with the original files without (1), such as ABCD(1).txt and ABCD.txt, following commands can be used. Beware, they are not steps but commands.

Use bash substring

  • Find out all *(1)* files and check whether have original file in same folder.
find . -name "*\(1\)*" | while read line
do
    if test -e "${line/(1)/}"; then
        echo "$line"
    fi
done

Then can clean up them one by one.

  • Move them to another directory

  • Rename them to be the same as original file in same folder

find . -name "*\(1\)*" | while read line
do
    if test ! -e "${line/(1)/}"; then
        mv "$line" "${line/(1)/}"
    fi
done
  • Compare them with original files in same folder

Note: This method only work with the original filename has no (1) string.

Use sed

Following sample script can be used for same task.

#!/bin/bash

find . -name "*" -type f | while read line
do
        dname="`dirname -- \"$line\"`"
        bname="`basename -- \"$line\"`"
        # pattern='s/\(([0-9])\)\./\1/'         # remove "." if match "(1).", \1 == ([0-9])
        # pattern='s/(\([0-9]\))\./\1/'         # remove "(", ")" and "." if match "(1).", \1 == [0-9]
        # pattern='s/([0-9]).//'                # remove "(1)"+any_char
        # pattern='s/[0-9]\.//'                 # remove "(1)."
        # pattern='s/([0-9])\././'              # remove "(1)"
        pattern='s/\s*([0-9])\././'           # remove any_space+"(1)"
        # pattern='s/\s*\././'                  # remove any_space before "."
        # pattern='s/^\./11./'                  # add "11" in front if start with "."
        # pattern='s/^01\./10./'                        # replace starting "01." to "10."
        # pattern='s/^0\([2-9]\)\./1\1./'               # replace starting "01." to "10."
        nname="`echo \"$bname\" | sed -e "$pattern"`"
        # echo "$bname"; echo "$nname"

        if [ "$nname" != "$bname" -a ! -e "$nname" ] ; then
                pushd "$dname"
                echo "$bname"; echo "$nname"
                mv "$bname" "$nname"
                popd
        fi
done

Use vim

  • Use following command to get the list of file name
find . -name "*(1).*" -exec echo mv ~{}~ ~{}~ \; > list
  • Use vim to edit the file
vi list
  • Use lookahead to replace the last (1)
%s/.*\zs(1)//
  • Replace ~ to ", then save it
%s/\~/"/g
  • Run the script
sh list

References

How to change last occurrence of the string in the line?
Regex lookahead and lookbehind

TODO: Network boot for MacBook Pro

Posted on by Bian Xi Leave a Comment on TODO: Network boot for MacBook Pro

Network boot for MacBook Pro

Status

Tried iPXE, but failed after boot into kernel file.

Successfully load boot files

Able to boot by given filenames using similar method as below in iPXE configuration file tftp/boot.ipxe.

initrd ubuntu/12.10-desktop-${cpu_name}/casper/initrd.lz
chain ubuntu/12.10-desktop-${cpu_name}/casper/vmlinuz root=/dev/nfs boot=casper netboot=nfs nfsroot=192.168.1.17:/volume1/tftp/ubuntu/12.10-desktop-${cpu_name} quiet splash
boot

The error shows some sort of issue related to invalid function. Internet users mentioned that it was caused by converting EFI boot to MBR boot in iPXE but firmware doesn't support it.

Able to boot into EFI disk

Looks like MacbookPro supports EFI disk boot only

Issues

Secure boot

Secure boot verifies the signature of boot software whether trusted by firmware. This issue had been fixed after copy workable boot partition from other bootable images, such as ubuntu, fedora, or windows boot image, include /boot and /boot/efi.

Read kernel

Got issue with this stag, kernel read, but execution error with invalid function, didn't have time to troubleshoot.

Future

Grub boot

For network root partition boot, which has /bootlocally, root partition / on iSCSI disk, the grub should be configured as upgrade acceptable, including following requirements.

  • Kernel image should be a standard image to avoid manual kernel rebuilt process
  • Kernel image should include iSCSI driver
  • Kernel image should be able to configure fix IP Address, to avoid unstable iSCSI connection and unauthorized access
  • Kernel image should be able to configure bridge interface or macvlan interface, to support virtualization
  • Kernel image should be less network interface name dependent, to avoid network interface name changed

Network boot

For iPXE boot, iPXE firmware can be loaded by PXE boot process or a local disk, following requirements should be considered.

  • The kernel specification and detection are not part of iPXE configuration.
  • iPXE only detects iSCSI disk, and grub treats it as local disk, then boot from this local disk (iSCSI disk)
  • iPXE iSCSI disk should be able to be recognized by grub as local disk
  • Grub should not reset the network interface or renew IP address
  • MAC address should be the same in iPXE and Grub
  • OS should lock down the network interface, should not allow any services (Network Manager, etc) manage the interface.
  • OS should lock down iSCSI disk
  • Some requirements in Grub boot

References

Fix: System Found Unauthorized Changes on the Firmware, Operating System or UEFI Drivers

TODO: Synology SSD Cache Issues

Posted on by Bian Xi Leave a Comment on TODO: Synology SSD Cache Issues

Synology SSD Cache Issues

Synology SSD Cache have two issues as below

  • Unable to use one disk/array to support mulitiple volume.

    • No answer from Internet and some people mentioned that it is a new request.
    • Possible solution is to create partition/volume on SSD Storage Pool, then use volume as cache device.
    • Synology uses LVM cache, haven't checked whether native linux can do or not.

  • Utilization of cache is very low, about 5GB on fequently used volume, such as volume1.

    • Improved in DSM 7 which supports Pin all Btrfs metadata option. But haven't validated the utilization.

Rsync backup client

Posted on by Bian Xi Leave a Comment on Rsync backup client

Rsync backup client

Rsync backup command can be used as below.

rsync -avR --delete
        --backup-dir=/folder-`date +%Y.%m.%d` \
        --password-file=/root/password_file \
        --exclude-from='exclude-list.txt' \
        /app \
        /usr/local/share/ca-certificates/ \
        /etc/network/interfaces \
        /etc/NetworkManager/system-connections/ \
        /root \
        /home \
        rsync@server::NetBackup/folder

The password_file only contains rsync password without any other data.

In rsync@server::NetBackup/folder, rsync is user id, server is rsync server, NetBackup is rsync service (share folder), folder is where backup to be saved.

The backup-dir is the folder created in the rsync server to save changes.

TODO: Change wordpress TCP port

Posted on by Bian Xi Leave a Comment on TODO: Change wordpress TCP port

Change wordpress TCP port

After change port in settings, also redeployed dockers, the website is unreachable.

Change port

Update in setting of wordpress

Update docker-compose.yml file

Destory and recreate dockers

docker-compose down
docker-compose up -d

Note: Failed

Change back port

Change port back using by update option values database.

Access mariadb docker

docker exec -it wp_db_1 bash

Login to mariadb

mysql -u wordpress -p

Search option value

MariaDB [wordpress]> select * from wp_options where option_value like '%192.168.1.14%';
+-----------+-------------+------------------------+----------+
| option_id | option_name | option_value           | autoload |
+-----------+-------------+------------------------+----------+
|         1 | siteurl     | http://192.168.1.14:80 | yes      |
|         2 | home        | http://192.168.1.14:80 | yes      |
+-----------+-------------+------------------------+----------+
2 rows in set (0.058 sec)

Update value back

MariaDB [wordpress]> update wp_options set option_value='http://192.168.1.14:8080' where option_value='http://192.168.1.14:80';
Query OK, 2 rows affected (0.008 sec)
Rows matched: 2  Changed: 2  Warnings: 0

MariaDB [wordpress]> select * from wp_options where option_value like '%192.168.1.14%';
+-----------+-------------+--------------------------+----------+
| option_id | option_name | option_value             | autoload |
+-----------+-------------+--------------------------+----------+
|         1 | siteurl     | http://192.168.1.14:8080 | yes      |
|         2 | home        | http://192.168.1.14:8080 | yes      |
+-----------+-------------+--------------------------+----------+
2 rows in set (0.058 sec)

MariaDB [wordpress]> quit