Month: December 2021

Migrate USB boot TrueNAS to Proxmox VM

Posted on by Bian Xi Leave a Comment on Migrate USB boot TrueNAS to Proxmox VM

Migrate USB boot TrueNAS to Proxmox VM

After Proxmox installed, I also migrate TrueNAS to Proxmox as VM

Create disk image

Use dd command to copy USB drive to a disk file

dd if=/dev/sdi of=/tmp/truenas.raw bs=10m

Create Proxmox VM

  • Create a VM with SeaBIOS

  • Remove VM disk

  • Use following command to import disk

qm importdisk <vm_id> <raw_file> <storage_id>

For example

qm importdisk 100 vm.raw ds1812-vm_nfs1

  • Go to VM hardware page

  • Select unused disk and click Add button to add disk into VM

  • Select Options => Boot Order to check the iscsi controller

  • List all disks and find out the disks like to passthru

lsblk -o +MODEL,SERIAL,VENDOR
  • Find out device path
ls /dev/disk/by-id/*<SERIAL_NUM>*
  • Import storage disks as passthru devices
qm set 100 -scsi2 <device>

Boot TrueNAS VM

Change network configuration

  • Interface with IP address
  • Bridge network interface

References

How to run TrueNAS on Proxmox?
Export Virtual Machine from TrueNAS and Import VM to Proxmox

Unable to boot from USB in VMware Fusion

Posted on by Bian Xi Leave a Comment on Unable to boot from USB in VMware Fusion

Unable to boot from USB in VMware Fusion

Fusion reports error, and it can not connect to USB drive to boot.

Fusion does not support USB adapters for connecting displays to your virtual machines.

Note: I haven't tried to change UEFI mode to BIOS mode, maybe this is the cause.

References

Unable to connect a USB device to a virtual machine (2118442)
USB Device not detected by the virtual machine (57195)
Configuring the USB Controller and Connecting USB Devices

Proxmox with UEFI iSCSI root

Posted on by Bian Xi Leave a Comment on Proxmox with UEFI iSCSI root

Proxmox with UEFI iSCSI root

Note: I only record down the steps as draft, some steps are not required in this document, will update next time if I need to do same task.

I planed to convert TrueNAS to TrueNAS on Proxmox as many people implemented. The first step is install Proxmox.

As Proxmox uses local drive very frequently, slow USB disk is not suggested as mentioned in Internet, so I tried to convert Proxmox to iSCSI root.

Reasons

Some advantages

  • Proxmox is a proper virtual environment with container support (LXC)
  • Proxmox is on a customized debian environment, which uses apt to update. But TrueNAS, at least now, apt upgrade breaks installation.
  • Proxmox can perform passthru controller or disk, give VM better storage management
  • Boot partition only used during booting and Kernel update, and it is very small
  • The root partition on iSCSI can take iSCSI advantages, such as snapshot, etc.

Other reasons

  • TrueNAS VM can be converted to Proxmox as others do
  • TrueNAS can not be iSCSI boot using normal method, because it is not a normal Linux

Another reason is, 10 days ago, my TrueNAS self rebooted even night, I could not find reason. And I also want to compare the performance between TrueNAS VM and Proxmox VM.

Preparation

I installed Proxmox on TrueNAS VM first, then move VM EFI boot partition to physical USB drive.

  • Create VM on TrueNAS
  • 16GB USB drive for EFI boot partition (Only requires 513M USB drive)
  • 10GB local disk for OS installation
  • 16GB iSCSI LUN on NAS (10GB is enough)

Steps

Install OS

Download proxmox image from Proxmox VE 7.1 ISO Installer proxmox-ve_7.1-2.iso from website https://www.proxmox.com/en/downloads

Install proxmox in the VM with selection of zfs as filesystem.

It has 3 partitions

  • BIOS boot partition (23-2047 sector)
  • EFI partition, 512MB, vfat
  • /, zfs

The target is to move / to iSCSI LUN, and others to USB drive.

OS structure

After OS installed, the some system structured info as below

  • BIOS boot partition is not mountable
  • EFI partition contains kernel, this is to avoid /boot partition
  • EFI partition is not mounted after boot up, this is to avoid corruption
  • Proxmox kernel parameter file /etc/kernel/cmdline
  • Proxmox boot loader files loader/entries/entry.conf and loader/loader.conf
  • The zfs in proxmox uses partition device as disk, not partition id, better use partition id because it will not be changed.

Other files

  • (Not required) Ubuntu grub parameter file - /etc/default/grub

This file is used to build actual /boot/grub/grub.cfg, it is not required unless you like to boot into grub to verify configuration, which gives more error info.

Note: I used grub to detected error in cmdline. Installation of grub on boot disk is required, and reinstall proxmox boot is needed after troubleshooting.

Configure iSCSI service

The service was installed by default in proxmox.

  • Configure /etc/iscsi/initiatorname.iscsi
InitiatorName=<YOUR_INITIATOR_NAME>
  • Configure /etc/iscsi/iscsid.conf
node.session.auth.authmethod = CHAP
node.session.auth.username = <YOUR_USERNAME>
node.session.auth.password = <YOUR_PASSWORD>
  • Start iscsid service

After iscsid configuration changed, restart iscsid service is required.

systemctl restart iscsid
  • Discover and Login
# iscsiadm --mode discovery --type sendtargets --portal <YOUR_TARGET_IP>
# iscsiadm --mode node --targetname <YOUR_TARGET_NAME> --portal <YOUR_TARGET_IP> --login

Note: If can not login, restart iscsid and try again.

Identify block device

Use lsblk command to identify device file, it should be something like /dev/sdX.

Partitioning

Creating two partitions using fdisk, the first partition is to prepare following for future used, such as

  • Network boot
  • UEFI iSCSI boot
  • USB device backup

Partition /dev/sda2: vfat, 512MB (EFI)
Partition /dev/sda3: for root filesystem (Label as )

Note: The iSCSI LUN appears as /dev/sda

FYI, the first partition can not be created using fdisk command.

Format iSCSI LUN

Format /dev/sda1 as vfat

mkfs.vfat /dev/sdd1

Copy EFI data

Note: The disk in VM appears as /dev/vda

mkdir /mnt/1 /mnt/2
mount /dev/vda2 /mnt/1
mount /dev/sda2 /mnt/2
cd /mnt/1
cp -a . /mnt/2/
umount /mnt/1 /mnt/2
rmdir /mnt/1 /mnt/2

Note: Do not use dd command at this stage, because there will be two partitions have same partition id, proxmox-boot-tool will not update correctly

Duplicate /root data

Find uuid

blkid

Attach iSCSI LUN to local disk

zpool attach rpool vda3 <partition_id of iSCSI LUN>

Update initramfs

This is to enable ubuntu load iscsi driver during boot

echo "ISCSI_AUTO=true" > /etc/iscsi/iscsi.initramfs
update-initramfs -u
update-initramfs -v -k $(uname -r) -c

Note: I ran both update-initramfs command, one suggested by proxmox, another one I used before in other systems' migrations

Update /etc/kernel/cmdline

Append following code into first line

ip=192.168.1.51::192.168.1.254:255.255.255.0:<hostname>:[interface]::192.168.1.250:: ISCSI_INITIATOR=<YOUR_INITIATOR_NAME> ISCSI_TARGET_NAME=<YOUR_TARGET_NAME> ISCSI_TARGET_IP=<YOUR_TARGET_IP> ISCSI_TARGET_PORT=3260 ISCSI_USERNAME=<YOUR_USERNAME> ISCSI_PASSWORD=<YOUR_PASSWORD>

Note: interface can be empty if only has one network card.

Update Proxmox boot script

proxmox-boot-tool refresh

Verify Proxmox Loader files

Verify Proxmox boot loader files loader/entries/entry.conf and loader/loader.conf whether up to date.

Note: If two partitions has same partition id, the proxmox-boot-tool might updated other partition.

Disable iSCSI logout

The early iSCSI logout, can cause BTRFS filesystem closing issue, especially on root (/) filesystem. Disable iSCSI logout during service stop.

systemctl edit --full open-iscsi.service

Comment out following line

#ExecStop=/lib/open-iscsi/logout-all.sh

Test Reboot

This is the first time test reboot, the outcome should be

  • No hanging issue
  • New iSCSI disks can be found after reboot without running iscsiadm command manually. Verify using lsblk command.
  • The command zpool status shows both local and iSCSI LUN are listed correctly.

The objective of this reboot is to test iSCSI module and finding out any misconfiguration for grub.

Detach local disk

zpool detach rpool vda3

Reboot Test

  • Verify the rpool only has iSCSI LUN

Copy boot partitions to USB drive

This is to copy both BIOS boot and EFI partition. I used dd command because the first partition could not be created manually by fdisk, I think it can be created using proxmox-boot-tool, but I didn't try that.

  • Check size of partitions, and record down the first sector number of root partition
fdisk -l /dev/vda
  • Use dd create image
dd if=/dev/vda of=/tmp/efi.dsk count=<the first sector number of root partition>

*Note: the count should be equal to the sectors cover both BIOS boot and EFI partition. Bigger is ok too, because we will remove the root partition.

Copy disk image to USB drive

dd if=/tmp/efi.dsk of=/dev/sdd

Note: /dev/sdd is the USB device. I used another VM to do this

Remove third partition on USB drive

# fdisk /dev/sdd
d
3
w

Boot from USB drive

Insert USB drive into a physical server, and boot from USB

Troubleshooting

Booting can not detect iSCSI server

If you have multiple network card, need to update the interface name in ip parameter in boot menu.

  • After booting into ramdisk run ip a find correct interface name, then reboot
  • Press e key to editor boot menu when showing boot menu
  • Update network interface name
  • Press Enter to boot

iSCSI login error

  • Check multiple connection allowed option in iSCSI target configuration, especially if iSCSI logout disabled

Slow network caused iSCSI connection error

This is a strange issue, because it only happened before I successfully booted.

  • After booting into ramdisk, run ping <iSCSI server>, verify network connection
  • Reboot and press Control-s after iSCSI login, press Control-q to release after detected iSCSI LUN

Verify iSCSI configuration

Update grub

  • Ubuntu grub parameter file /etc/default/grub as listed in previous section.
...
GRUB_CMDLINE_LINUX_DEFAULT="ip=192.168.1.51::192.168.1.254:255.255.255.0:<hostname>:[interface]::192.168.1.250:: ISCSI_INITIATOR=<YOUR_INITIATOR_NAME> ISCSI_TARGET_NAME=<YOUR_TARGET_NAME> ISCSI_TARGET_IP=<YOUR_TARGET_IP> ISCSI_TARGET_PORT=3260 ISCSI_USERNAME=<YOUR_USERNAME> ISCSI_PASSWORD=<YOUR_PASSWORD>"
  • Mount EFI partition
mount /dev/vda2 /boot/efi
  • Update both /boot/grub/grub.cfg and /boot/efi/EFI/ubuntu/grub.cfg
dpkg-reconfigure grub-efi-amd64
  • Install grub on disk
grub-install.real /dev/vda

Note: grub-install is disabled by default

Return back to Proxmox Boot

proxmox-boot-tool init /dev/vda2
proxmox-boot-tool refresh

Follow up actions

bridge network

Change bridge network interface depending on the correct physical interface.

USB disk free space

Use following command to create zfs on USB free space

  • Create partition to cover USB drive free space

  • Create zpool

zpool create upoolb <new_free_partition>
  • Add /upoolb directory to save ISO images or LXC templates

Both type of data are not updated fequently.

References

Host Bootloader
Installation
Proxmox ISCSI installation
Install Proxmox VE on Debian Buster

JQuery retrieve page and display

Posted on by Bian Xi Leave a Comment on JQuery retrieve page and display

JQuery retrieve page and display

Just completed the troubleshooting HTML + jQuery in Markdown, which is used in wordpress.

Script

<div><button id='related_posts_1853'>Display Learning Posts</button>

<script type="text/javascript">
(function($) {
    $(document).ready(function() {
        $("#related_posts_1853").click(function(event) {
            el = $(document.getElementById( $(this).attr('id') ).parentElement);
            $.get("?p=1733", function(data, status) {
                el.html(/h2>References<\/h2>([\s\S]*?)<\/div>/.exec(data)[1]);
            });
        });
    });
}) (jQuery);
</script>
</div>

Note: Can not have empty line in script if it is embedded in Markdown lauguage, because there will be a <p> added.

Explanation

Following jQuery script is to do

  • Define a button in div

  • When button clicked, retrieve html page

  • Then filter information using regular expression

  • Then find out the div element using following script

  document.getElementById( $(this).attr('id') ).parentElement

Note: $(this).attr('id') is to get event element ID

<script>
$("button").click(function() {
    var t = $(this).attr('id');
    ...
});
</script>
  • Then replace the div html using retrieved info.

References

How to get the ID of the clicked button using JavaScript / jQuery ?
document.getElementById vs jQuery $()
Node.parentElement
Category: Selectors
.find()
How get body element from the html which one have as a string
How to get the html of a div from a different page with AJAX?

Learning – Fat-Free PHP Framework (Data Sets)

Posted on by Bian Xi Leave a Comment on Learning – Fat-Free PHP Framework (Data Sets)

Learning - Fat-Free PHP Framework (Data Sets)

Data Sets

Do some RESTfull, serializable...

Display messages

  • Create Nav Sidebar Menu /messages
<li><a href="/messages">Messages on UI f3 template</a></li>

  • Update routes.ini

    Add following

GET /messages=MainController->displayMessages

  • Update MainController.php

    Add displayMessages in MainController class

class MainController extends Controller
{
    ...

    function displayMessages() {
        $messages = new Messages($this->db);

        $this->f3->set('messages', $messages-all());
        $this->f3->set('view', 'messages.htm');
        $template = new Template;
        echo $template->render('layout.htm');
    }
}
  • Create view messages.htm
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
    <h1 class="page-header">Messages UI from template</h1>

    <repeat group="{{ @messages }}" value="{{ @message }}">
        <p>{{ @message.id }} {{ @message.key }} {{ @message.message }}</p>
    </repeat>
</div>

Dipplay messages as JSON using AJAX call

  • Create Nav Sidebar Menu /api/messages
<li><a href="/api/messages">Messages JSON REST API</a></li>

  • Update routes.ini

    Add following

GET /api/messages=MainController->apiMessages

  • Update MainController.php

    Add displayMessages in MainController class

class MainController extends Controller
{
    ...

    function apiMessages() {
        $messages = new Messages($this->db);
        $data = $messages->all();

        $json = array();

        foreach ($data as $row)
            $items = array();

            foreach ($row as $key => $value) {
                $items[$key] = $value;
            }

            array_push($json, $items);
        }

        echo json_encode($json);
    }
}

Note: The echo json_encode($data); doesn't work, because $messages->all() from model has returned some non public attributes

*Note: As the output is the JSON data, no view need to be defined.

Display messsages using REST API

Create new display messages page utilizes the REST API.

  • Add menu in app/views/nav.htm
<li><a href="/messageajaxview">Message UI with AJAX</a></li>
  • Create route in routes.ini
GET /messageajaxview=MainController->displayMessagesAjaxView
  • Create function in app/controllers/MainController.php
function displayMessagesAjaxView() {
    $this->f3->set('view', 'messagesajax.htm');
    $template=new Template;
    echo $template->render('layout.htm');
}
  • Create template app/views/messageajx.htm
<div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main">
    <h1 class="page-header">Message UI with AJAX</h1>

    <button type="button" class="btn btn-default btn-lg">Fetch messages</button>

    <div id="msg-container"></div>
</div>

  • Create javascript file app/js/f3sample.js

  • Include jquery and javascript in app/views/header.htm

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js"></script>
<script src="app/js/f3sample.js"></script>

Then the f3sample.js should be shown the resources in browser development tools.

References

Displaying data sets and creating a Rest API with the Fat-Free PHP Framework

Learning – Fat-Free PHP Framework – Posts List

Posted on by Bian Xi Leave a Comment on Learning – Fat-Free PHP Framework – Posts List

Learning - Fat-Free PHP Framework - Posts List

This is the summary list of my posts during learning.

References

Learning - Fat-Free PHP Framework
Learning - Fat-Free PHP Framework (Bootstrap & Authentication)
Learning - Fat-Free PHP Framework (Template Hierarchy)
Learning - Fat-Free PHP Framework (Data Sets)

JQuery $ is not a function

Posted on by Bian Xi Leave a Comment on JQuery $ is not a function

JQuery $ is not a function

When adding JQuery script in WordPress, the error occurred as $ is not a function.

Fix

Add a function wrapping the $ function as below.

<script type="text/javascript">
    (function($) {
        // You pass-in jQuery and then alias it with the $-sign
        // So your internal code doesn't change
    }) (jQuery);
</script>

References

jquery - is not a function error

HP ProDesk Error – Not fully configured

Posted on by Bian Xi Leave a Comment on HP ProDesk Error – Not fully configured

HP ProDesk Error - Not fully configured

Error

Got following error when bios booting up.

warning your system is not fully configured please contact hp support

Fix

Download NbDmiFit-2.13.zip file from following website, unzip it, then run sssm.bat from command line in folder wndmifit.

https://www.finalfixer.com/2020/02/hp-dmi-tool-nbdmifit-wndmifit-all.html