Blog

Blog

Using wildcard DNS record

Posted on by Bian Xi Leave a Comment on Using wildcard DNS record

Using wildcard DNS record

Using wildcard in DNS A or CNAME can reduce maintenance work, but this also has some drawback.

Pros

Updating DNS will take time, not only needs time to edit DNS entries, but also the DNS broadcast time, which can take more than 30 minutes to update an existing entry in other DNS server, and some DNS server can take even longer.

If use widecard, there is no editing requirement.

Cons

Search suffix

If /etc/resolv.conf has entry as below

search example.com

When searching for google.com, It might hit google.com.example.com if wildcard used, because google.com.example.com will return IP address as same as example.com.

Caching

Without editing entry, the some DNS servers might still caching old data. For example, if you have both wildcard *.example.com and a specific entry ftp.example.com, and you decided to remove ftp.example.com, the wildcard might not take effect.

Accessing non-exist domain

If using browser access undefined domain name, and hit the web server, using NGINX can perform redirect to change URL. This can fix the issues which web coding based on current URL.

server {
    listen 80;
    server_name *.mywebsite.tld;
    return 301 $scheme://mywebsite.tld$request_uri;
    }

server {
    listen  80;
    server_name mywebsite.tld;
    [...]
    }

References

Is a wildcard DNS record bad practice?

Reboot hung on lvm2-monitor.service on Ubuntu

Posted on by Bian Xi Leave a Comment on Reboot hung on lvm2-monitor.service on Ubuntu

Reboot hung on lvm2-monitor.service on Ubuntu

Keep getting following error message during reboot

...a stop job is running for monitoring of lvm2 mirrors...

But the system has no lvm volume at all.

Some people said, this service is to fix bug on BTRFS snapshot.

Action

I disabled it, because I also don't use BTRFS snapshot currently.

References

Thread: Are lvm2- services necessary on my system

Change Apache SSL certificate

Posted on by Bian Xi Leave a Comment on Change Apache SSL certificate

Change Apache SSL certificate

Steps

Retrieve certificate

Download certificate from Synology

  • Open Control Panel
  • Select Security
  • Select Certificate tab
  • Right click certificate and select Export certificate

The output includes cert.pem, chain.pem, and privkey.pem in archive file or folder.

Merge certificate and chain

Concatenate certificate file and chain file into one file called cert-with-chain.pem or fullchain.pem

Deploy certificate files

  • Create a folder in apache configuration folder, such certs/
  • Copy cert-with-chain.pem and privkey.pem file into certs/ folder.

Setup Apache

This can be done by changing Apache configuration or change the make soft link to the files which configuration used.

Apache configuration items

SSLCertificateFile      /data/certs/cert-with-chain.pem
SSLCertificateKeyFile /data/certs/privkey.key

Replace Certificate in Synology NAS

Posted on by Bian Xi Leave a Comment on Replace Certificate in Synology NAS

Replace Certificate in Synology NAS

Fill up info

Following steps can be used to replace certificate (not renew) in Synology NAS user interface.

  • Go to Control Panel -> Security -> Certificate
  • Select Add -> Add
  • Select Replace an existing certificate
  • Choose the certificate to be replaced
  • Select Get a certificate from Let's Encrypt
  • Fill up info, includes domain, email, alias (seperated by semi-colons)

Change port forwarding

Now, make sure Synology NAS can be accessed from internet via port forwarding at port 80 and 443 if required.

Suggest using A * record in DNS entry to avoid DNS change. Use NGINX to redirect traffic to this host.

Generate

Then generate certificate.

DNS Configuration Basic

Posted on by Bian Xi Leave a Comment on DNS Configuration Basic

DNS Configuration Basic

Define subdomain using IP address

A record to define a subdomain to IP

A   www 192.168.1.1

Define subdomain using host

CNAME record to define a subdomain to IP

CNAME   www www.example.com

Default host for all subdomains

Following record is to give IP address of www.example.com to all subdomains.

CNAME   *   www.example.com

Default IP for domain

Use @ in A record.

A   @   192.168.1.1

Default EMAIL server

MX  mail.example.com    20

Note: The number is Priority

Install a NextCloud server using old MacBook Pro

Posted on by Bian Xi Leave a Comment on Install a NextCloud server using old MacBook Pro

Install a NextCloud server using old MacBook Pro

The plan is to install ubuntu OS on MacBook Pro with core 2 due CPU, then install NextCloud as docker container.

Install Ubuntu Server

The installation had been done on MacBook Pro with iSCSI root partition.

Refer to Ubuntu with UEFI iSCSI root on x86_64 for details.

Install NextCloudPi docker

The NextCloudPi docker image has all necessary components to run NextCloud server, and it is easy for start up.

Steps

Installation

Run following command, IP is the IP address of the server itself.

docker run -d -p 4443:4443 -p 443:443 -p 80:80 -v /app/nc/data:/data --name nextcloudpi ownyourbits/nextcloudpi-x86 $IP

Activate

Access URL https://$IP:4443/, record two pair of user id and password

  • NextCloudPi web interface at port 4443
  • NextCloud at port 443

Update patches

Login to NextCloudPi web interface (port 4443), execute tasks under Updates.

Set Maintenance Mode off

If there was a failure, server went to maintenance mode, then update config/config.php accordingly.

'maintenance' => false,

Another method is using NextCloudPi web interface (port 4443), Tools -> nc-maintenance to disable.

Configure incoming access

Enable new trust domain

Add the newly registered domain name in NGINX and Let's Encrypt in config/config.php

'trusted_domains' =>
array (
  0 => '192.168.0.29',
  1 => 'cloud.example.com',
),

Another method is using NextCloudPi web interface (port 4443), config -> nc-trusted-domains to configure.

Clients’ configuration

Using CalDAV to add account for Calendar and Contacts.
Install Password Manager App

Local by pass proxy

To by pass proxy, the internal DNS server needs to point the IP address of nextcloud server to internal server IP. For example, add alias for nextcloud server IP in dnsmasq host entry.

After that, needs to install same certificates in proxy into nextcloudpi server. There are quite number of answers in Internet, but none of them works.

End up, I changed the certificates used in apache2 configuration using following steps

Change Apache SSL certificate

References

NextCloudPi dockers for x86 and ARM
NextCloudPi docker for Raspberry Pi
How to get started with NCP docker
HowTo: Add a new trusted domain
Synchronizing with iOS
Synchronizing with macOS

BerryBoot on Raspberry Pi

Posted on by Bian Xi Leave a Comment on BerryBoot on Raspberry Pi

BerryBoot on Raspberry Pi

Raspberry Pi 2/3/4 requires SD card to be used for booting, which saves OS filesystems. There are some issues with SD card with OS boot.

  • Limited size
  • Hard to backup

I was thinking a solution to move root filesystem to iSCSI, but grub is not used in OS of Raspberry Pi. As the design of BerryBoot, it can install various OSes on iSCSI disk with multi-boot environment. So started testing it with Ubuntu OS.

Hardware

Although, BerryBoot declares supporting Raspberry Pi 1/2/3/4, I tried Pi 3B+, but very slow and hanging. End up used Pi 4 (8GB).

Boot up error

After installation completed, got following error. It is caused iSCSI LUN import requires time, at time of error reported, LUN had not detected by iSCSI driver.

iSCSI target does not have any partitions

The solution give is changing iscsi.sh file in boot partition (the SD card), adding loop for delay.

IP address and VNC

The BerryBoot interface used same IP as ubuntu OS, but it doesn't configure DNS. So after bootup, the ubuntu OS has no DNS setting, then manual update of file /etc/systemd/resolv.conf is required.

In order to remotely control BerryBoot, add following command to enable VNC. Then when booting error or like to select another OS, VNC viewer can be used to connect to BerryBoot.

Wired network headless installation

  • Append following at end of line in cmdline.txt (same line append)
... vncinstall ipv4=192.168.88.88/255.255.255.0/192.168.88.1

Wireless headless installation

  • Append following at end of line in cmdline.txt (same line append)
vncinstall ipv4=192.168.88.88/255.255.255.0/192.168.88.1/wlan0
  • Create a file called wpa_supplicant.conf
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
ap_scan=1

network={
    ssid="ssid-of-accesspoint"
    psk="wpa-password"
}

Update error

The most important task is update, which includes package update and kernel update

Package update

It is quite smooth during update, but complaining /boot doesn't exist. I checked the OS structure, there is a /boot folder, with one kernel, and a filesystem /media/user_id/boot, which saves SD contents.

Kernel update

Kernel update was totally failed with following error

Can't find /boot/vmlinuz- (see /tmp/flash-kernel-no-kernel-error.log)

The solution given by other is removing two packages

sudo apt remove u-boot-rpi:arm64
sudo apt remove flash-kernel

Note: According to internet, BerryBoot uses its own kernel, meaning it can not be updated using package update command.

do-release-update

This is a very strange action I took. The OS I installed via BerryBoot is Ubuntu 20.10 Desktop, but the OS still can perform do-release-update to same version. The even more strange thing is, after I upgrade, it still appear as can perform do-release-update, then I spent another upgrade, another few hours upgrade, then it can fininally upgraded.

I think the package was not at the correct release stage, but don't know why got 980+ packages to upgrade after first run. Felt like perform exactly the same action once more.

The output of uname after patched ubuntu screen as below

  • BerryBoot ubuntu
root@camel:~# uname -a
Linux camel 5.10.43v64 #1 SMP PREEMPT Tue Jun 15 00:52:48 CEST 2021 aarch64 aarch64 aarch64 GNU/Linux
root@camel:~#
  • Normal ubuntu
root@ubuntupi:/usr/lib# uname -a
Linux ubuntupi 5.13.0-1009-raspi #10-Ubuntu SMP PREEMPT Mon Oct 25 13:58:43 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
root@ubuntupi:/usr/lib#

Filesystem structure

The BerryBoot ubuntu has different filesystem structure, the root filesystem is mapped to none, and no entry in /etc/fstab, I think the reason behind, is root filesystem is mounted via kernel, which is the iSCSI disk, and cannot be changed.

Output of df

root@camel:~# df
Filesystem     1K-blocks    Used Available Use% Mounted on
none            16646144 6422360   8396520  44% /
tmpfs            3957172       0   3957172   0% /dev/shm
tmpfs            1582872    1404   1581468   1% /run
tmpfs               5120       4      5116   1% /run/lock
tmpfs            3957172       0   3957172   0% /run/qemu
tmpfs             791432      72    791360   1% /run/user/126
tmpfs             791432      64    791368   1% /run/user/1000
root@camel:~#

Output of mount

root@camel:~# mount | grep -w /
none on / type overlay (rw,relatime,lowerdir=/mnt/shared:/squashfs,upperdir=/mnt/data/Ubuntu_20.10_Desktop.img192,workdir=/mnt/data/Ubuntu_20.10_Desktop.img192.work,redirect_dir=on)
root@camel:~#

The normal ubuntu installed has same structure of other distributions, using /etc/fstab for root filesystem as well. So the normal operations can be taken on root filesystem

Output of df

root@ubuntupi:/usr/lib# df
Filesystem     1K-blocks     Used Available Use% Mounted on
tmpfs             794548    12732    781816   2% /run
/dev/mmcblk0p2  30358480 24453720   4573672  85% /
tmpfs            3972736        0   3972736   0% /dev/shm
tmpfs               5120        4      5116   1% /run/lock
tmpfs            3972736        0   3972736   0% /run/qemu
tmpfs            3972736     8364   3964372   1% /var/log
tmpfs            3972736        8   3972728   1% /tmp
tmpfs            3972736        0   3972736   0% /var/tmp
/dev/mmcblk0p1    258095    97201    160894  38% /boot/firmware
tmpfs             794544       92    794452   1% /run/user/126
tmpfs             794544       84    794460   1% /run/user/1000
root@ubuntupi:/usr/lib#

Output of mount

root@ubuntupi:/usr/lib# mount | grep -w /
/dev/mmcblk0p2 on / type ext4 (rw,noatime,nobarrier)
root@ubuntupi:/usr/lib#

Pros

Following advantages over normal ubuntu

  • Root filesystem type can be selected during installation. (Normal ubuntu on Raspberry Pi can not)
  • The iSCSI module is loaded by default.
    Note: Recent update of my normal ubuntu caused iSCSI module missing issue.
  • OS multiboot

Cons

Duo to BerryBoot has different filesystem structure and kernel, some diffculties would be facing later.

  • Specific Kernel
    • No kernel upgrade can cause package incompatible, for example, docker was mentioned by others
    • Auto rebuild kernel is almost impossible
  • Filesystem
    • Unable to verify root filesystem, scrubbing wasn't allowed during my testing
    • Unable to know root filesystem type even mounted
    • Cannot increase root filesystem size easily

Conclusion

The way BerryBoot manage the system is highly customized, which might not be adapted anywhere else. The customization also leads system compatibility issues later, and migration, backup, restore, etc., cannot be implemented in common way. So there could be no future-proof for this implemenation.

Compare local boot partition solution, which only holds root partition on iSCSI, the advantage of BerryBoot is iSCSI dependency issue reduced and multiboot OS capability. But the root on iSCSI uses bootload, which reduced OS compatibility issue.

My decision is still use normal bootload way for future systems to achieve maintennance free system.

References

Headless installation
Problems with update/upgrade Ubuntu 20.10 on Raspberry pi 4 with SSD
Raspberry Pi iSCSI Root on Ubuntu 20.04
How to Dual Boot a Raspberry Pi Using BerryBoot
BerryBoot v2.0 - bootloader / universal operating system installer
Storing your files on a Synology NAS (using iSCSI)

Configure different target based on incoming domain in NGINX

Posted on by Bian Xi Leave a Comment on Configure different target based on incoming domain in NGINX

Configure different target based on incoming domain in NGINX

NGINX can divert incoming request to different server based on domain name given in browser.

Usage

If there are a few application, such as 192.168.1.1 for faq.example.com, 192.168.1.2 for www.example.com, etc.

Configuration

Following configuration can be used for diverting request for faq requests.

server {
    server_name  faq.example.com;

    # SSL configuration
    listen 443 ssl;

    ssl_certificate     conf.d/www.example.com.crt;
    ssl_certificate_key conf.d/www.example.com.key;

    location / {
        proxy_pass  'https://192.168.1.1:443';
        proxy_http_version 1.1;
        proxy_buffering off;
        proxy_read_timeout    90;
        proxy_connect_timeout 90;
        proxy_redirect        off;
        proxy_set_header Host $host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Port 443;
        proxy_set_header Proxy "";
    }

    client_max_body_size 64M;
}

server {
    listen       80;
    server_name  faq.example.com;

    return 301 https://$host$request_uri;
}

Application Stopped due to upstream unreachable

Posted on by Bian Xi Leave a Comment on Application Stopped due to upstream unreachable

Application Stopped due to upstream unreachable

During Chef Server troubleshooting time, found chef is unreachable from localhost

Description

  • Nginx was shown as started in chef-server-ctl status command, pid could be found
  • TCP port 443 was inaccessable from localhost.
  • Reboot server, but still the same issue
  • Used ps -ef | grep nginx, found nginx is running without indication of master
  • Run nginx command line which found in ps
  • Shows upstream server and port was not reachable, which is automate server

Result

  • Can not find IP address, then add IP and host into /etc/host file, result shows

    • NGINX running with master indicator
    • TCP port 443 was listening

  • Still can not reach upstream server

Consolution

The problem could be related to following issues caused application stopped due to no IP can be found for upstream servers.

  • Routing issue
  • Firewall issue