Blog

Blog

Enable 2FA for TrueNAS Core

Enable 2FA for TrueNAS Core

The 2FA in TrueNAS Core uses pam_oath.so module, supports Two-factor time based (TOTP) SSH authentication.

Setup

Enable 2FA

  • Go to Credentials => 2FA

  • Click on Enable Two-Factor Authentication

  • Click on Show QR, use Authy to record it. This is token for root account.

  • Save

Test GUI

Use another browser login with user name, password, and pin code (Authy generated).

Make sure it is working.

Enable SSH

  • Go to Credentials => 2FA

  • Select Enable Two-Factor Auth for SSH

  • Save

Enable root login

  • Go to System Settings => Services

  • Select Configure button, which is a pencil icon

  • Check Log in as Root with Password

  • Save

Test root login with 2FA

Use terminal

$ ssh host.example.com
Password: 
One-time password (OATH) for 'root':
Linux truenas.bx.net 5.10.70+truenas #1 SMP Wed Nov 3 18:30:34 UTC 2021 x86_64

Test root login successful.

Disable root login

  • Go to System Settings => Services

  • Select Configure button, which is a pencil icon

  • Uncheck Log in as Root with Password

  • Save

Setup for normal user

After enable 2FA, normal user can not login, got error in /var/log/auth.log as below:

error: PAM: User not known to the underlying authentication module for ...

Use these steps to enable 2FA for user.

Note: If you lost SSH connection, the root shell can be accessed from GUI, System Settings => Shell

Generate a random code

# head -10 /dev/urandom | md5sum | cut -b 1-30
15ad027b56c81672214f4659ffb432

Get oath configuration file name

The usersfile name can be found using following command:

# grep oath /etc/pam.d/sshd
auth    required    pam_oath.so    usersfile=/etc/users.oath    window=0

Update /etc/users.oath

Setup the oath seed in /etc/users.oath:

HOTP/T30/6  user    -   15ad027b56c81672214f4659ffb432

Install oathtool

Use another linux server, such as ubuntu server:

ubuntu# apt install oathtool

I chose another server, because TrueNAS server is not fully customized debian server, better don't change it structure and packages.

Test pin code for SSH

Open another terminal, and run following command, and run second command in the linux server when prompting OATH code.

$ ssh host.example.com
Password: 
One-time password (OATH) for 'user':

Now, quickly run following command,

ubuntu# oathtool --totp -v 15ad027b56c81672214f4659ffb432
960776

Input OATH code in SSH login terminal. The code should be accepted.

Get Base32 secret

In the previous ubuntu server, install qrencode package

ubuntu# apt install qrencode

Run following command to collect Base32 secret:

ubuntu# oathtool --totp -v 15ad027b56c81672214f4659ffb432
Hex secret: 15ad027b56c81672214f4659ffb432
Base32 secret: CWWQE62WZALHEIKPIZM77NBS
...
329770

Generate QR code

qrencode -t ansiutf8 "otpauth://totp/user@host.example.com?secret=CWWQE62WZALHEIKPIZM77NBS"

Save into Authy

Use Authy scan QR code, then type in TrueNAS in textbox to search icon, then save it.

Persistent change

As TrueNAS is a fully customized OS, it has startup process to regenerate /etc/users.oath file, results only root id stays.

In order to overcome this issue, create a startup command in System Settings => Advanced => Init/Shutdown Scripts, add following command:

Name: Append oath codes
When: POSTINIT
Command:

echo "HOTP/T30/6\t<user_name>\t-\t<user_code>" >> /etc/users.oath

Note: There are many ways to archive this, such as backup users.oath files you created, and restore it. I just chose the most easy and maintenance free way.

TrueNAS GUI

I could not find any place to setup in TrueNAS GUI for user, and the user id I created in TrueNAS can not login to GUI at all. In fact, TrueNAS doesn't support normal user login to GUI.

Possible enhancements

There are the limitations of pam_oath.o implemenation

Only one usersfile

Only one usersfile can be specified in pam_oath.o, there are some suggestions, such as:

  • Enhance source code to allow pam_oath.o accepts %h as usersfile parameter's value to point to user's home directory.

Missing entry allowed

If the user is not in usersfile, then they can not login, this makes administrator very busy.

I like one example implementation as below:

WARNING: I didn't test the following codes which downloaed from Two-Factor Authentication with OTP (Admin Guide), just for reference.

  • Create a group called otpusers, the users are not in this group do not require 2FA. This implemented in PAM
auth [success=2 default=ignore] pam_succeed_if.so uid = 0                        # skip 2 lines for root
auth [success=1 default=ignore] pam_succeed_if.so user notingroup otpusers       # ignore users not yet in otpusers
auth requisite pam_oath.so usersfile=/var/security/auth/users.oath window=20     # accept one of 20 consecutive keys 
 (in case clocks of user and server are out of sync)
 ```

* Create profile script to check whether the user is in `otpusers` group, if not, create oath code and allow user save it.

*WARNING: The below script that I copied from Internet got syntax error, and I didn't test it as well.*

`/etc/profile.d/create_secret.sh`:

/bin/bash

RRZK, 2015-12-10 (CO)

OATH_FILE="/var/security/auth/users.oath"
OTPGROUP="otpusers"

ME=$(/usr/bin/whoami)

ME=${PAM_USER}

HOST=${HOSTNAME}

RET=0
/usr/bin/id -Gn ${ME}|/bin/grep ${OTPGROUP} >/dev/null 2>&1
RET=$?

if [ ! ${ME} = "root" ] && [ ${RET} -ne 0 ]; then

Disable CTRL-C

trap '' 2

/bin/echo -e "

Hello ${ME}

I will generate a TOTP (time based) OATH Secret for you...
"

generate secret

/bin/echo "... generating secret"
SECRET=$(/usr/bin/head -10 /dev/urandom | /usr/bin/sha512sum | /bin/cut -b 19-50)

generate base32 secret

/bin/echo "... generating base32 secret"
BASE32=$(/usr/bin/oathtool --totp -v ${SECRET}|/bin/grep 'Base32'|/bin/awk '{print $NF}')

generate qrcode

/bin/echo "... generating qrcode"
/usr/bin/qrencode -l H -v 1 --background=FFFFFF -o ${ME}_oath.png "otpauth://totp/${ME}@${HOST}?secret=${BASE32}"

insert secret in oath database

/bin/echo "... adding secret to oath database"
/bin/echo "... adding user to otpuser group"

TMPFILE=$(/bin/mktemp ) || exit 1
/bin/echo -e "HOTP/T30/6\t${ME}\t-\t${SECRET}" > $TMPFILE
/usr/bin/sudo -u root /usr/local/sbin/add_secret.sh ${TMPFILE} ${OTPGROUP} ${ME}
/bin/rm -f TMPFILE

/bin/echo "... finished"
echo "Secret: ${SECRET}
BASE32 Secret:${BASE32}" > ${ME}_oath.dat

/bin/echo "
Your Secret is: ${SECRET}
Your BASE32 Secret is ${BASE32}
Your QR-Code is: ${ME}_oath.png

Enter your secret in your OTP Token (enter BASE32 without the trailing '=')
or
Display this file and scan it with your OTP Token APP. (X11Forward only)
"
/bin/echo "To display your QR-Code, press "
read INPUT
if [ "$INPUT" = "d" ]; then
/usr/bin/display ${ME}_oath.png
fi

logout
fi


* Then add oath code into *usersfile*.

*WARNING: The below script that I copied from Internet got syntax error, and I didn't test it as well.*

`/usr/local/sbin/add_secret.sh`:

/bin/bash

RRZK, 2015-12-10 (CO)

OATH_FILE=/var/security/auth/users.oath

TMPFILE=$1
OTPGROUP=$2
USER=$3

/bin/cat ${TMPFILE} >> ${OATH_FILE}
/usr/sbin/usermod -a -G ${OTPGROUP} ${USER}
exit 0



## References

[pam_oath](https://wiki.archlinux.org/title/Pam_oath)
[Two-factor time based (TOTP) SSH authentication with pam_oath and Google Authenticator](https://spod.cx/blog/two-factor-ssh-auth-with-pam_oath-google-authenticator.shtml)
[How to Create QR Codes From the Linux Command Line](https://www.cloudsavvyit.com/8382/how-to-create-qr-codes-from-the-linux-command-line/)
[How to generate a QR Code for Google Authenticator that correctly shows Issuer displayed above the OTP?](https://stackoverflow.com/questions/34520928/how-to-generate-a-qr-code-for-google-authenticator-that-correctly-shows-issuer-d)
[Enable user to login to webui](https://www.truenas.com/community/threads/unable-to-login-to-gui-with-non-admin-root-user.19921/)
[Two-Factor Authentication with OTP (Admin Guide)](https://hpc-wiki.info/hpc/Admin_Guide_Two-Factor_Authentication_with_OTP)
[sshd: How to enable PAM authentication for specific users under](https://serverfault.com/questions/222637/sshd-how-to-enable-pam-authentication-for-specific-users-under)

Split Home Network

Split Home Network

Note: Tested using KVM network only

Let's say, you want to have a network which is different than the one ISP setup for you.

Why

  • You may want to have a guest network, which outside your home network for guests.
  • You may want to have a network with full functional network devices, firewall, DNS server, DHCP server, etc.
  • You may want to monitor the network inbound and outbound traffic.
  • You may want to setup PXE, iPXE, BOOTP environment, configure DHCP options such as TFTP server, etc.
  • You may want the VMs running on KVM routable network provide services to all machines, including Internet.
  • You may want to have more IPs, want to have IP segments.
  • You may want to have a more secure environment. In fact, if your router was setup by ISP, then they got full control of it, not you.

So, can you setup your own router to replace ISP route? Yes, you can, but you need to know whether you are allowed to do so by ISP. For my case, they asked me sign new agreement to say, service level drop and no technique support provided.

Before split

  • You have a network which connect to ISP via the ISP route, which have DHCP and DNS service.

Requirement

  • The ISP route can add route.
  • One device or router with two network interfaces, and they are or can be on separate LAN.

Simple solution

If you have another router, let's say RouterA

  • Connect WAN port of RouterA to ISP route LAN port
  • Configure ISP LAN IP on RouterA's WAN port, such as 192.168.1.100,
  • Make sure RouterA is in router mode, with it's own LAN, such as 192.168.100.0/24
  • Add a route in ISP Route, network 192.168.100.0/24 gateway 192.168.1.10.

Done.

Difficult solution

Replace RouterA with your own home build router in the simple solution.

Create timemachine share in TrueNAS

Create timemachine share in TrueNAS

Note: I got issue suddenly after sometimes, and spent hours to fix it. End up, I switched back to Synology NAS as timemachine. Check Update section.

Tried many hours on setting up time machine backup in TrueNAS Scale, encountered many issues. To save time next time, record important steps and parameters for for next setup.

Create dataset

Create a dataset called timemachine under zpool pool1. Set ACL Type of dataset to POSIX.

Create user

Creating new user/group is not a good way if the NAS is also used for other user from same machine at same time. This is because the TrueNAS will use both user to login, which is hard to troubleshoot and confusing. If do not create new user, the ownership/permission of sharing dataset is also hard to decide. To use dedicated user for timemachine, following steps can be used.

Create a new named as user id tm with group tm. It will create a sub-dataset in ZFS under timemachine dataset. Change home directory of tm to the dataset created in Create dataset section.

The Auxiliary Groups have a group named as builtin_users. Beware of this group, it will appear later in this post again.

Note: If the backup folder was copied from other system, change the owner/group to tm:builtin_users, and permission to 775 to avoid permssion issue.

Set permission of dataset

Strip ACL

This is most easy way with lesser headache. Remove ACL if ACL is wong or just use traditional UNIX permission system.

Use ACL

Change owner and group to both tm.

The dataset permission set to POSIX_HOME, this will enable owner and group both have read/write/execute permission, others will be read only.

Create share

In TrueNAS, the actual implementation for time machine done by option in SMB sharing, named Multi-user time machine, so select Multi-user time machine as Purpose, otherwise, Time Machine option will not be set and it is unchangable.

Beware of option Time Machine is selected, and Path Suffix is set to %U, both are unchangable. The %U means, a sub-dataset will be created under shared dataset for each user, they will not share the same view.

The individual host to be backed up will create a folder <hostname>.sparsebundle, which will be shown as disk in MacOS.

File/Folder permission

Remove ACL

If got strange issue on permission, uncheck ACL in SMB sharing, and then restart SMB service.

Note: The error can be verified by using Windows access share drive.

Workable permission

The actual file created in sub-dataset, will be under builtin_users group with permission 770, not tm for unknown reason. Sample output is shown below.

truenas# ls -la /mnt/pool2/timemachine/tm
total 70
drwxrwx---+ 3 tm tm               4 Oct  3 22:32 .
drwxrwxr-x+ 6 tm tm               6 Oct  3 21:23 ..
-rwxrwx---+ 1 tm builtin_users 6148 Oct  3 21:32 .DS_Store
drwxrwx---+ 4 tm builtin_users   10 Oct  3 23:02 shark.sparsebundle
truenas#

Update

Suddently, timemachine on TrueNAS stopped working, error message shows macOS could not find server.

I tried to manually map SMB drive, but failed too. I could not access the timemachine shared folder, but rest of shared folders are accessible.

mDNS

During the troubleshooting, found that macOS could not see TrueNAS in Finder, the issue could be related to mDNS missing.

Finial settings

The following settings make timemachine work again.

  • Purpose: Multi-user time machine
  • Enable ACL
  • Browsable to Network Clients
  • Time Machine (Can not change)
  • Legacy AFP compatibility
  • Enable Shadow Copies
  • Enable Alternate Data Streams
  • Enable SMB2/3 Durable Handles

But the the shared folder still requires to be opened before macOS Time Machine can see it. So, mDNS issue is still there.

Enable 2FA for Ubuntu

Enable 2FA for Ubuntu

Steps

Note: Do not logout from system before testing completed

  • Installing the Google Authenticator PAM module
sudo apt install libpam-google-authenticator
  • Configuring SSH

Add the following line at end of /etc/pam.d/sshd file

auth required pam_google_authenticator.so
  • Enable Challenge Response Authentication

Modify /etc/ssh/sshd_config

ChallengeResponseAuthentication yes
  • Disable password authentication

Modify /etc/ssh/sshd_config

PasswordAuthentication no
  • Restart the sshd daemon
sudo systemctl restart sshd.service

Configuring authentication for user

In a terminal, run following command:

google-authenticator

Anwser using recommended configuration:

Make tokens “time-base””: yes
Update the .google_authenticator file: yes
Disallow multiple uses: yes
Increase the original generation time limit: no
Enable rate-limiting: yes

Use Authy scan QR code and save it.

Testing

  • Disable user authorized_keys using root account
cd ~user
mv .ssh/authorized_keys .ssh/authorized_
  • Connect as the user using SSH

Note: Use Authy generate Verification code.

ssh host.example.com
Password: 
Verification code: 
Welcome to Ubuntu 20.04.3 LTS (GNU/Linux 5.4.0-90-generic x86_64)
  • Enable authorized_keys
mv .ssh/authorized_ .ssh/authorized_keys
  • Exit
exit
  • Connect as the user again using SSH
$ ssh host.example.com
Welcome to Ubuntu 20.04.3 LTS (GNU/Linux 5.4.0-90-generic x86_64)
  • Test sudo
$ sudo -i
[sudo] password for user:
#

Consolution

  • Disable SSH PasswordAuthentication is not effected, password is still required.

  • Enable public key authentication using authorized_keys will disable 2FA key, this is good for service account

  • Do not configure sudo, /etc/pam.d/sudo is not modified, especially when using it for none password execution, such as running in services.

  • Console access will not using 2FA, because this confiugration is for SSH.

References

Configure SSH to use two-factor authentication
How To Set Up Multi-Factor Authentication for SSH on Ubuntu 20.04
Note: This was not successful, the document is wrong at following point. I think the second publickey should not be there

AuthenticationMethods publickey,password publickey,keyboard-interactive

2FA App – Authy

2FA App - Authy

There are many 2FA Apps, Microsoft, Google, Duo, etc. But I select Authy.

Pros

As an end user, I'm not the one looking at how secure a 2FA App is, it is only generating a code as expected. But I need to two major considerations as my experiences.

  • Backup - My phone was broken, then I lost all my codes, I don't even remember what they are. Very sad experience.

    Yes, Authy backs up data to their own server.

  • Multiple Devices - To make sure my codes won't be lost, then they should be available via multiple locations.

    Yes, Authy can be used in many devices, and can track them in devices list.

  • Offline - To make sure I got my own copy on my hand, I need it works offline.

    Yes, Authy has a local copy.

  • Acceptable by others - When you hear supported by Google Authenticator, it should work.

    Yes, it supports standard TOTP.

Cons

  • Don't know how to remove the code from iOS App
  • There is a Default device in devices list, which is quite confusing.
  • How to save backup codes? No solution given.

References

Easy, 2FA for Any Account
Download Authy
8 Best Two-Factor Authentication Plugins for WordPress
Deprecating Authy for WordPress
Authy Lists an Unknown or Unrecognized Device as "Default"

NextCloud password manager – Passwords

Table of Contents

NextCloud password manager - Passwords

NextCloud password manager, Passwords is an add-on feature of NextCloud, it can be used to save passwords.

Support

It supports Firefox, Chrome, but not Safari. It also supports iOS and Android.

It doesn\'t support OS, such as SSH.

Interface

The interface is too complex compare with other system default password saving features, not user friendly.

Pros

Have notes for password, which allows saving more info.

References

Password managers for Nextcloud

Move Docker Directory

Move Docker Directory

Docker directory is /var/lib/docker. As /var should be a system directory, and most of the files in /var are log files, security related files, etc.

Docker is a application, which contains huge container data, move /var/lib/docker to data disk should be the good practice.

Methods

The are two methods I used to move /var/lib/docker directory.

Change Docker config

In /etc/docker/daemon.json file, add following entry.

{
   "data-root": "/path/to/your/docker" 
}

This makes the Docker implementation become non-standard, but as some other settings in this file also need to be set as well, such as Docker log file size, etc., this can be a normal practise for those system not many administrators.

Create soft link

Move /var/lib/docker to another location using mv command, then create a soft link /var/lib/docker point to new location.

This can be a simple and clear way, because system administrator can easily find the location without pre-check docker configuration, especially if predefined commands are required to be provided to others.

Drawback

For the first method, changing Docker configuration, although the non-standard configuration used, but Docker knows the location of data.

For the second method, adminstrator knows file location, but Docker doesn't know. So, if Docker likes to optimize the system, it might get wrong info.

References

How to move docker data directory to another location on Ubuntu

iMac waked up regularly

iMac waked up regularly

My iMac keeps waking up, although this wasn't a problem for my Mac mini, it is an issue for my iMac. The difference between is, iMac got very annoying fan noise, even no activity.

Many factors

There are so many factors to wake iMac up according to Apple, but I think the may reason is Timemachine. The option of Wake for network access in Energy Saver is for the tasks to be taken during sleep, including Timemachine. But this option is related to wake up on LAN as well, then if I need WoL, I can not turn off this option.

There is also no Do not disturb option in my this iMac, it has older macOS High Sierra.

Issue

The actual issue caused annoying problem, is Timemachine has no scheduling.

Another issue

Unlike some other OS, iMac won't maintain ssh connection after sleeped. This is a huge issue to me because I use ssh heavily. Thinking to configure VPN, and use VPN to connect to other servers, maybe can maintain connection.

References

If your Mac sleeps or wakes unexpectedly

Domain Name ICP Filing in China

Domain Name ICP Filing in China

Domain name ICP Filing in China is a process required if you want to open a web page in China, I spent more than 2 days, but still can not get it done.

Blocking if no ICP Filing

If no ICP Filing, Cloud provide will block the server access according to TCP package info, meaning they will drop the TCP packet if there is a domain name did not complete ICP Filing.

Certification Required

If you like to do ICP Filing, then you need to provide Certification of Domain Name.

I bought domain name from Geocity last time, then they changed to Yahoo, now they move to Verizon... Anyway, none of them provided such certificate... I sent request to Verizon helpdesk, no reply at all...

Buy a new one

Believe me, no point to argue with anyone in China, you will always be a loser. So, buy the new one in Cloud service provider. Then...

WARNING

Registrant Profile

According to the requirements of ICANN policies, the domain name holder's email address must be truthful and accurate. Before using a template, you must complete email address authenticity verification.Learn More

Tips:1. The common domain names(None CN domain) you bought at Alibaba Cloud can not be either ICP filinged in the mainland of China or pass the Real Name Verification(RNV).
2. The CN domain names you bought at Alibaba Cloud, request by CNNIC, please complete real name verification in time, or else you cannot use it normally. And even you have done for real name verification, it still cannot be ICP filinged.

Look carefully of the last statement...

Then what?

You buy one with CN domain name, passed real name verification, but still can not... Then why buy such domain?

References

GoChina ICP Filing Assistant