Setting up a server for diskless clients using Ubutnu 18.04LTS server

  • Step 1 – install Ubuntu server
  • Step 2 – configure network interfaces
  • Step 3 – install dnsmasq
  • Step 4 – install atftpd
  • Step 5 – install kernel-nfs-server
  • Step 6 – struggle

For installing Ubuntu server, do whatever needed. Not going to describe. You don’t need to install any extra package but openssh server so you can log into it after installation. Unless you are going to install X11-based desktop on it, you want a web browser and terminal running at same time.

It’s far easier to have 2nd network interface, one for actual network clients and one for connecting to the server from other machine. You are going to run dnsmasq for DHCP/PXE boot so you do not want to run it on your existing network. For the machine I am installing, unfortunately has one Ethernet port so I’m using a USB Ethernet adapter.

Existing netplan is at /etc/netplan/50-cloud-init.yaml which is created at installation time. I don’t need this so I deleted it. netplan reads every .yaml file so you can name it as you like. Just for the good measure to make sure it’s not coming back, I followed the message in it and created /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg file

network: {config: disabled}

Then proceed to create a new network set up. I named it as /etc/netplan/installation-server.yaml. As you can see, I chose 10.3.2.0/24 as the subnet. Make sure there is no TAB character in this file. (I did, and netplan doesn’t like it.)

network:
  version: 2
  ethernets:
    enx803f5d08c3a0:
      optional: true
      dhcp4: true
    enp3s0:
      optional: true
      dhcp4: false
      addresses: [10.3.2.1/24]
      gateway4: 192.168.10.1
      nameservers:
        addresses: [10.3.2.1]
# netplan generate
# netplan apply
# # wait a couple of seconds for new DHCP lease
# ip addr

Install dnsmasq (sudo apt install dnsmasq) and proceed to set it up for PXE boot. You can make this fancy to put a config in /etc/dnsmasq.d but let’s go for simpler route and write up the /etc/dnsmas.conf. In case you want to see what’s in the original, keep the original in /etc/dnsmasq.d-available. (mv /etc/dnsmasq.conf /etc/dnsmasq.d-available). The new /etc/dnsmasq.conf looks as follow. It’s important to not offer DHCP on the network you are using. Oh, the server name is “wcesrv”. 

no-dhcp-interface=enx803f5d08c3a0
no-hosts
expand-hosts
no-resolv
#
address=/wcesrv/10.3.2.1
#
dhcp-range=10.3.2.100,10.3.2.199,2h
# router and dns server
dhcp-option=3,10.3.2.1
dhcp-option=6,10.3.2.1
#
pxe-service=x86PC, "Boot from local disk"
pxe-service=x86PC, "Install WCE Ubuntu", pxelinux

dnsmasq can do tftp but I’m not going to use it. I use “atfpd” with inetd. There is not a lot of reasons for this but I find it easier to understand the logs coming out from tftpd during setting up. Install “atftpd”. (not sure atftpd is any better than tftpd but I like the name “advanced”. Not a great reason.) apt install -y atftpd. Then tell inetd to start it. Following is the content of /etc/inetd.conf. When you install atftpd, it adds a line to inetd.conf. Depending on the package version, the line may different. Only thing to make sure is the last arg – /srv/tftp. I’m going to use /var/lib/netboot

tftp		dgram	udp	wait	nobody /usr/sbin/tcpd /usr/sbin/in.tftpd --tftpd-timeout 300 --retry-timeout 5 --mcast-port 1758 --mcast-addr 239.239.239.0-255 --mcast-ttl 1 --maxthread 100 --verbose=5 /var/lib/netboot

Install nfs-kernel-server. Set up the client’s file system, and export it. I’m using /var/lib/netclient. Then, populate the files under /var/lib/netclient. I find it easier to use rsync. I create /var/lib/netclinet/wcetriage.0.1.20 subdirectory and copy the installer there. This is so if I want to try out a new version, I can have separate set, and easier for the future. Reload the config afterward. “reload” is probably doing “exportfs -a” underneath. You can manually do so by “exportfs -av” (where v is verbose.)

/var/lib/netclient  *(ro,sync,no_wdelay,insecure_locks,no_root_squash,insecure,no_subtree_check)
# systemctl restart nfs-kernel-server.service

Install PXELINUX and populate them to the client file system. apt install -y pxelinux syslinux. Then copy. Use “-p”. Actual modules is in the package “syslinux-common”. I don’t need all those modules but disk is cheap. I think I need dhcp, disk, elf, gfxboot,ldlinux, pxechn, vesa, vesainfo, vesamenu. (not 100% sure, and also changes depending on the pxelinux.cfg.)

cp -p /usr/lib/PXELINUX/pxelinux.0 /var/lib/netboot/
cp -pr /usr/lib/syslinux/modules/bios/* /var/lib/netboot/
mkdir /var/lib/netboot/pxelinux.cfg

Here is the “default” file (/var/lib/netboot/pxelinux.cfg/default)

DEFAULT vesamenu.c32
TIMEOUT 100
TOTALTIMEOUT 600
PROMPT 0
NOESCAPE 1
ALLOWOPTIONS 1
# MENU BACKGROUND wceboot2.png
MENU MARGEIN 5

MENU TITLE WCE PXE Triage

LABEL WCE Triage
  MENU DEFAULT
  MENU LABEL WCE ^Triage
  KERNEL wcetriage.0.1.20/vmlinuz
  APPEND initrd=wcetriage.0.1.20/initrd.img hostname=bionic nosplash noswap boot=nfs netboot=nfs nfs=on nfsroot=10.3.2.1:/netclient/wcetriage.0.1.20/ toram acpi_enforce_resources=lax edd=on ip=dhcp ---
  TEXT HELP
  * WCE Traige V2 alpha 0.1.20
  ENDTEXT

LABEL Local
  MENU LABEL Local operating system in harddrive (if available)
  KERNEL chain.c32
  APPEND sda1
  TEXT HELP
  Boot local OS from first hard disk if it's available
  ENDTEXT

Now, let’s fun begin. (not really.) First test is to make sure dnsmasq serves DHCP. Connect a computer to the dnsmasq serving port and test. Gigabit ethernet’s auto-negotiation does wonder and you don’t even need a hub. Just a cable, and point-to-point is just fine. If netplan/network service and dnsmasq are working, you should get a DHCP lease.

Then, reboot the computer and try PXE boot. Watch the tftp server log. Hummm. TFTP is not working. Seems like I forgot to reload inetd.conf.

systemctl restart inetutils-inetd.service

I actually stopped the inetd, and run the inetd with “-d” debug option so I can observe. I made a mistake in /etc/inetd.conf, giving a wrong directory for tftp. So, I stopped inetd, and restart after changing inetd.conf. Then, I tried again. Still no go. It turns out that vmlinux is 0600 so inetd/tftp cannot read it. initrd file has 0644 so that would work. This must be one of those newer security thing. For now, solution is to copy the file and set different permission. I’m afraid to change the file perm of original. I created a directory in netboot (/var/lib/netboot/wcetriage.0.1.20), and copied the vmlinuz and intird.img from /var/lib/netclient/boot/. Now the boot process reads them, and stops at mounting NFS file root and drops into busybox. Progress!

Looks like nfsroot= needs a full path. Set it to root=10.3.2.1/var/lib/netclient/wcetriage.0.1.20 and try again.

Now, the error message is “mount: cant’ find /root in /etc/fstab”. After couple of hours of head scratching, it came to the conclusion. The /var/lib/netclient hosted file system has no nfs client installed. (Doh!) I installed “nfs-common”, and repopulated the client file system Also the fstab. Not only it needs to mount nfs as ‘/’, it also needs /tmp, etc.

#                
proc            /proc           proc    nodev,noexec,nosuid 0       0
#
10.3.2.1:/var/lib/netclient/wcetriage.0.1.20  /               nfs   (soft,rsize=32768,wsize=32768,proto=tcp,nolock)   0       0
#
none            /tmp            tmpfs   defaults        0       0
none            /var/run        tmpfs   defaults        0       0
none            /var/lock       tmpfs   defaults        0       0
none            /var/tmp        tmpfs   defaults        0       0

At this point, it boots, but a service – X server is failing. The log says – /root/.Xauthority. Right. It is mounted as read-only. The client side has “aufs as root file system” thing put in, but I try to boot with it, it fails without any further information. It turns out, the exported NFS root file system doesn’t have the mount points for /aufs, /ro and /rw that I used for the aufs to work. I created the three directories, and it finally booted correctly. So the final bootstrap config file looks as follow. Also, I did not mention about how to do “aufs=tmpfs” part. This comes from Ubuntu community doc, and it mostly works. (Yes, mostly… cuz, it was written for older kernel.) 

DEFAULT vesamenu.c32
TIMEOUT 100
TOTALTIMEOUT 600
PROMPT 0
NOESCAPE 1
ALLOWOPTIONS 1
# MENU BACKGROUND wceboot2.png
MENU MARGEIN 5

MENU TITLE WCE PXE Triage

LABEL WCE Triage
  MENU DEFAULT
  MENU LABEL WCE ^Triage
  KERNEL wcetriage.0.1.20/vmlinuz
  APPEND initrd=wcetriage.0.1.20/initrd.img hostname=bionic nosplash noswap boot=nfs netboot=nfs nfs=on nfsroot=10.3.2.1:/netclient/wcetriage.0.1.20/ toram acpi_enforce_resources=lax edd=on ip=dhcp aufs=tmpfs ---
  TEXT HELP
  * WCE Traige V2 alpha 0.1.20
  ENDTEXT

LABEL Local
  MENU LABEL Local operating system in harddrive (if available)
  KERNEL chain.c32
  APPEND sda1
  TEXT HELP
  Boot local OS from first hard disk if it's available
  ENDTEXT

Leave a Reply

Your email address will not be published. Required fields are marked *