Creating direct access disk node for ESXi server in order to replace dead disk on XigmaNAS

Sadly, one of hard disks for ESXi server died. I put in a replacement disk but one of VMs doesn’t start as not having the disk. So, I need to delete the old disk node, create new one, and set up in the VM.

Before installing disk, take a note on the disk serial number. (S/N) This comes in handy to ID the disk in the process. More over, you should label the S/N near SATA port to make life easier.
Log in to ESXi via SSH as root. Then, check the disks.

# ls -l /vmfs/devices/disks

You’ll see a bunch but you should be able to ID the disk you just put in if you know the S/N. Since I’m stubborn, I want to use the same “hitachi_2tb_2.vmdk”, I first deleted the vmdk file. You actually need to delete two files.

“`rm /vmfs/volumes/5bb18636-fccb7c8a-78c8-001b214ff43c/XigmaNAS/hitachi_2tb_2.vmdk /vmfs/volumes/5bb18636-fccb7c8a-78c8-001b214ff43c/XigmaNAS/hitachi_2tb_2-rdmp.vmdk“`

Then, time to recreate using the disk I just installed. Following command creates a pass-through disk for the VM.

NOTE: This step is no longer needed for ESXi 7 as you have an option to add the raw disk to VM and no need to ues vmkfstools anymore.

vmkfstools -z /vmfs/devices/disks/t10.ATA_____Hitachi_HDS722020ALA330_______________________JK11A8YBKTGMDF /vmfs/volumes/5bb18636-fccb7c8a-78c8-001b214ff43c/XigmaNAS/hitachi_2tb_2.vmdk

Now time to go to the ESXi web interface and create a new disk. First, delete the dead disk from the settings. Then “Add New Disk” and pick “Existing disk” and choose the VMDK file you just created.

You are done with the VM setting. Now, go into the XigmaNAS.
First, “Disks” > “Management” > HDD Management. If you don’t see the new device not showing up, “Import Disks” [Import].

Second, format the disk you just put in. It’s “Disks” > “Management” > “HDD Format”. Pick “ZFS Storage Pool”. Choose the disk. (Serial Number shows up here as well.) Click “Next” and format. (pretty quick.)

Third, “Disks” > “ZFS”> “Pools” > “Tools”. ZFS Pool knows that the disk is changed. Chose “Replace a device” to replace the dead disk to the new one. Once you are done, it should start recovering the mirror.

Installing Jenkins on Ubuntu 18.04LTS

Step 1: Install Java
sudo apt update && sudo apt install openjdk-11-jdk
As of August 2019, Jenkins only supports Java 8 or 11.

Step 2: Add Jenkins repo
wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'

Step 3: Install Jenkins
sudo apt update && sudo apt install jenkins

Step 4: Edit Jenkins config
For me, port 8080 is taken so I generally use port 9000.
sudo emacs /etc/default/jenkins
sudo systemctl restart jenkins.service

Install Chrome

wget -qO- https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' | sudo tee /etc/apt/sources.list.d/google-chrome.list

Using add-apt-repository using google’s source list result in error as it exports both i386 and amd64 but 18.04 doesn’t support i386 and complains.

sudo apt update
sudo apt install --no-install-recommends xorg openbox google-chrome-stable pulseaudio
sudo usermod -a -G audio $USER

Chrome Kiosk mode on Ubuntu 18.04 LTS

As of 18.04, X11 server requires root priv to run. This made Chrome kiosk mode to complain if you run Chrome as root too. Previously, the whole X11/Chrome was running as a unprev user. Here is the service unit, and starting Chrome as normal user.

UPDATE: 2023-01-10 Chrome became not-liked app in Ubunte, and Firefox has a kiosk mode too. Switching to Firefox

[Unit]
Description=Kiosk Web Browser
After=dbus.target network.target sound.target network-online.target
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=30
User=root
ExecStart=/usr/bin/startx /etc/X11/Xsession /usr/local/bin/kiosk.sh --

[Install]
WantedBy=multi-user.target

And here is /usr/local/bin/kiosk.sh

#!/bin/bash
xset -dpms
xset s off
xhost + localhost SI:localuser:
sudo -H -u  DISPLAY=$$DISPLAY openbox-session &
sudo -H -u  DISPLAY=$$DISPLAY start-pulseaudio-x11
while true; do
  sudo -H -u  rm -rf /home/triage/.{config,cache}/google-chrome/
  sudo -H -u  google-chrome --display=$$DISPLAY --kiosk --no-first-run 'http://localhost'
done

GIT private server on XigmaNAS

Here is the steps to run git private server on XigmaNAS.

  • Install git package
  • Create “git” account
  • Set up git directory
  • Set up the ssh public key auth for easy login

I know I can do this from the XigmaNAS web GUI’s command but it’s too tedious so please use the terminal of your choice. You also need a text editor most likely. MYVOLUME should be your data store of choice.

# pkg install -y git
# GITHOME=/mnt/MYVOLUME/git
# mkdir -p $GITHOME/projects
# mkdir -p $GITHOME/.ssh
# cd $GITHOME/.ssh
# ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): mygit_rsa
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in mygit_rsa.
Your public key has been saved in mygit_rsa.pub.
The key fingerprint is:


# cat mygit_rsa.pub >> authorized_keys
# chmod 600 authorized_keys

Now, you need to create “git” account. From XigmaNAS UI, Access>User&Groups, first go to Groups and add “git” group. GID can be anything so I picked a random number 3178. Then, create “git” user.

Click “+” and add “git”.
“git” user setup. “git-shell” will show up once you hack a php file.

So the “git-shell”. Unfortunately, the shell selection is not picked up from /etc/shells. I sniffed around and you need to hack the php file for this to show up. You need to be root to edit the PHP file. The file is /usr/local/www/access_users_edit.php so become root and open it with a text editor. Look for $l_shell. Add a line for git-shell. Snippet and diff follow. I use ksh a lot so I added ksh as an option as well.

UPDATE: On XigmaNAS 12, /etc/inc/system/access/user/grid_properties.php contains the list of shells.
                        $l_shell = [                                                                                                
                                'nologin' => 'nologin',                                                                             
                                'scponly' => 'scponly',                                                                             
                                'sh' => 'sh',                                                                                       
                                'bash' => 'bash',                                                                                   
                                'csh' => 'csh',                                                                                     
                                'tcsh' => 'tcsh',                                                                                   
                                'ksh' => 'ksh',                                                                                     
                                'git-shell' => 'git-shell'                                                                          
                        ];
*** access_users_edit.php.orig	2018-11-11 05:43:42.000000000 +0000
--- access_users_edit.php	2019-06-01 20:14:57.350303000 +0000
***************
*** 239,245 ****
  				'sh' => 'sh',
  				'bash' => 'bash',
  				'csh' => 'csh',
! 				'tcsh' => 'tcsh'
  			];
  			html_radiobox2('shell',gettext('Shell'),$pconfig['shell'],$l_shell,gettext('Set user login shell.'),true);
  			$l_grouplist = [];
--- 239,247 ----
  				'sh' => 'sh',
  				'bash' => 'bash',
  				'csh' => 'csh',
! 				'tcsh' => 'tcsh',
! 				'ksh' => 'ksh',
! 				'git-shell' => 'git-shell'
  			];
  			html_radiobox2('shell',gettext('Shell'),$pconfig['shell'],$l_shell,gettext('Set user login shell.'),true);
  			$l_grouplist = [];

Reloading the user setting page should bring up the git-shell for shell selection. Now you need to go back to the terminal and finish the set up.

# cd $GITHOME
# mkdir projects
# chown -r git:git $GITHOME
# chmod 775 projects

As a git server, all set. The remaining thing is to add “git” group to the users on the server so users can create new repo under projects, and handing out the private key mygit_rsa to users, or add the public key to the authorized_keys of “git” user.

Example:
Let’s say I want to have a “config.git” on the server. This repo stores all my Linux machine’s configuration files so when I have to set up a new machine, I can see how I set up my account in the past.
First, since I don’t know how to create fresh repo from client side, I will create a fresh repo on XigmaNAS.
Here is the steps:

  • SSH-Login to NAS. Since “git” account is not shell account, you have to do this as root unfortunately.
  • Create a repo directory “mkdir $GITHOME/config.git
  • Still as root, cd $GITHOME/config.git && git init --bare
  • chown -R git:git $GITHOME/config.git

From the client side, now repo is ready, if you set up the ssh keys right, you do:
git clone ssh://git@nas/~/config.git

Arduino Leonardo on macOS Mojave

Quick note – When you migrate macOS disks from old old old ones. The system extensions are copied over and stored in the “staging” part. Some are used, some are not compatible.

All I wanted to do was to program an Arduino Leonardo compatible device. Loaded up Arduino but the board didn’t show up as serial device at all. I dug (aka google) a lot but couldn’t find any. Some say to check the “Privacy” part, which I did to allow the extensions to load. The problem started there. I looked at the system profile, and there were quite a few “Disabled” extensions in the system profiler. I recognize some of them. Soundflower for routing audio over network, Dymo printer, old camcorders and digicams, etc. There are a couple of “unknown” developer extensions as well, on the latest MacBook Pro.

A lesson learned. It is not always good idea to migrate everything. I managed to create a USB disk bootable with Mojave which itself was a bit of challenge. The tech moved on and using SuperDuper to make one isn’t enough anymore. First of all, I had to reformat the disk to AFPS. Then, SuperDuper can clone the disk. Then, a few more like going into recovery mode and changing the security prefs.

Anyhow, I managed to erase the MBP internal flash, did a clean install from the external disk, then only copied the user documents and not anything else. Which meant to install a few applications again. However, it’s mostly to copy apps over from the external disk to internal so not that big of deal. In the end, ones not working (likes of old Final Cut.) was left for dead.

Now, I can install the drivers. First I thought that the Leonardo uses Silicon Labs’ USB serial, so installed. No go. I hook up the arduino to Linux laptop and works just fine. Scratched my head for a while, and realized this uses FTDI USB. Downloaded VCP driver. Voila!

I put it in a paragraph here, but it was actually a two day ordeal, from creating bootable external disk to trying out Arduino on other machines to figuring out the device and manufacturer USB IDs. For some reason, the Atmel thing didn’t show up with my google-fu. So, I banged my head after loading up Silicon Labs driver.

Anyhow, after FTDI driver, hook up Leonardo. Immediately, the “this keyboard is not recongized.” dialog shows up! Woohoo!

The driver – https://www.ftdichip.com/Drivers/VCP.htm

Fake Fake with Real

I wanted to learn something new. I know nothing about JavaScript, TypeScript or any of web framework. When I found time, I started taking a look at Angular. I don’t know I’d ever use it but it’s all the industry rage. (Friend of mine mentioned React is better, but I know nothing so I’ll see.)

I was happily following the tutorial, and when it came to the part of using fake RESTful in-memory database angular-in-memory-web-api, I hit a wall. The piece just doesn’t want to work at all. I know very little of anything at this point about npm or node or JavaScript. But, I understood that it’s mimicing the database. So, “fine. I’ll use real database instead.”.

So, I started a project, thanks to evaluation of IntelliJ, and googling like storm to find out how to write the RESTful service using PostgreSQL.

I found this post to be pretty much what I need. Just swap out the puppies with heroes. (Yes, the post uses puppy database.) But, I also wanted to use the express/router.

The part accessing database and returning records started working pretty quickly. I spent most of the day Googling and learning JavaScript, Observables, Express router, and then write a few lines of code to access database.

The pain started once I swapped out the web service URL from the fake URL of Angular Tutorial to my own. As a complete newbie, I didn’t know how to debug but it just doesn’t work at all. After a few more Google, someone said, “console output shows up on Chrome.”. I was hoping IntelliJ’s “debug” to work but I couldn’t manage it to work. (Another time to learn this.) So, I opened the tutorial’s endpoint, I can see the console log and errors! Horay!.

And the error says No Access-Control-Allow-Origin present in the header. What on earth? Yet another googling. (I don’t know how people write code before google.)

Now, I learned when you make a call from other domain (well, in this case port 4200 to port 3001.), it’s crossing boundaries and web services don’t like that kind of deal for security reason. The way to do this is to add the header. Rather than doing it by myself, I found a library “cors”. I confess that I googled. I have no clue what it does. Also, the location of app.use(cors()) seems to matter. When I moved it around, it got totally wedged. So, be careful using this piece.

app.use(cors());
app.options('*', cors()); // enable pre-flight

Second missing piece was, the Angular’s example’s accessing service didn’t like the header that node’s service returns. Googling, googling and more googling. And here was the answer. 42! No. JK.

app.disable('etag')

Apparently, this “etag” thing needs to be suppressed. I’m not quite sure of what it is (I’ll learn some day), but disabling this bit made the Angular tutorial happy.

https://github.com/ntai/realherodb

Here is the result of it. I faked the fake database with real database. Named it “Real Hero DB”