Author: Naoyuki Tai

Create Postgresql database and allow remote access on Ubuntu

  1. Log in to the postgres server
    ssh me@pgserver
  2. Become postgres user.
    sudo su - postgres
  3. Create database
    createdb mynewdatabase
  4. Create new user
    createuser -P newuser
  5. Give the new user access over network
    Edit /etc/postgresql/11/main/pg_hba.conf file. (well, 11 depends on the version of postgres you are using.) The CIDR depends on your network.
#
host    all             newuser        192.168.1.0/24         md5
  1. Make sure the server is listening to remote
    Edit /etc/postgresql/11/main/postgresql.conf.
listen_address = '*'
  1. Restart postgresql
    sudo systemctl restart postgresql

So what just happened here. When you create a new user, Postgres doesn’t know the access right of the user. It kind of sucks that you need to edit pg_hba.conf file in order to just add a new remote user.

Now, try
psql -h pgserver -U newuser newdatabase

Azure Two-Factor Authentication from Mac Terminal

tell application "Terminal"
    activate
    tell application "System Events"
        tell application process "Terminal"
            tell menu bar 1
                click menu item "Copy" of menu "Edit"
            end tell
        end tell
    end tell
end tell

tell application "Google Chrome"
    activate
    open location "https://login.microsoftonline.com/common/oauth2/deviceauth"
    delay 2
    tell application "System Events"
        tell application process "Google Chrome"
            tell menu bar 1
                click menu item "Paste" of menu "Edit"
            end tell
        end tell
    end tell
end tell

Waking up Ubuntu with USB keyboard

The new machine – Ubuntu 20.04LTS / Ryzen 3700x / Asus TUF GAMING B450M-PLUS, is working well.
Minor glitch is that it doesn’t want to wake up from keyboard.

# grep . /sys/bus/usb/devices/*/power/wakeup
/sys/bus/usb/devices/1-10/power/wakeup:disabled
/sys/bus/usb/devices/1-7.1/power/wakeup:enabled
/sys/bus/usb/devices/1-7.2.3.1/power/wakeup:enabled
/sys/bus/usb/devices/1-7.2.3.2.3/power/wakeup:enabled
/sys/bus/usb/devices/1-7.2.3.2/power/wakeup:disabled
/sys/bus/usb/devices/1-7.2.3.4.4/power/wakeup:enabled
/sys/bus/usb/devices/1-7.2.3.4/power/wakeup:disabled
/sys/bus/usb/devices/1-7.2.3/power/wakeup:disabled
/sys/bus/usb/devices/1-7.2/power/wakeup:disabled
/sys/bus/usb/devices/1-7/power/wakeup:disabled
/sys/bus/usb/devices/1-9/power/wakeup:disabled
/sys/bus/usb/devices/3-1/power/wakeup:enabled
/sys/bus/usb/devices/usb1/power/wakeup:disabled
/sys/bus/usb/devices/usb2/power/wakeup:disabled
/sys/bus/usb/devices/usb3/power/wakeup:disabled
/sys/bus/usb/devices/usb4/power/wakeup:disabled

I am guessing that /sys/bus/usb/devices/usb[1-4] are the root hub. Ones enabled is prob. keyboard and mouse. (For testing I plugged in 2nd set of keyboard mouse.) It might even wake up from bluetooth. (Again, I am guessing 1.7 is usb/bluetooth). So, only thing not cooperating is the root hubs.
I may be wrong, but I’m pretty sure. That means, I want to enable the root hubs to be able to relay the wakeup. (OTOH, this might be wrong, and the machine might wake up too often.)
Anyhow, it’s time to experiment.

First, something needs to happen to the root hub so the wakeup state becomes enabled. Here is my /usr/local/bin/enable-usb-roothub-wakeup.

#!/bin/sh

for roothub_wakeup in /sys/bus/usb/devices/usb*/power/wakeup ; do
  echo 'enabled' > $roothub_wakeup
done

chmod +x /usr/local/bin/enable-usb-roothub-wakeup
to make it executable.

Second, you need to run this thing at start up. You need a systemd’s unit file.
Here is my /etc/systemd/system/usb-wakeup.service

[Unit]
Description=Configure USB wake up for root hub

[Service]
Type=oneshot
ExecStart=/usr/local/bin/enable-usb-roothub-wakeup

[Install]
WantedBy=basic.target

Once you create the file,
systemctl daemon-reload to load the unit file, systemctl enable usb-wakeup so this would run at start up. Also, you could do systemctl start usb-wakeup.

I am hoping this fixes up the usb wakeup. I’ll update once the experiment’s result comes back.

MacOS X Xquarz and xmodmap

I finally bit a bullet and set up the keymap for Mac. Here is my current ~/.Xmodmap file content.

! left-alt 66
! right-alt 69
! left-command 63
! right-command 71
! right-menu 118

clear Mod1
keycode 118=Alt_L
add Mod1 = Alt_L

clear Mod2
keycode 66=Meta_L
keycode 69=Meta_R
add Mod2 = Meta_L Meta_R

clear Mod4
keycode 63=Super_L
keycode 71=Super_R
add Mod4 = Super_L Super_R

The key to figuring out the keycode is to use a command xev and it prints out all of events to stdout. Run it in xterm, press keys, and you can see what X wants.

I thought that Mod1 is the Meta, but somehow it didn’t work. I used Mod2 for Meta. Mod4 is Super.

My fingers are extremely happy to use Emacs under Xquartz at last.

Django CORS headers – You want to use ‘Cache-Control’

I was happily writing React app, and at one point, I realized I have to not have any cache at all to access Django API. So, I added,

headers: { "cache-control": "no-store" }

Then, Django API complains. I knew I already set up CORS middleware so I thought I am all set. For development, Django/React are being developed and need to do a bit of HTTP dance so I disabled the CORS check, I thought.
Long sotry short, as I read the code, what React app is complaining is that the “cache-control” is not part of approved headers.
Take a look at "site-packages/corsheaders/conf.py"

class Settings:
    """
    Shadow Django's settings with a little logic
    """

    @property
    def CORS_ALLOW_HEADERS(self):
        return getattr(settings, "CORS_ALLOW_HEADERS", default_headers)

    @property
    def CORS_ALLOW_METHODS(self):
        return getattr(settings, "CORS_ALLOW_METHODS", default_methods)

    @property
    def CORS_ALLOW_CREDENTIALS(self):
        return getattr(settings, "CORS_ALLOW_CREDENTIALS", False)

    @property
    def CORS_PREFLIGHT_MAX_AGE(self):
        return getattr(settings, "CORS_PREFLIGHT_MAX_AGE", 86400)

    @property
    def CORS_ORIGIN_ALLOW_ALL(self):
        return getattr(settings, "CORS_ORIGIN_ALLOW_ALL", False)
...

In my setting, I did "CORS_ORIGIN_ALLOW_ALL=true"so that part Okayes any origin, but to use cache-control header, you now have to set "CORS_ALLOW_HEADERS".
Since setting CORS_ALLOW_HEADERS in settings.py completely replaces the value, I copied the default header, and set it as follows:

CORS_ALLOW_HEADERS = [
    "accept",
    "accept-encoding",
    "authorization",
    "content-type",
    "dnt",
    "origin",
    "user-agent",
    "x-csrftoken",
    "x-requested-with",
    "cache-control",
    "credentials",
    "http-access-control-request-method",
]

# if you want to allow all. You could use whitelist instead.
CORS_ORIGIN_ALLOW_ALL = True

Now, my React app is back to accessing Django.

Refreshing JWT Access Token with Redux – Concept

I spent multiple days trying to figure out how to refresh JWT access token in Redux store. (Getting JWT token in first place – some other time but it’s straightforward.)

My React app works for like 5 minutes, and gets “access denied”, and it turns out the JWT access token has the life expectancy of 5 minutes, and you have to use refresh token to get new access token once it expires.

I googled and everyone is solving the problem using “stash the actions, create new JWT toekn refresh action, do that first, and re-run the stashed actions”.
This works fine as long as you don’t use Redux Thunk to do some kind of processing after dispatching action. In my case, Material Table needs the result to be given back to the table data once the fetch succeeds, and I was using Thunk for it.

This approach – “stash actions and refresh JWT token” does not work for me. This morning, I asked myself “is there a different way?”, and I answered myself – “if a token is expiring, refresh it before it expires with timer.”

When you store the JWT access token, parse it, get the expiration, and set a timeout 30 seconds before it expires, and refresh the token.
The all of changes needed 30 minutes. I already had all the Redux actions and types shaped out. Rewiring of refreshing access token with timeout took very little time, and I get to keep the existing Thunks’ post-action processing.

Only downside – the access token may be refreshed if the user is dormant for long time and no need to refresh the token. if you are using the web app intereactively, this would probably be very neglegible downside. Overall, this is a big win for me.

If you came here googling “Refresh JWT Access Token Redux”, don’t follow the examples out there. Use setTimeout or setInterval.

I will prep some code, and write another post about this.

Django testing with django-pytest

Pfff. Just finished writing mere 3 tests, and took me a long time. As usual, it’s probably a good idea to read the docs carefully. I didn’t and it bit my shiny hiny badly.

This is about “Testing Transaction” of Django pytest. Quote “Django itself has the TransactionTestCase which allows you to test transactions and will flush the database between tests to isolate them.”

I wrote a small test fixture that creates a few records. However, as each test runs within single transaction, subsequent query against database does not return the records created by the fixture function. For example, as you create a user, and the post-save creates one-to-one user profile, quering the user against database in normal operation does work but not for the test, as the User records have not been commited to the database yet. This puzzled me like a day. First, I checked that the database connection is right, the database is working, etc. After checking this, it became clear that the test is running in a transaction, and I had to commit for the fixtures to be on the database so my tests can query.

As a Django and pytest noob, this took me a day to get to a meaningful google question, and I hit this blog titled “Transaction tests in Pytest”. I finally saw a magic keyword “transaction=True” in the db test marker, and went back to the doc.

“Testing Transactions” is right under “Enabling Database Access in Tests so I should not have missed, but I glanced over and didn’t realize what it meant.

My excuse was that, when I was writing the test, I did something funky and the fixture setup was running before the test transaction in place. After I refactored the test files around, the feature kicked in, and made me really confused. So, this was my stupidity, and this is what I learned today.