I love bash and the gnu tools provided on a Linux system. I am always writing little oneliners that help me do my work. Today I was generating some fixtures for django (yes I am finally properly learning to use unit testing in django). I wanted to know how many records where in each table in my database. Here is the little one liner I came up with:

for TABLE in $(psql foo-test -c "\dt" | awk '{print $3}'); do \
psql foo-test -c "select '$TABLE', count(*) from $TABLE;" | grep -A1 "\-\-\-\-" | tail -1; done

It produces output that looks something like this:

auth_group |     0
auth_group_permissions |     0
auth_message |     0
auth_permission |   273
auth_user |   366
auth_user_groups |     0
auth_user_user_permissions |     0
etc.

I love bash and the gnu tools provided on a Linux system. I am always writing little oneliners that help me do my work. Today I was generating some fixtures for django (yes I am finally properly learning to use unit testing in django). I wanted to know how many records where in each table in my database. Here is the little one liner I came up with:

for TABLE in $(psql foo-test -c "\dt" | awk '{print $3}'); do \
psql foo-test -c "select '$TABLE', count(*) from $TABLE;" | grep -A1 "\-\-\-\-" | tail -1; done

It produces output that looks something like this:

auth_group |     0
auth_group_permissions |     0
auth_message |     0
auth_permission |   273
auth_user |   366
auth_user_groups |     0
auth_user_user_permissions |     0
etc.

What do you do when you are managing a remote server and you need to make some critical changes (like to the networking configs) and you feel uncomfortable about the possibility of losing access to the server and never getting it back? This was the situation we were in today. The server is a little esoteric – its a headless box and even in the server center the engineers don’t have any way to log in interactively at the server itself. Luckily the server is running Debian linux and has a usb port so help is at hand via bash!

I wrote this little script which is designed to be run from a cron job, for example every minute.

#!/bin/bash

# This script is to rescue the system from usb while
# testing migration to the new vpn.

# It will mount the last partition of any inserted usb,
# cd to the mount point and try to run a script
# called 'rescue.sh'
# After the script is run it will be renamed to
# rescue.ok
#
# You should set this script to run as a cron job
# at minute intervals.
#
# e.g. # m h  dom mon dow   command
#      * * * * * /root/usbrescue.sh
#

RESCUEFILE=rescue.sh
OKFILE=rescue.ok
LOGFILE=rescue.log
MOUNTPOINT=/mnt/rescue
SCRIPTPATH=${MOUNTPOINT}/${RESCUEFILE}
OKPATH=${MOUNTPOINT}/${OKFILE}
LOGPATH=${MOUNTPOINT}/${LOGFILE}
# Note we ignore partitions on devices sda - sdd as they are internal disks
LASTPARTITION=$(cat /proc/partitions  | awk '{print $4}' | grep -v 'sd[a-d]' \
| grep -v name | grep -v '^$' |sort | tail -1)
if [ $LASTPARTITION != "" ]
then
  if [ ! -b /dev/$LASTPARTITION ]
  then
    echo "Error /dev/$LASTPARTITION is not a block device"
    exit
  else
    echo "OK /dev/$LASTPARTITION is a block device"
  fi
  echo "Device found creating mount point"
  if [ ! -d "$MOUNTPOINT" ]
  then
    mkdir $MOUNTPOINT
  fi
  echo "Mounting...."
  mount /dev/$LASTPARTITION $MOUNTPOINT
  echo "Checking if rescue script exists"
  # Test the rescue script exists(-e) and is not 0 length (-s)
  if [ -e $SCRIPTPATH -a -s $SCRIPTPATH ]
  then
    echo "Making $SCRIPTPATH executable"
    chmod +x $SCRIPTPATH
    echo "Running script"
    $SCRIPTPATH > $LOGPATH 2>&1
    echo "Disabling script"
    mv $SCRIPTPATH $OKPATH
  else
    echo "No Rescue script found"
  fi
  echo "Unmounting.."
  cd /
  umount $MOUNTPOINT
else
  echo "No rescue device found"
fi
echo "done"

If you place the script in /root/usbrescue.sh and add a cron job as outlined in the comments, it will poll for devices regularly, mount the last partition available. If it finds a script on that partition labelled rescue.sh, it will run it then rename the script to rescue.ok and write any stderror and stdout logs to rescue.log on the partition. The script could perhaps be improved by adding a lock file so that it does not get run again if it is already running (if it takes longer than a minute to run for example),  buts its a good starting point for a system rescue if things go wrong. Now the engineer on site can simply pop in his usb stick and any recovery commands will be run from it.

During the recent QGIS hackfest in Poland, we spent some time discussing the use of GIT and I spent a bit of time to learn the basics of using GIT. One thing about GIT that is different for users like myself coming from an SVN background is the way that creating and applying patches is done. Typically under svn I would do something like this to create a patch:

svn diff > foo.diff

And then to apply that same patch I would do:

patch -p0 < foo.diff

Which is all quite simple. Git’s process is a little different. Here is my workflow for creating a patch:

• Create a branch (GIT seems to encourage you to work in branches and to branch often)
• Checkout your branch
• Work and change things in your branch
• Commit your work to your branch
• Generate a patch as a diff between your branch and master
• Submit your patch
• Apply your patch

So here is a simple session that does the above:

git branch patch-testing
git checkout patch-testing
( do some work in that branch now)
git commit -m "Important changes" -a
git format-patch master

After you are done with that,  Git will have created a nice little diff for you. To apply the patch to another checkout do:

git apply foo.path

Then commit and push your changes to the origin repository.