How to fix slow DNS look-up’s in Firefox running under Linux

I’ve had this problem for some time now and chose today to do some research. Luckily, I found a workaround pretty quickly.

I use the Mozilla based browser GNU IceCat which is functionally equivalent to Mozilla Firefox. When a page is requested, the browser displays a small spinning icon in the left corner of the tab. Loading a page involves three basic steps: converting the URL into an IP address, contacting the web server at the IP address, receiving the web page (and other resources). During the first two steps, the icon will be grey and spinning counter-clockwise. The status bar will show messages like “Looking up google.com” and “Waiting for google.com”. Typically, DNS queries (the first step) take on the order of 100 ms to complete. Therefore if you see the first message, you may have the same problem.

The symptom was that IceCat (Firefox) was taking about 3 to 5 seconds to make DNS queries every time I tried to navigate to a new address. The cause of the problem seems to be that my system is making IPv6 DNS queries first by default but these types of requests are being ignored by the DNS resolver which doesn’t support IPv6. My system sends a query and then waits a certain amount of time before giving up and trying an IPv4 query. That time waiting was the delay I was experiencing.

The ideal fix would be to change the way the DNS resolver responds to IPv6 DNS queries. That’s obviously out of my control. A system-wide fix involves setting up your own DNS resolver – a bit more work than I’m interested in. The easy fix was to address the issue for IceCat (Firefox) only which is good enough for me as web-browsing is my primary internet activity.

The fix is to change the value for the key “network.dns.disableIPv6” from “false” to “true” in IceCat’s (Firefox’s) “about:config” page.

For more information, the issue has been discussed extensively here:

https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/417757

The ‘Register’ page is now the ‘Log’ page

I renamed the ‘Register’ page to the ‘Log’ page because the former was misleading. While I used the term as a noun (a list or record) it is more typically used online in its verb form (as in ‘to enroll’). Given the context, I decided that ‘log’ was less confusing. I refuse to use the term ‘blog’, since that term more often than not refers to useless narcissistic ramblings made specifically with a public audience in mind. In my case, my ‘log’ is as much for me as anyone else.

Edit 07/26/14:

Can someone tell me why this is the most popular page on this site? I’m very confused.

Binder clip alligator clip

I needed alligator clips to connected some wires from a heater to the terminals of a power source but didn’t have any. The solution I came up with was to solder small metal binder clips on the ends of the wires instead.

To prepare the binder clip, I scrubbed the bottom, nearly flat side with steel wool and applied flux to the surface. I wetted the entire bottom of the cleaned surface with solder. I also pre-tinned the wired. Connecting the two went smoothly, simply heating both simultaneously with the iron until the solder melted on both and merged to a single junction. I’m really happy with this solution since I rarely have alligator clips handy whereas the opposite is true of binder clips.

binder alligator clip

How to make a hole in a light bulb without shattering it

In order to use a light bulb to illuminate the inside of a vacuum chamber, I needed to release the gas contained within the bulb else it would likely have exploded the bulb when the external pressure was reduced. However, I wanted to keep the glass intact in order to protect the fragile filament from my clumsy hands and to provide that soft white glow.

I first tried scoring and breaking only a small piece of the bulb, but that always resulted in an uncontrolled cracking/shattering of the entire bulb. The best suggestion I found on the internet involved using a CO2 laser to make the hole. That would probably work, but I’ll save building a CO2 laser from scratch for another day. Other suggestions typically involved drilling under running water at very low speeds with a very sharp drill bit. Since both patience and drills were in short supply, I tried another method.

I found that by using a metal file at a tangent to the glass, one can very slowly thin the glass in a relatively small area. In my experience, eventually the thinned glass breaks away leaving a small hole. The hole isn’t round and doesn’t form in an absolutely controllable way. Perhaps it may be possible to “punch” a smaller hole after thinning the glass but before it breaks.

light bulb

Automatically mount USB drives in Linux without a GUI

The helloNULL web server is a headless system, i.e. it does not run a windowing/desktop environment. One feature that is therefore lacking is the automatic mounting of USB flash drives upon plugging them in. For example, in the Debian/XFCE environment of my mobile desktop, a flash drive will be mounted to /media/label upon plugging in to the port, where ‘label’ is the drive label. This post will show you how to duplicate that functionality on a terminal-only system using udev rules and a simple script.

When a USB drive is plugged in udev handles initiating the device and by using udev rules, it is possible to specify a script to be run whenever this event occurs. In Debian, udev rules are written in text files under /etc/udev/rules.d/. Each file in that directory is parsed by order of filename. Rules that run scripts should begin with a  number in the 80’s. Our script can be called /etc/udev/rules.d/81-custom.rules and should contain the following code:

# Custom udev rules. Note that udev rules are run in order based
# on the filename. Rules in filenames that start with numbers
# are run before this one.
# This rule detects USB drives that are added to the system so
# that they can be mounted automagically by the script.
# It is based off of the replies to a question on superuser.com.
# http://superuser.com/questions/53978/ubuntu-automatically-mount-external-drives-to-media-label-on-boot-without-a-u
ENV{ID_FS_USAGE}=="filesystem", SUBSYSTEMS=="usb", ACTION=="add", RUN+="/root/auto-usb.sh mount %k"
ENV{ID_FS_USAGE}=="filesystem", SUBSYSTEMS=="usb", ACTION=="remove", RUN+="/root/auto-usb.sh cleanup %k"

The first rule will trigger whenever a usb device (SUBSYSTEMS) is added (ACTION) and the device contains a filesystem (ENV{ID_FS_USAGE}). The second rule is similar but triggers upon a ‘remove’ event. In either case, udev will add the script (RUN) specified to the list of scripts to be run for that event. Our script has two functions, mount and cleanup, depending on the event. In the former case, our script will need to know the kernel name of the device (e.g. sda1) to use for the mount command. The auto-log.sh script looks like this:

#!/bin/bash
#
# This script is called by a udev rule whenever a usb drive is
# plugged into the system or removed from the system.
# Usage:
# auto-usb.sh MODE DEVICE
# where MODE is either mount or cleanup
# and DEVICE is the device basename, i.e. /dev/DEVICE

if [ "$1" = "mount" ]; then
    # The ID_FS_LABEL enviroment variable is only available
    # When this script is caleld by udev
    mkdir -p "/media/$ID_FS_LABEL"
    $(mount | grep -q "/media/$ID_FS_LABEL") || mount /dev/$2 "/media/$ID_FS_LABEL"
elif [ "$1" = "cleanup" ]; then
    rmdir "/media/$ID_FS_LABEL"
else
    echo "ERROR: Mode $1 should be 'mount' or 'cleanup'."
fi

This is pretty straight-forward. In the case of mounting a usb drive, the device name is created from the passed argument while the mount point is generated from the disk label taken from an environment variable. The same environment variable is used to remove the mount point when the disk is removed. The proper way to remove the drive is with the eject command, like this:

That’s it! This is perhaps not the most robust solution but should work for most cases. Put the 81-custom.rules file in your /etc/udev/rules.d/ directory and the auto-usb.sh script in your /root directory (or wherever you want). Don’t forget to give the auto-usb.sh script execute permissions.

Edit 06/03/2013: I changed my mount line to give read-write permissions to the plugdev group. The plugdev group is for users who are allowed to mount and unmount removable devices. On my system, all people-users (as opposed to system accounts) are members of the plugdev group. This means that any real person logged in can access the mounted USB drive. The new mount line looks like this:

$(mount | grep -q "/medial/$ID_FS_LABEL") || mount -o gid=plugdev,dmask=007,fmask=117 /dev/$2 "/media/$ID_FS_LABEL"

Edit 12/16/2013: I discovered that my mount command from the 06/03/2013 change only works right for file systems of type vfat and would fail if the usb drive connected was formatted as ext3, for example. The problem is the options specified (gid, etc.) which aren’t valid for ext3 and ext4 filesystems which preserve permissions. The solution I decided on was to specify two different udev rules, one for each type of file system. Different mount options will be set in an environment variable depending on the file system detected. Then in my auto-usb.sh script, this environment variable is put directly into the mount call.

New udev lines:

# the mount options are different depending on the file system type
ENV{ID_FS_USAGE}=="filesystem", ENV{ID_FS_TYPE}=="vfat", SUBSYSTEMS=="usb", ACTION=="add", ENV{MOUNT_OPTIONS}="-o gid=plugdev,dmask=007,fmask=117", RUN+="/root/auto-usb.sh mount %k"
ENV{ID_FS_USAGE}=="filesystem", ENV{ID_FS_TYPE}=="ext?", SUBSYSTEMS=="usb", ACTION=="add", ENV{MOUNT_OPTIONS}="", RUN+="/root/auto-usb.sh mount %k"

New auto-usb.sh lines:

$(mount | grep -q "/media/$ID_FS_LABEL") || mount $MOUNT_OPTIONS /dev/$2 "/media/$ID_FS_LABEL"

Rebuilding a Raid 1 array: Using mdadm to add the new drive back to the array

Because of frequent disk failures that occur in my Mobile Desktop for reasons I’ve yet to conclusively identify, I decided early on to run two drives in a Raid 1 array. In this way, a single drive failure was easily overcome. A week or so ago one of my current drives started producing read errors and it was removed from the array by mdadm (linux’s multi-disk administrator). Today I’m adding the failed drive back to the array (currently only 1 active drive) and decided to post the commands I use so I can find them easily in the future. For more information on creating, administering, and repairing Raid arrays in linux, I suggest an article on howtoforge.com by Falco Timme.

As of right now, my Raid 1 array (md0) has one connected drive (really partition), /dev/sda1. The drive (partition) that I want to add back to the array is /dev/sdb1. Both drives are from Western Digital but sda is a 250 GB disk while sdb is a 320 GB disk. Because sdb was previously in the array, there’s nothing special that I’ll need to do to use it, but I will be limited to using only 250 GB of the space on that drive. Here is the output from some mdadm commands to clarify the picture:

ecellingsworth@MD1-LMDE ~ $ cat /proc/mdstat
 Personalities : [raid1]
 md0 : active raid1 sda1[3]
 244194841 blocks super 1.2 [2/1] [_U]
 ecellingsworth@MD1-LMDE ~ $ sudo mdadm --detail /dev/md0
 [sudo] password for ecellingsworth:
 /dev/md0:
 Version : 1.2
 Creation Time : Thu Sep 15 10:36:29 2011
 Raid Level : raid1
 Array Size : 244194841 (232.88 GiB 250.06 GB)
 Used Dev Size : 244194841 (232.88 GiB 250.06 GB)
 Raid Devices : 2
 Total Devices : 1
 Persistence : Superblock is persistent
Update Time : Fri May 31 11:41:27 2013
 State : active, degraded
 Active Devices : 1
 Working Devices : 1
 Failed Devices : 0
 Spare Devices : 0
Name : MD1-Ubuntu:0
 UUID : 7d0d271d:04fdd2c1:de65ca3f:4c375489
 Events : 1279820
Number   Major   Minor   RaidDevice State
 0       0        0        0      removed
 3       8        1        1      active sync   /dev/sda1

First I copy the partition table from the good drive to the failed drive. If you use any of these commands yourself, be sure you use the right drive identifier. If you switch sda and sdb, you will destroy the good drive.

ecellingsworth@MD1-LMDE ~ $ sudo sfdisk -d /dev/sda | sudo sfdisk --force /dev/sdb
 Checking that no-one is using this disk right now ...
 OK
Disk /dev/sdb: 38913 cylinders, 255 heads, 63 sectors/track
 Old situation:
 Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start     End   #cyls    #blocks   Id  System
 /dev/sdb1   *      0+  30400   30401- 244196001   fd  Linux raid autodetect
 /dev/sdb2          0       -       0          0    0  Empty
 /dev/sdb3          0       -       0          0    0  Empty
 /dev/sdb4          0       -       0          0    0  Empty
 New situation:
 Units = sectors of 512 bytes, counting from 0
Device Boot    Start       End   #sectors  Id  System
 /dev/sdb1   *        63 488392064  488392002  fd  Linux raid autodetect
 /dev/sdb2             0         -          0   0  Empty
 /dev/sdb3             0         -          0   0  Empty
 /dev/sdb4             0         -          0   0  Empty
 Successfully wrote the new partition table
Re-reading the partition table ...
If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
 to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
 (See fdisk(8).)

Now I remove any traces of the previous Raid information from the failed drive (partition).

ecellingsworth@MD1-LMDE ~ $ sudo mdadm --zero-superblock /dev/sdb1

All that’s left to do is put /dev/sdb1 back into the array.

ecellingsworth@MD1-LMDE ~ $ sudo mdadm -a /dev/md0 /dev/sdb1
 mdadm: added /dev/sdb1

Mdadm will then resync the two drives by copying everything from /dev/sda1 to /dev/sdb1. This process takes three to four hours for my 250 GB drives and will be longer for larger drives. You can monitor the rebuilding process using the following ‘cat /proc/mdstat’. I like to run this command with ‘watch’ every few seconds to see the rebuilding progress in real time.

ecellingsworth@MD1-LMDE ~ $ watch -n 5 'cat /proc/mdstat'
Every 5.0s: cat /proc/mdstat                            Fri May 31 11:51:17 2013
Personalities : [raid1]
 md0 : active raid1 sdb1[2] sda1[3]
 244194841 blocks super 1.2 [2/1] [_U]
 [>....................]  recovery =  2.6% (6465024/244194841) finish=124.7
 min speed=31746K/sec
unused devices: <none>

Don’t trust the estimated time. The rebuilding process will run in the background, but if you are using the PC at the time, the rebuilding has to pause every time you need access to the file system. And, for some reason, rebuilding tends to slow way down near the end.

Edit 12/17/14:

I’ve used the above process to add back to the array a failing drive several times now. However, in my case, since I’m adding back the same drive that was previously in the array, it’s not necessary to copy the the partition table from the good drive nor is it necessary to zero the superblock on the failed drive. Instead, I use one command “mdadm -a” to add the drive back to the array and it rebuilds fine. This has the advantage that it’s much less risky. There’s no chance of destroying the good drive by wiping it’s partition table or superblock. And, I doubt anything would happen if you told mdadm to add a drive that’s already in the array, though I’m not going to test this theory myself.

Update the DNS record for a Namecheap hosted domain automatically

I thought I’d share this technique I use to update the dns records for hellonull.com automatically from the web server every 5 minutes. In the event that my ISP issues me a new IP address, my site will hopefully only be down for 5 minutes.

First, you must obtain the dynamic dns (ddns) update password for your domain from Namecheap. At the time of this post, this was done by clicking the ‘Dynamic DNS’ link from the left hand menu, once your domain was already selected. When you enable dynamic DNS, a password will be generated that looks like a long string of random letters and numbers. Copy this down.

Screenshot - 05182013 - 02:59:52 PM

Next, put this password into the updateddns.sh script (below) and edit the wget commands to point to your hosts (@, www, etc.) and domain (hellonull.com). Make this script executable. It’s a good idea to make the owner of this script root and save it in a protected location, like /root because it contains ‘plain-text’ passwords. Anyone who is able to access the script is able to read the password and is therefore able to change your site’s DNS records.

#!/bin/sh
# This script updates the dynamic DNS record for several domains
# hosted on this server. It is run automatically by cron. The crontab
# file that makes this happen is stored in /etc/cron.d/ddns.
# While this script can execute as any user, it should be owned by
# root because it contains 'plain text' passwords.
# Each host (as in host.hellonull.com) must get updated individually.
# In addition, all subdomain records must be updated individually.

logFile="/var/log/updateNamecheapDDNS.log"

# this is the password as generated by namecheap
pass="You'll have to look at the original or get new ones"

# add a wget line for each host and subdomain
wget -a $logFile -O /dev/null "https://dynamicdns.park-your-domain.com/update?host=www&domain=hellonull.com&password=$pass"
wget -a $logFile -O /dev/null "https://dynamicdns.park-your-domain.com/update?host=@&domain=hellonull.com&password=$pass"
wget -a $logFile -O /dev/null "https://dynamicdns.park-your-domain.com/update?host=teachingwiki&domain=hellonull.com&password=$pass"

If you run the updateddns.sh script it should update your DNS records. To automate the process simply schedule the script using cron. I added a crontab file called ddns in /etc/cron.d which executes the script every 5 minutes.

# use this file to schedule ddns updates
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# every 5 minutes
*/5 * * * * root /root/updateddns.sh >/dev/null 2>&1

GtkGrid alignment issues

Recently I ran into a problem laying out some controls with a GtkGrid widget that took quite a while to figure out. My controls were contained within a vertical GtkBox with three rows. The first row contained a grid with two columns. The left column housed labels and the right housed the actual controls. The next two rows of the vertical box were similar except the grids within were contained within a frame. The main vertical box itself is in the left column of a 2 column horizontal box. The right column contained a GtkScrolledWindow with a GtkTextView. The scrolled window was set to expand to take up all of the remaining horizontal space in the window. Problem was, the controls on the left weren’t aligned properly. A picture should help the explanation.

Screenshot - 05172013 - 11:48:09 AM

In the image, the vertical box (red) has been enclosed within a GtkFrame (frame13) to show its borders. As you can see, the controls in the yellow grids are aligned to the left and do not fill the cells of the grid. I would like them to be either right aligned or to fill the cells of their parent grids so that the right edge of all controls are aligned. This turned out to be nearly impossible. Most solutions I tried would cause the scrolled window in the adjacent column to shrink for apparently no reason to about 1/3 of the total horizontal area. The image below is an example of one trial solution in which I set the horizontal expand property of the GtkLabel “Loop” to True. The label expands to push the neighboring control (a GtkComboText) to the right side of the vertical box extents. But for some reason, the space outside of the vertical box is allocated to the label’s parent grid instead of the scrolled window.

Screenshot - 05172013 - 11:43:50 AM

The behavior is the result of the GtkGrid inheriting the hexpand property its child, the “Loop” label, and so on up the chain. The vertical box doesn’t appear expanded, but it is allocated the space anyway. To override this reverse inheritance behavior, one must set the hexpand property of the parent container (the vertical box in this case) to False. I’m designing in Glade and by default, this property is “No” for all widgets. This is not the same as setting the property to False. The workaround is to add a line of code to force the property to be false. I have unfortunately not found a fix that can be applied within Glade. The line looks something like this:

    builder = Gtk.Builder()
    builder.get_object('verticalBox').set_hexpand(False)

From there, setting the horizontal expand on the labels pushes the controls to the right, but no space is allocated outside of the vertical box parent. The successful alignment, with frame13 removed and some column spacing added between the controls and the scrolled window looks like this:

Screenshot - 05172013 - 12:11:45 PM

This project is being developed in Python 2.7+ using the Gtk+3 framework. The UI is designed in Glade 3.

Saving the image drawn with Cairo on a GtkLayout or GtkDrawingArea using a pixbuf

In my previous post, I discussed how to get the draw() event to fire for Gtk3 widgets in order to draw on them using cairo, specifically the GtkLayout and GtkDrawingArea widgets. Since that time I’ve had some success writing code to plot data from a connected source meter (through the RS232 port) in real time. I got stuck again trying to save the image drawn on the GtkLayout to a file. Most of the material I found on the internet wasn’t up to date and not relevant for the GTK3 framework. Finally I figured out how to save the image and it’s pretty easy to do too.

The solution is to get a GdkPixbuf from the widget’s GdkWindow and use the GdkPixbuf’s .savev() method to write the image to disk. For some reason, when I use the .get_window() method of GtkWidget’s, I don’t get a GdkWindow returned but rather a GdkX11Window object (gtk.gdk.X11Window). I’m not sure why there’s a difference but it seems to be ok to feed this into the Gdk.pixbuf_get_from_window() function. Please note that you must use the .savev() method of the pixbuf. Most sources online use the .save() method but that seems to be gone in Gtk3 (or perhaps there is no python binding?).

For example, assume you have a GtkLayout stored in the variable layout (I find mine with layout = self.builder.get_object(‘mylayout’) – because I’m using Glade). To capture an image of the layout and whatever you’ve drawn on it, you would do the following:

pixbuf = Gdk.pixbuf_get_from_window(layout.get_window(), 0, 0, width, height)
pixbuf.savev("foobar.png","png", [], [])

In this example, width and height need to be the width and height of the widget. The rectangular area defined by (0,0) to (width, height) is saved. In my case, I did something like this:

rect = layout.get_allocation()
width = rect.width
height = rect.height

In the .savev() method, the first argument is the file name, the second is the type, and the third and forth options are arrays of options and their settings. I pass empty arrays because I don’t set any options.

Note that this works with GtkLayout and should work with GtkDrawingArea and GtkWindow. For other widgets, the window returned by .get_window() seems to be that of the parent GtkWindow. If you want to capture an image of those wigdets, you’ll need to specify the x,y position of their top left corners in the Gdk.pixbuf_get_from_window() call.

layout