The Fugue Counterpoint by Hans Fugal

13Aug/130

tmux C-Left in Terminal.app

I use Terminal.app on OS X, and I want to bind ctrl-left/right to cycle through tmux windows (like I did with screen).

The tmux incantation is easy to find online:

bind-key -n C-Left prev
bind-key -n C-Right next

This doesn't work though, because Terminal.app is sending ^[[5D for left instead of what tmux expects. In my case, with TERM=xterm-256color, tmux is expecting ^[[1;5D for C-Left and ^[[1;5C for C-Right. You can change this in the Terminal.app settings. Ideally Terminal.app would magically send the right values based on the TERM setting, if there is such a thing as the right values in the world of terminfo and modified arrow keys.

I would prefer to tell tmux to accept ^[[5D instead of ^[[1;5D, which is what I did in my screen config, but I can't see any way to tell tmux to take a raw escape sequence instead of logical keys. I prefer that so that I don't have to remember (or research) magical incantations to configure Terminal.app the next time I start from scratch on OS X. So if you know how, let me know.

4Jan/110

Thunderbird new message badge

Even as I am trying to get back on top of my email backlog, I find a more effective way to ignore emails and just get work done. The cycle of life, eh?

Those little red badges are annoyingly effective at getting me to drop what I'm doing and check out what's new. When the number is in the hundreds, I can't be bothered to remember what the last count was and so I don't know whether there's new mail, so I'm too frequently opening it to see what's new and whether it needs immediate action. In Thunderbird 2 on OS X, you could configure the new message badge to only show messages new since the last time you looked, not unread messages. Then, if there is a little red circle you know there is new mail that needs attention, and if not there is nothing new. Much more productive.

Initially in Thunderbird 3, this feature was removed by haughty but well-meaning developers who couldn't fathom why anyone would want to see anything but the number of unread messages. There was a beta user hissy fit (I participated), the usual stubborn standoff, a patch, and a whole bunch of inaction. I gave up and went on with my life. Apparently, sometime in the interim reason prevailed and it's back in Thunderbird 3.1. The preference is mail.biff.use_new_count_in_mac_dock.

28Dec/101

Asynchronous Kindle Transfer

I got a Kindle for Christmas, and it's pretty cool. Project Gutenberg never looked so nice. Getting ebooks onto it is pretty simple, just drag and drop. But that only works when the Kindle is plugged in, and I know I will think to download a book usually when the Kindle is not plugged in. Then I have to dig through my downloads folder (which makes a teenager's room look tidy) and find the ebook(s) and drag onto the Kindle. This needs to be automated.

It's actually pretty simple. First, make a "staging" folder. I put mine at ~/Desktop/Kindle. Then, fire up Automator and create a new "Folder Action". Choose /Volumes: whenever anything is added to /Volumes (e.g. when mounting a drive like the Kindle) it will execute. Add the action "Run a Shell Script". Here is my script:

src=$HOME/Desktop/Kindle
dst=/Volumes/Kindle

cd "$src"
while read item; do
  if [ "$item" = "$dst" ]; then
    rsync -av --remove-source-files "$src"/ "$dst"/ | logger -t Kindle
  fi
done

The --remove-source-files option to rsync makes this essentially a recursive move operation. So your staging folder will be empty once it has successfully synced—if you want to keep an off-Kindle backup of your ebooks you won't keep them in the staging folder. (You could omit --remove-source-files, but then if you deleted a book from your kindle it would just be put back the next time you plugged it in which would be annoying.)

The staging folder has the same directory structure as the Kindle, so put the documents you want to copy over into ~/Desktop/Kindle/documents/.

6Aug/100

OSX ignores ownership on external drives by default.

I had reason to copy my harddrive off and back (to reformat as case-sensitive), and I missed one very important detail.

http://www.egg-tech.com/mac_backup/

IMPORTANT STEP: disable "Ignore ownership on this volume".

Yeah, with everything owned by user 99, it won't even boot. You can boot into single-user mode and "chown -R root:wheel /" which will at least let it boot, then you can go into Disk Utility and repair permissions (which will actually do something useful and vital in this situation). Then chown -R your home directory. But it's still a mess to clean up, and you lose all that user and group info.

Tagged as: , , , , No Comments
11Jun/100

Lilypond 2.12 in the OSX Terminal

Getting this error?

$ /Applications/LilyPond.app/Contents/Resources/bin/lilypond
dyld: Library not loaded: @executable_path/../lib//libstdc++.6.dylib
  Referenced from: /Applications/LilyPond.app/Contents/Resources/bin/./lilypond
  Reason: image not found
Trace/BPT trap

I don't know why this is screwed up, but I have a workaround:

ln -s /usr/lib/libstdc++.6.dylib /Applications/LilyPond.app/Contents/Resources/lib

While I'm at it, here's my script in ~/bin:

$ cat ~/bin/lilypond
#! /bin/bash
exec /Applications/LilyPond.app/Contents/Resources/bin/$(basename "$0") "$@"

You can symlink it to midi2ly or whatever other executables you want to execute. Or you could add that directory to your path, of course.

15Jan/090

VPN DNS

I have a VPN and a DNS server that serves up forward and reverse DNS for my VPN hosts, which zone I call wan. When I want to look at my Cacti graphs, I go to gwythaint.wan and as long as my laptop is on the VPN I can see them wherever I am. In theory anyway. In practice, getting this to work without screwing up other things is harder.

I'll leave out the myriad permutations that I tried over the past couple of weeks and show you the one that actually works well. That is to have a caching and forwarding name server on your laptop, and to add localhost to your list of nameservers. For best results, you would have it forwarding to the name server your DHCP server gives you, with an explicit forward over the VPN for the wan zone (and its reverse). resolvconf on Linux can do this. Your situation may warrant a static forwarder for non-wan addresses, in which case you just set that forwarder and be done with it. If your various DHCP nameservers are a bit more subtle—perhaps serving up internal domains of their own—then you may have to not forward and/or recurse except explicitly for wan.

I just took the default BIND9 configuration on my system and tweaked it thus:

// local/vpn stuff
zone "wan" {
    type forward;
    forwarders { 172.17.77.1; 172.17.0.1; };
};
zone "17.172.in-addr.arpa" {
    type forward;
    forwarders { 172.17.77.1; 172.17.0.1; };
};

On most systems the default named.conf is already some reasonable caching setup, so you wouldn't have to tweak it beyond that. Then I added localhost to the nameserver list (/etc/resolv.conf on Linux, in the network preferences pane on OS X) and checked that it works with a dig @localhost gwythaint.wan.

Things got tricky because dig and host on my laptop were taking forever to
return when I queried localhost—6 seconds or so. I chased this wild goose for
awhile and in the end I didn't find the reason (it still does it), but I
verified that it's not a problem. If you use the -v flag to host you notice
that the actual queries took <1ms, so whatever else host and dig are doing may
not be relevant. Even stranger, if you do host -v gwythaint.wan and don't
specify to query localhost, everything resolves instantly and yet it reports
that it queried localhost (which you can verify with the non-traffic on repeat
requests via tcpdump). It hasn't slowed down any other applications (a 6-second
slowdown on DNS lookups would be very obvious), so I chalk it up to "who
cares?" If host and dig on OS X return the right answer, and you verify they're
querying the right server, then you're good to go.

6Jan/096

Putting OpenVPN in its place

Update: I had some errors and oversights in my general config that didn't have any direct bearing on the main message of this post. I have fixed them below and I beg you to pretend they never happened.

OpenVPN is a fantastic piece of software. No, it's an essential piece of software. A godsend.

But it has this tendency to try to be all that and a bag of chips.

My primary gripe with OpenVPN over the years has been what I call "psuedo-DHCP". It pretends, poorly, to be a DHCP server. If you have the audacity to prefer a real DHCP server you find very little help and sometimes even resistance from the tools and the community. I once tried to get it working and failed.

This week I was refreshing my OpenVPN setup and reading through the manpage for version 2.1, and saw a few references to people actually using DHCP. Still no explicit documentation, but it gave me hope. So I duly tilted at that windmill.
Now I will show you how to get DHCP working with OpenVPN. What's more, we'll get rid of ifconfig and route options (for the most part). In short, we'll put OpenVPN in its place: as a secure tunnel manager.

The important paradigm shift here is that you aren't required to do anything from withing OpenVPN to configure the interface. You can just bring up the tunnel and your TUN/TAP device will be alive but unconfigured. At that point you could do something like this:

ip link set tap0 up
ip addr add 172.17.0.1/24 dev tap0

You could do this manually, or in an up script, or whatever. Or you could let your distro do it. Ah, so we can have a tap0 stanza in /etc/network/interfaces (Debian-based distros) that will configure tap0 when we ask it to. Let's look at a client example:

# in /etc/network/interfaces
iface tap0 inet dhcp
    hostname falcon
    # dhclient doesn't pay attention to this, so if you use dhclient (you
    # probably do) see /etc/dhcp3/dhclient.conf
    client falcon

# in the openvpn config
dev tap0
route-delay 10
cd /etc/openvpn
up "up.sh"
down-pre
down "down.sh"
…

# up.sh
#! /bin/bash
ifdown tap0 2>/dev/null
ifup tap0 &

# down.sh
ifdown tap0

There's some subtlety here, let's talk about it. Note that we're specifying both the DHCP client id and the DHCP hostname—more on that later. We use an external script because of the way OpenVPN's up option works, so that we can background the ifup call. This is important because the tunnel isn't fully up at this point, so your DHCP client won't succeed unless we background it (I tried up-delay to no avail). I have the ifdown bit in there as a safety measure—if for whatever reason Debian thinks the interface is already up it won't start the DHCP client and that would be bad. But hopefully this doesn't happen much thanks to the down option. Finally, the route-delay option gives the DHCP negotiation a chance to finish before any routes are applied (and in my setup there is one important route that I push to clients).

On the server side, we need to set up the DHCP server. ISC DHCP (dhcp3-server on Debian) isn't very intelligent about interfaces that materialize out of nowhere, so we'll need to set up a persistent TAP device.

# in /etc/network/interfaces
auto tap0
iface tap0 inet static
    address 172.17.0.1
    netmask 255.255.255.0
    pre-up openvpn --dev tap0 --mktun

# in openvpn config
dev tap0

Now tap0 will be brought up automatically at boot, and will stay up even if you restart OpenVPN (you can bring it up now with ifup tap0). Notice that no ifconfig option is needed in the OpenVPN config. Now you can configure your DHCP server for the subnet:

# in dhcpd.conf
subnet 172.17.0.0 netmask 255.255.255.0 {
    # example options for VPN hosts
    option domain-name "vpn.example.com";
    option domain-name-servers 172.17.0.1;
    option netbios-name-servers 172.17.0.1;
    option ntp-servers 172.17.0.1;

    range 172.16.0.100 172.17.0.199;
}

host falcon {
    option dhcp-client-identifier "falcon";
    fixed-address 172.17.0.77;
}

Observe the dhcp-client-identifier option, and its matching entry in foo's /etc/network/interfaces (or /etc/dhcp3/dhclient.conf). This is important because TAP MAC addresses don't persist—you get a new one every time. dhcpd will use the client identifier to match a host, but alternatively you could spoof a static MAC address in foo's /etc/network/interfaces config. I think the client identifier is cleaner. Even if you don't use static leases, this way dhcpd will know it's the same client and give him the IP address he had before. Of course if you don't need (semi-)static leases you don't need to worry about client identifiers. You'll have some cruft leases but they should expire and disappear.

Unfortunately dhcpd doesn't use the client identifier for dynamic dns updates (one of the big reasons I wanted to use real DHCP in the first place), which is why I specify the hostname option in foo's /etc/network/interfaces. dhclient (as configured on Debian) sends the hostname whether or not you specify it in /etc/network/interfaces.

Other DHCP clients that do honor /etc/network/interfaces are available. See interfaces(5). I'm kind of partial to udhcpc, especially for hand-testing, though I usually end up sticking with dhclient.

Caveats: I haven't been able to get DHCP working with an OS X client. I tried initiating DHCP on the TAP interface with ipconfig set tap0 DHCP but it didn't work and once locked up my machine. So for this situation, or for any other reason you may have, you can still push ifconfig and route options in the client configuration directory entry for that client.

I haven't tried DHCP over OpenVPN on Windows clients yet but I see no reason why it wouldn't work.

Finally, I tried briefly to do it with a TUN device and though I can think of no obvious reason why it shouldn't work, it didn't. I like TAP better anyway.

Now after all this I can see some of you shaking your heads wondering what the point of all this is. "Surely this is more complicated than ifconfig and route in OpenVPN." Yes, it's more complicated, but it's more powerful. If all you need is pseudo-DHCP, then by all means use pseudo-DHCP. But if you are a sysadmin serving a gaggle of clients you soon find yourself pining for a real DHCP server. Or perhaps you want dynamic dns updates, or proper DHCP option support. (You do realize DHCP options sent by OpenVPN's dhcp-option are not applied on linux unless you do so manually by reading the environment variables in an up script, don't you?)

When you realize OpenVPN can just set up the tunnel and get out of the way, you realize that all your fancy networking knowledge and tools can come into play to create the ultimate VPN tailored exactly to your needs. Plus, I think it snaps things into focus so that things just make more sense in your head.

And now, I present my OpenVPN configs (sanitized) for the server (frodo) and a client (falcon):

## frodo (server)
dev tap0
mode server
tls-server

cd /home/fugalh/vpn
ca cacert.pem
dh dh.pem
cert frodo.pem
key frodo.pem

keepalive 10 60
comp-lzo
client-to-client
# this new option is nifty
passtos

client-config-dir ccd

# See /etc/network/interfaces for interface configuration and routing.
# (reproduced here for our web audience)
# auto tap0
# iface tap0 inet static
#         address 172.17.0.1
#         netmask 255.255.0.0
#         pre-up openvpn --dev tap0 --mktun
#         up ip route add 172.17.64.0/24 via 172.17.0.64
#         up ip route add 172.17.77.0/24 via 172.17.0.77
#         up ip route add 172.17.82.0/24 via 172.17.0.82
#         up ip route add 172.17.83.0/24 via 172.17.0.83
push "route 172.17.0.0 255.255.0.0 172.17.0.1"

#verb 3
mute 2
status /var/log/openvpn.status 60


## falcon (client)
dev tap0
client
remote frodo.fugal.net
nobind

cd /etc/openvpn
ca falcon-cacert.pem
cert falcon-cert.pem
key falcon-key.pem
tls-remote frodo.fugal.net

comp-lzo
passtos

route-delay 10
cd /etc/openvpn
up "up.sh"
# (reproduced here)
# #!/bin/bash
# ifdown tap0 &>/dev/null
# ifup tap0 &

down "down.sh"
# (reproduced here)
# #!/bin/bash
# ifdown tap0

mute 2
#verb 3

In my setup the 172.17.0.0/24 subnet is for the OpenVPN server and clients, and each client is a gateway to a 172.17.x.0/24 subnet for his LAN. Assuming a static route on the LAN for 172.17.0.0/16 via the OpenVPN client, frodo will route everything so people on one LAN can find people on another.

I also have dynamic dns updates for both forward and reverse DNS in my vpn.fugal.net zone.

One thing I haven't set up which is feasible is for the LAN DHCP servers to do ddns to frodo.

OpenVPN is in its place, and our relationship is that much stronger. Good luck with yours!

20Mar/080

What was that macro?

I find myself asking this question a lot: "Now what was that define that detects annoying platform that makes porting otherwise perfectly-portable code difficult?"

This will narrow your search down to something useful:

echo | cpp -dM

Go ahead, try it. I wish I'd known this one earlier.

12Feb/082

OS X Terminal Emulation Woes

OS X's Termina.app is the terminal I've been using since I switched to Leopard, because it has tabs now and it's beautiful. Oh, and iTerm gave me too much grief with odd, illogical and unpredictable bugs.

One of the drawbacks to Terminal.app is that it's broken. This is what Aptitude looks like with TERM=xterm (I think this is the default):

TERM=xterm

This is what it should look like:

TERM=dtterm

How to get from there to here? The short answer is to choose dtterm as your terminal emulation (in Preferences, on the Advanced tab).

The long answer is that the problem here is that xterm supports this capability called back-color-erase (bce). If you tell programs that you are an xterm (with TERM=xterm), they will assume you support bce. The same goes for rxvt and xterm-color and even vt100 (even though that one doesn't seem to support color). bce isn't the only problem, either. There's also redraw problems that are difficult to show with a screenshot.

Setting TERM=dtterm seems to get rid of at least the major breakage. It would seem that the actual capabilities of Terminal.app are closest to dtterm, or at least closer to dtterm than to xterm or rxvt. It solves all the issues I've been having with aptitude, mutt, and screen locally and on remote linux boxes. But there's a caveat—not all remote systems will have the dtterm entry in their terminfo databases. Ubuntu 7.10 didn't by default, for example. The package you want on Debian-based systems (like Ubuntu) is ncurses-term.

Alternatively, you can install it in your home directory. To do this, on OS X type

infocmp > /tmp/dtterm
scp /tmp/dtterm username@example.com:/tmp
ssh example.com tic /tmp/dtterm

tic (terminfo compiler) will create a terminfo database entry in ~/.terminfo/d/dtterm, and you should be good to go.

18Feb/072

gvim on OS X

In spite of being the author of one of the more (in)famous vim color schemes,
desert.vim, I have been
vimming primarily in the console for quite some time now. The primary reason is
that gvim on OS X sucks.

Still, it's nice to have gvim working for various reasons. One of the more
important reasons is dragging stuff to the Vim icon on the dock.

As a follow-up to my post on unicode in the terminal, I decided to install the latest Vim.app from macvim.org and see how it handled unicode. It didn't handle it very well, but it's only a few settings away.

First, there's the issue of font. I don't like Monaco very much. I use
Anonymous for the terminal
and I think it is an excellent font. In fact, I find myself likeing Anonymous
just as much as Terminus, but it
has the added advantage of being TrueType and looking great with antialiasing.
Unfortunately, in Vim at least it looks terrible without antialiasing. So I
got the pseudo-TrueType Terminus
font
by Eric Cheng. It took
some fiddling, because of various Bad Things™ happening with Vim and OS X
dealing with a TrueType-wrapped bitmap font, but I found the following settings
work great (put them in ~/.gvimrc):

set enc=utf-8
set macatsui noanti gfn=Terminus:h12

It's not antialiased, but it is Terminus and it handles Unicode just fine (as
far as the Terminus font itself does, which is pretty good).

You can use antialiased fonts, e.g. Anonymous, with the following .gvimrc:

set enc=utf-8
set nomacatsui anti termencoding=macroman gfn=Anonymous:h10

but Bad Things™ happen when you use Unicode, so I'm sticking with the Terminus setup.

Tagged as: , , , , 2 Comments