The Fugue

I was a beta tester for VMware Fusion. Fusion is a quality product, as I've come to expect from VMware. Unfortunately, the beta is over and a dialog box popped up when I tried to run it the other day. "The long wait is over!" Yeah, I've been waiting with bated breath for my beta to stop working until I fork over $40. You read my mind!

$60 ($40 after the current promotion) is the most affordable virtualization option for OS X right now, and in my opinion the most technically superior to boot. But I'm still a cheapskate, so I decided to use VMware Server on Linux over NX instead. NX is indistinguishable from magic. It works great, with one problem: the keyboard mapping in the virtual machine is completely skewampus.

In googling for the answer, I found a few people who had trouble with a key here or a key there. No, I had a completely unusable keyboard. e was the backspace key, backspace was a comma, escape was a letter, and everything else in between was equally unreasonable.

So I tried it over ssh (btw, it works a lot faster if you use ssh -Y instead of ssh -X), and it worked fine. So the gauntlet was down.

After some stabbing in the dark, reading, more stabbing in the dark, more reading, finally understanding, and one last stab in the dark, I got it to work. The theory is in this article. In short, VMware tries to map key codes to emulated PC scan codes (v-codes). If it can't do that, it maps keysym codes to v-codes. The former is apparently foolproof but isn't always an option. The problem here is that VMware thinks it will work, but it won't (probably because Apple's X11 isn't XFree86). So we simply need to put this in ~/.vmware/config:

xkeymap.nokeycodeMap = true

I love Ruby, and I use it whenever I can for all kinds of tasks. When I was working as a professional system administrator, I put it to good use a few times there too. But using ruby always left me feeling a little guilty, because whoever came after me would probably curse my name for using a "nonstandard" language. It didn't help that Ruby was rarely installed, if even available, by the popular server distributions of the day. But all that has changed in the past two years, and using Ruby is often acceptable even in the conservative world of system administration.

You can imagine my interest then, in a book titled "Practical Ruby for System Administration" by André Ben Hamou. The title is telling; Ruby is known for being recommended by The Pragmatic Programmers because, well, it's a pragmatic language. System administration is all about pragmatism, above all else. Doing things right is important, but doing them now is more important. I knew all along that Ruby was an excellent tool for the enlightened sysadmin; now there's a book to back me up.

The book starts out with why you would want to use Ruby, and nips the counterarguments in the bud. Hamou does a good job on both counts, though as I already know Ruby I can't say whether the language introduction is sufficient of its own accord to get started with actually writing Ruby, or whether you'll need to refer to other sources (which are available online).

In chapter 2 he discusses one-liners. One-liners are the staple of many sysadmins, but I never fell for them. I'd rather write a throwaway script than try to cram it all on one line, or if it can really be done well on one line it can probably be done even more concisely with shell script and UNIX tools. However, the chapter does discuss a lot of fascinating switches that the ruby executable takes. I can see myself using many of these, and I was ignorant of most of them. You might learn the same from a careful study of the man page, but Hamou presents it in an easier-to-digest format. It's not always easy to inject humor into a book while maintaining concise brevity, but Hamou does a decent job of that. I found myself laughing aloud more than once, and yet I feel that the book is concise enough to serve as a reference, and that the jokes won't get annoying after a few readings of a section.

This book is not only a great book for the sysadmin hoping to use Ruby, but also an excellent book for any sysadmin who may not even be interested in Ruby. Although not even pretending to be a book on system administration best practices, there are many gems in here that will leave you saying "Why didn't I think of that?" and "I really need to implement that. It will be so helpful and it's astoundingly simple to do with Ruby!"

Chapter 3 gives you not only a quantitative feel for the speed of Ruby versus other performance-oriented languages (i.e. C), but it teaches you when to be concerned with execution speed, when to be concerned with implementation speed, and makes you think twice about your own feelings about performance. "Ruby is slow" is one of the favorite counterarguments against Ruby, but it's rarely a good one. This is even more true in system administration, where the sysadmin's time is much more important than whether the script runs in 1/10 second or 3 seconds.

Chapter 4 discusses "metaprogramming". I found myself at odds with Hamou most in this chapter, but it was all academic differences in terminology. He's fairly sloppy with the term metaprogramming and other terms (e.g. "macros") in this chapter, but the content is nonetheless a useful and vital part of making the most out of Ruby. I even learned a thing or two. He discusses domain-specific languages (DSLs) in Ruby, but mostly at the level of recognizing one when you see one. I think the world needs a paper or book on rapidly making your own DSLs, something sysadmins could really leverage if it were truly easy to do. I've written a DSL in Ruby, and while easier than I could possibly have imagined it's still not at the level of "easy and completely generic" that I feel DSLs can one day reach.

Chapter 5 is where the fun really begins. With the basics of the language under our belts we can really look at specific examples. This is where the book really shines, both as a tool for applying Ruby and as a minefield of good sysadmin practice. We learn to read and write files, with examples for the most common ones. I don't mean we discuss IO.open and IO.close in detail, I mean we talk about the concepts that apply across all kinds of file reading/parsing and generation, including issues of locking.

In chapter 6 we explore the storage and retreival of data, in a variety of approaches including inspect, marshalling, YAML, and ActiveRecord. Most interesting to me was the section on memcached. I think he ought to also have introduced SQLite (perhaps in the context of ActiveRecord) and DRb.

Chapter 7 is dedicated to dealing with "enterprise data". You know that of which we speak. XML, CSVs, protocols like XML-RPC, SOAP. Most valuable to me in this chapter was a coherent discussion of what REST is (finally!) and how to do it in Ruby.

Chapter 8 discusses network, including writing simple clients and servers. One tendency Hamou has in this book is to use pure Ruby and steer clear of system and backticks. This tendency sticks out in this chapter, where much time is spent discussing that which could be acheived better with a shell script, or at least a Ruby script with judicious use of system or backticks. The general argument for doing things in pure Ruby is portability, and to a lesser extent performance. Neither of these is the first concern of a sysadmin, who is generally not going to write a script that must work on more than one platform (even if it runs on several different Linux/UNIX distributions, which have the same GNU tools).

Chapter 9 deals with that task which we all hate: network and log monitoring. He has some gems of wisdom herein, but all in all we're left feeling about the same as when we went in: we can do it (now we can do it with Ruby) but it's a pain. The gains are not as great here as they are in other areas. This isn't Ruby's fault, nor Hamou's, it's just that nobody's really come up with a good and general way to attack this problem yet.

Chapter 10 discusses RubyGems. I think it's fitting that such a chapter be at the end of the book, and that's all I have to say about that today.

Chapter 11 discusses testing. You should do it in some form and he discusses a few forms. Nothing earth-shattering here, from where I sit. Chapter 12 talks about the future of Ruby, and will probably cause you to salivate.

All in all, an excellent book on using Ruby as a sysadmin. Go ahead, come out of the closet. Ruby is not only OK, it's often the best choice.

Imagine my suprise when a PDF created by Quartz (from the application Pages), turned out to be corrupt. It worked ok with Quartz apps, but not with more strict PDF apps like acrobat and various special-purpose PDF tools. ps2pdf give a nice summary of what's wrong:

$ ps2pdf mv.pdf 
   **** Warning:  File has an invalid xref entry:  28.  Rebuilding xref table.

   **** This file had errors that were repaired or ignored.
   **** The file was produced by: 
   **** >>>> Mac OS X 10.4.9 Quartz PDFContext <<<<
   **** Please notify the author of the software that produced this
   **** file that it does not conform to Adobe's published PDF
   **** specification.

I found a neat program called pdftk - the PDF toolkit. It is able to repair such PDFs. From its manpage:

Repair a PDF's corrupted XREF table and stream lengths, if possible:
pdftk broken.pdf output fixed.pdf

Et voilá! Worked like a charm. pdftk has a bunch of other neat tricks up its sleeve, like concatenating PDFs, etc., and it's Free software and cross-platform. Worth a look.

In spite of being the author of one of the more (in)famous vim color schemes, desert.vim, I have been vimming primarily in the console for quite some time now. The primary reason is that gvim on OS X sucks.

Still, it's nice to have gvim working for various reasons. One of the more important reasons is dragging stuff to the Vim icon on the dock.

As a follow-up to my post on unicode in the terminal, I decided to install the latest Vim.app from macvim.org and see how it handled unicode. It didn't handle it very well, but it's only a few settings away.

First, there's the issue of font. I don't like Monaco very much. I use Anonymous for the terminal and I think it is an excellent font. In fact, I find myself likeing Anonymous just as much as Terminus, but it has the added advantage of being TrueType and looking great with antialiasing. Unfortunately, in Vim at least it looks terrible without antialiasing. So I got the pseudo-TrueType Terminus font by Eric Cheng. It took some fiddling, because of various Bad Things™ happening with Vim and OS X dealing with a TrueType-wrapped bitmap font, but I found the following settings work great (put them in ~/.gvimrc):

set enc=utf-8
set macatsui noanti gfn=Terminus:h12

It's not antialiased, but it is Terminus and it handles Unicode just fine (as far as the Terminus font itself does, which is pretty good).

You can use antialiased fonts, e.g. Anonymous, with the following .gvimrc:

set enc=utf-8
set nomacatsui anti termencoding=macroman gfn=Anonymous:h10

but Bad Things™ happen when you use Unicode, so I'm sticking with the Terminus setup.

This is the 21st Century. It's time for Unicode. OS X does a decent job of supporting Unicode overall, but it is far from perfect. Perhaps the most noticeable is the mass confusion that is Unicode in the terminal. The problem is actually legion, and it takes a little effort to fix it. But you can fix it and you can do it today.

Start at this article on Internationalizing the Mac OS X terminal. Following the instructions there will get you 90% of the way there. You will install a new version of bash and coreutils (for ls, cd, etc.), you'll set up your locale information (actually you only need to set LANG, in my experience. I export LANG=en_US.UTF-8 in my .bashrc).

If you use OS X's default vim (/usr/bin/vim), it will Just Work. But if you have vim 7.0 from some other source, it may or may not work. In particular if you do sudo port install vim you'll probably get a broken vim. Be sure to use the multibyte variant if you install from ports.

If you use iTerm, it might work worse than before you made the changes above. But notice that it works fine if you run another bash instance, or screen. I'm not sure but I think you fix it with the LC_CTYPE hack detailed in this article. Those .inputrc lines probably won't hurt either, though I'm not sure what problem they fix. One of those two changes seems to have fixed the iTerm problem.

Speaking of screen, you probably want to add defutf8 on to your ~/.screenrc.

I'm curious if things work out of the box in $YOURFAVORITEDISTRO. I've previously mucked about with unicode so much in Debian and Ubuntu I have no recollection of what the default state of affairs is. Try the following and let me know in the comments how it worked.

echo æøÜÉ > /tmp/ƒøø
ls /tmp/ƒøø
cat /tmp/ƒ<tab>
vim /tmp/ƒøø
screen vim /tmp/ƒøø

For me, I see exactly that and exactly what you'd expect on the terminal and in vim (both in and out of screen), which has never all happened in OS X for me before. It does work over ssh on my Debian box, but as I say I've played with things there so it may or may not have worked out of the box.

I was less than fair in my previous assessment of the MacBook's ability to run FlightGear. It is capable of running it at passable speeds if you disable some things. I went through the output of fgfs -h -v and disabled everything that could be disabled (almost) and I get about 20fps when I'm not looking at the KSFO terminal or something equally busy. Even then it's 14 or 15. That's flyable. Using the 2D cockpit (or no cockpit and just the HUD) will buy you a couple of frames per second. But so will using a less-complex 3D cockpit, e.g. the Piper Cherokee Warrior II or the Northrop T-38. Flying during the day will buy you a lot (something about airport lights). One nice thing is that you get basically the same frame rate with the window maximized (or full-screen) as at 640x480. It's not pixel-bound, in other words. Most of the checkboxes in View|Rendering Options make no noticeable difference, except shadows, and I don't know about the lighting options since any nighttime flying with airport lights is too slow to bother.

So here's the ~/.fgfsrc that gets me 23fps average in the air away from the airport in the Piper Cherokee Warrior II.

--disable-random-objects
#--enable-ai-models
--fog-fastest
#--disable-enhanced-lighting
#--disable-distance-attenuation
#--disable-specular-highlight
#--disable-anti-alias-hud
#--disable-hud-3d
#--disable-clouds3d
#--enable-fullscreen
--shading-flat
#--disable-skyblend
#--disable-textures
--aircraft=pa28-161

One of these days I'll see which of the three remaining I can trim also, but it looks good enough for now.

My GAANN fellowship provides me with a laptop, and that laptop has finally arrived. It's a MacBook 13" and it's very nice. Unfortunately the University claims it as its own because it was bought through them and was over $1k, so I will have to give it back when my GAANN stint is over. Hopefully that means when I graduate, but if our department doesn't get the GAANN award that it's applying for that means GAANN is kaput next August. So I'm keeping the iBook in cold storage and we'll cross that bridge when we come to it.

In the process of setting up my essential applications I discovered a few had been updated without telling me, and I took the chance to investigate some alternatives to the programs I wasn't fully satisfied with. iTerm was updated, and actually has a useful release now. It looks a little nicer, seems to behave better, although I've come across some glitches when it's set to always show tabs. I skipped the space-consuming menu bar system monitoring stuff that I had on the iBook and gave iPulse another look. Last time I was turned off by the clown colors and that it was hard to see what was going on. This time I was looking for succinctness and was willing to invest some time in learning to read the display, and I tried out a few of its jackets (I settled on Doppler), and I'm quite pleased. It also has a menubar display that's much more relevant and compact than what I was using, as well as an animated dock icon. And it doubles as an analog clock down there in the corner. I'm very impressed.

I am disappointed, however, in the video card. Flightgear still runs at a crawling 7fps sitting on the runway at KSFO with the default settings. I might be able to get it running at a suitable speed, but based on this experience this Intel GMA 950 card is nothing like the NVidia Quadro4 in my desktop. Flightgear on the iBook worked fairly well in Linux, so I'll try that out when I get Linux installed and perhaps I will be pleasantly surprised. This wasn't to be my primary Flightgear station anyway so it's not the end of the world.

I'll report on installing Ubuntu 6.10 when I get around to it in the next few days.

PortAudio is a portable cross-platform API. But it seems like none of the developers has access to a mac because the docs for getting started with PortAudio on OS X are a sorry joke.

I took the liberty of figuring it all out in XCode and creating a PortAudio.framework which is a piece of cake to use. Download it, move the resulting PortAudio.framework folder to /Library/Frameworks, include as #include <PortAudio/portaudio.h>, and link with -framework PortAudio. If you use XCode, include the same way and just add the framework to your project and it takes care of the linking.

If you want the XCode project that I used, contact me. I don't have any qualms about sharing it, I'm just too lazy. I didn't change any code, I just made an XCode project to build the framework. Incidentally, all the documentation and READMEs and license files are still in the framework, which is just a folder which you can browse in the finder.

Enjoy!

XCode has a feature called Zero Link, which apparently more or less skips the linking step when you build your project. I don't know the details but it's probably fair to say that it's like just-in-time compiling, but for linking. The raison d'être of this feature is to reduce the time spent waiting for things to compile during development. It's not to be used for released code, so it is on by default in the development building style and off by default for the release style.

That sounds really neat, but we just wasted a bunch more time than we could possibly have saved with Zero Link, because there was no linker to complain about this:

// foo.h
#ifndef FOO_H
#define FOO_H

short foo[2];
short *p;

#endif

The problem, if you don't already see it, is that those variables should be declared extern and instantiated in one and only one .c file. ld would complain about duplicate symbols when trying to link two .o files that had both included this header file. But with Zero Link, there's no error. Instead, you get crazy behavior from the debugger: assignments that seem to have no effect and other odd things that make you think you've fallen into an alternate universe.

So be careful.

QCad is a really nifty open-source 2D CAD program for POSIX systems. They don't distribute free binaries, though, so you have to build it yourself.

There's two catches when building. First is trying to figure out where to start. It looks to me like this is the starting point:

cd scripts
./build_qcad.sh notrans

The second catch is if you get an error about ISO C++ and 'long long'. This actually applies to Qt in general, but I know that my Qt build environment is fine so I was a little confused. Turns out that the -pedantic compiler flag is what brings this error to the surface. So this patch will fix your QCad build:

--- qcad-2.0.5.0-1-community.src/mkspecs/defs.pro.orig  2006-03-31 09:12:28.000000000 -0700
+++ qcad-2.0.5.0-1-community.src/mkspecs/defs.pro       2006-03-31 09:07:55.000000000 -0700
@@ -1,6 +1,6 @@
# $Id: defs.pro 606 2004-12-25 03:08:40Z andrew $
-QMAKE_CXXFLAGS_DEBUG += -pedantic
-QMAKE_CXXFLAGS += -pedantic
+#QMAKE_CXXFLAGS_DEBUG += -pedantic
+#QMAKE_CXXFLAGS += -pedantic

win32 {
QMAKE_CFLAGS_THREAD -= -mthreads

What do I need QCad for? I'm designing two Really Neat™ DIY projects: a keyboard stand and a compact against-the-wall under-the-keyboard music stand. If that piqued your interest, stay tuned. If it didn't, well I'm sorry for you.