The Fugue

I've occasionally needed to pipeline some processes in Ruby, especially when doing sysadmin-type tasks. You can always do:

system "foo | bar"

but that gets interpreted by the shell. That can be a problem when you need to include filenames that could have all kinds of nastiness including spaces, semicolons, exclamation points, etc. You should be shaking in your boots about now.

Well, for single processes, it's easy:

system "foo", arg1, arg2

But if you want to pipe them together it's a bit more difficult. I came up with this solution, and it seems to be working well in my "convert my cool music collection into a lame (wink) format my car stereo can understand" script:

  def pipeline(*args)
    pids, r = args.inject([[],nil]) do |memo, cmd|
      pids, rr = memo
      r,w = IO.pipe
      if (pid = fork)
        pids.push pid
        rr.close if rr
        w.close
        [pids, r]
      else
        r.close
        $stdin.reopen rr if rr
        $stdout.reopen w
        cmd = [cmd] if String === cmd
        exec *cmd
      end
    end
    r.close
    pids.each {|p| Process.waitpid(p)}
  end

  pipeline ['oggdec','-o','-',oldpath],['lame','-',newpath]

Do take the time to figure out what's going on, but don't do it without a piece of paper handy or you might sprain something.

Update

I had an error in the original post—if you don't close rr in the parent of the fork you will end up with too many open files eventually.

I love Ruby, and I use it whenever I can for all kinds of tasks. When I was working as a professional system administrator, I put it to good use a few times there too. But using ruby always left me feeling a little guilty, because whoever came after me would probably curse my name for using a "nonstandard" language. It didn't help that Ruby was rarely installed, if even available, by the popular server distributions of the day. But all that has changed in the past two years, and using Ruby is often acceptable even in the conservative world of system administration.

You can imagine my interest then, in a book titled "Practical Ruby for System Administration" by André Ben Hamou. The title is telling; Ruby is known for being recommended by The Pragmatic Programmers because, well, it's a pragmatic language. System administration is all about pragmatism, above all else. Doing things right is important, but doing them now is more important. I knew all along that Ruby was an excellent tool for the enlightened sysadmin; now there's a book to back me up.

The book starts out with why you would want to use Ruby, and nips the counterarguments in the bud. Hamou does a good job on both counts, though as I already know Ruby I can't say whether the language introduction is sufficient of its own accord to get started with actually writing Ruby, or whether you'll need to refer to other sources (which are available online).

In chapter 2 he discusses one-liners. One-liners are the staple of many sysadmins, but I never fell for them. I'd rather write a throwaway script than try to cram it all on one line, or if it can really be done well on one line it can probably be done even more concisely with shell script and UNIX tools. However, the chapter does discuss a lot of fascinating switches that the ruby executable takes. I can see myself using many of these, and I was ignorant of most of them. You might learn the same from a careful study of the man page, but Hamou presents it in an easier-to-digest format. It's not always easy to inject humor into a book while maintaining concise brevity, but Hamou does a decent job of that. I found myself laughing aloud more than once, and yet I feel that the book is concise enough to serve as a reference, and that the jokes won't get annoying after a few readings of a section.

This book is not only a great book for the sysadmin hoping to use Ruby, but also an excellent book for any sysadmin who may not even be interested in Ruby. Although not even pretending to be a book on system administration best practices, there are many gems in here that will leave you saying "Why didn't I think of that?" and "I really need to implement that. It will be so helpful and it's astoundingly simple to do with Ruby!"

Chapter 3 gives you not only a quantitative feel for the speed of Ruby versus other performance-oriented languages (i.e. C), but it teaches you when to be concerned with execution speed, when to be concerned with implementation speed, and makes you think twice about your own feelings about performance. "Ruby is slow" is one of the favorite counterarguments against Ruby, but it's rarely a good one. This is even more true in system administration, where the sysadmin's time is much more important than whether the script runs in 1/10 second or 3 seconds.

Chapter 4 discusses "metaprogramming". I found myself at odds with Hamou most in this chapter, but it was all academic differences in terminology. He's fairly sloppy with the term metaprogramming and other terms (e.g. "macros") in this chapter, but the content is nonetheless a useful and vital part of making the most out of Ruby. I even learned a thing or two. He discusses domain-specific languages (DSLs) in Ruby, but mostly at the level of recognizing one when you see one. I think the world needs a paper or book on rapidly making your own DSLs, something sysadmins could really leverage if it were truly easy to do. I've written a DSL in Ruby, and while easier than I could possibly have imagined it's still not at the level of "easy and completely generic" that I feel DSLs can one day reach.

Chapter 5 is where the fun really begins. With the basics of the language under our belts we can really look at specific examples. This is where the book really shines, both as a tool for applying Ruby and as a minefield of good sysadmin practice. We learn to read and write files, with examples for the most common ones. I don't mean we discuss IO.open and IO.close in detail, I mean we talk about the concepts that apply across all kinds of file reading/parsing and generation, including issues of locking.

In chapter 6 we explore the storage and retreival of data, in a variety of approaches including inspect, marshalling, YAML, and ActiveRecord. Most interesting to me was the section on memcached. I think he ought to also have introduced SQLite (perhaps in the context of ActiveRecord) and DRb.

Chapter 7 is dedicated to dealing with "enterprise data". You know that of which we speak. XML, CSVs, protocols like XML-RPC, SOAP. Most valuable to me in this chapter was a coherent discussion of what REST is (finally!) and how to do it in Ruby.

Chapter 8 discusses network, including writing simple clients and servers. One tendency Hamou has in this book is to use pure Ruby and steer clear of system and backticks. This tendency sticks out in this chapter, where much time is spent discussing that which could be acheived better with a shell script, or at least a Ruby script with judicious use of system or backticks. The general argument for doing things in pure Ruby is portability, and to a lesser extent performance. Neither of these is the first concern of a sysadmin, who is generally not going to write a script that must work on more than one platform (even if it runs on several different Linux/UNIX distributions, which have the same GNU tools).

Chapter 9 deals with that task which we all hate: network and log monitoring. He has some gems of wisdom herein, but all in all we're left feeling about the same as when we went in: we can do it (now we can do it with Ruby) but it's a pain. The gains are not as great here as they are in other areas. This isn't Ruby's fault, nor Hamou's, it's just that nobody's really come up with a good and general way to attack this problem yet.

Chapter 10 discusses RubyGems. I think it's fitting that such a chapter be at the end of the book, and that's all I have to say about that today.

Chapter 11 discusses testing. You should do it in some form and he discusses a few forms. Nothing earth-shattering here, from where I sit. Chapter 12 talks about the future of Ruby, and will probably cause you to salivate.

All in all, an excellent book on using Ruby as a sysadmin. Go ahead, come out of the closet. Ruby is not only OK, it's often the best choice.

lircr (pronounced 'lurker') is a LIRC client library for Ruby. It's simple, it's easy, it's fun. If you got LIRC, get lircr. I am not responsible if you grow more hair on your index finger or thumb.

require 'lirc'
lirc = LIRC::Client.new
event = lirc.next
if event.name == "play"
    system "xmms", "/av/music/3.mod"
end

And now, ladies and gentlemen, I will answer the questions you have been dying to have answered! Right before your very eyes I will demystify The Texas Instruments TMS320C6416DSK RTDX OLE API!

    require 'win32ole'
    rtdx = WIN32OLE.new('RTDX')
    rtdx.ole_methods.each do |m|
      print "#{m.return_type}\t#{m}("
      print m.params.collect{|p| 
        "#{p.ole_type} #{p}" +
        if p.default
          "=#{p.default}"
        else
          ""
         end
       }.join(", ")
      puts ")"
    end

VOID  QueryInterface(GUID riid, VOID,VOID ppvObj)
UI4  AddRef()
UI4  Release()
VOID  GetTypeInfoCount(UINT pctinfo)
VOID  GetTypeInfo(UINT itinfo, UI4 lcid, VOID,VOID pptinfo)
VOID  GetIDsOfNames(GUID riid, I1,I1 rgszNames, UINT cNames, UI4 lcid, I4 rgdispid)
VOID  Invoke(I4 dispidMember, GUID riid, UI4 lcid, UI2 wFlags, DISPPARAMS pdispparams, VARIANT pvarResult, EXCEPINFO pexcepinfo, UINT puArgErr)
I4  Open(BSTR Channel_String, BSTR Read_Write)
I4  Close()
I4  Read(VARIANT pArr, I4 dataType, I4 numBytes)
I4  ReadI1(UI1 pData)
I4  ReadI2(I2 pData)
I4  ReadI4(I4 pData)
I4  ReadF4(R4 pData)
I4  ReadF8(R8 pData)
I4  ReadSAI1(VARIANT pArr)
I4  ReadSAI2(VARIANT pArr)
I4  ReadSAI4(VARIANT pArr)
I4  ReadSAF4(VARIANT pArr)
I4  ReadSAF8(VARIANT pArr)
VARIANT  ReadSAI2V(I4 pStatus)
VARIANT  ReadSAI4V(I4 pStatus)
I4  WriteI1(UI1 Data, I4 numBytes)
I4  WriteI2(I2 Data, I4 numBytes)
I4  WriteI4(I4 Data, I4 numBytes)
I4  WriteF4(R4 Data, I4 numBytes)
I4  WriteF8(R8 Data, I4 numBytes)
I4  Write(VARIANT Arr, I4 numBytes)
I4  Rewind()
I4  Flush()
I4  Seek(I4 MsgNum)
I4  SeekData(I4 numBytes)
I4  StatusOfWrite(I4 numBytes)
I4  GetNumMsgs(I4 pNum)
I4  GetChannelID(BSTR Channel_String, I4 chanId)
I4  GotoNextMsg()
I4  GetMsgID(I4 pMsgId)
I4  GetMsgNumber(I4 pMsgNum)
I4  GetMsgLength(I4 pLength)
I4  EnableRtdx()
I4  DisableRtdx()
I4  EnableChannel(BSTR ChannelName)
I4  DisableChannel(BSTR ChannelName)
I4  GetChannelStatus(BSTR ChannelName, I4 pChannelStatus)
I4  ConfigureRtdx(I2 Mode, I4 MainBufferSize, I4 NumOfMainBuffers)
I4  ConfigureLogFile(BSTR FileName, I4 FileSize, I2 FileFullMode, I2 FileOpenMode)
I4  GetRTDXRev(I4 RevNum)
I4  GetStatusString(BSTR StatusString)
I4  GetCapability(I4 Capability)
I4  RunDiagnostics(I2 TestType, I4 TestMode, I4 TestInfo)
BSTR  GetDiagFilePath(I2 TestType)
I4  SetProcessor(BSTR Board, BSTR Cpu)
HRESULT  GetTypeInfoCount(UINT pctinfo)
HRESULT  GetTypeInfo(UINT itinfo, UI4 lcid, VOID,VOID pptinfo)
HRESULT  GetIDsOfNames(GUID riid, I1,I1 rgszNames, UINT cNames, UI4 lcid, I4 rgdispid)
HRESULT  Invoke(I4 dispidMember, GUID riid, UI4 lcid, UI2 wFlags, DISPPARAMS pdispparams, VARIANT pvarResult, EXCEPINFO pexcepinfo, UINT puArgErr)

I don't know whether to be jumping for joy or throwing things across the room for living without this knowledge for so long.

~/.irbrc is a really nifty file. I should take the time to explore its possibilities sometime. But where does one find out the irb tricks like that one, and this one? If you know please enlighten us!

Here's my current ~/.irbrc:

IRB.conf[:AUTO_INDENT]=true

# ri
def ri arg 
  puts `ri #{arg}` 
end 

class Module 
  def ri(meth=nil) 
    if meth 
      if instance_methods(false).include? meth.to_s 
    puts `ri #{self}##{meth}` 
      end 
    else 
      puts `ri #{self}` 
    end 
  end 
end

# vim: filetype=ruby