The Fugue

I can't believe I never thought of this before. It's the ultimate in couch potatoism.

We don't use the phone much. When it rings, the house fills with moaning and groaning because nobody wants to answer it, but we often don't dare ignore it because we don't know who it is. As I usually have my laptop with me, or at least in sight, I realized (with the idea coming from a client who wants me to do some Jabber integration) that the perfect solution is to have Asterisk send me a notification via Jabber with the caller ID.

It's very simple. I use the sendxmpp program to do the heavy lifting. Here's the Batphone script:

#!/usr/bin/ruby

## Configuration
# sendxmpp config file has to be mode 600 and owned by the asterisk user
CONFIG="~fugalh/telephony/.sendxmpprc" 
RESOURCE="Asterisk"
RECIPIENTS="hans@fugal.net erin@fugal.net"

require 'agi'
agi = AGI.new
IO.popen("sendxmpp -r #{RESOURCE} -f #{CONFIG} #{RECIPIENTS}", "w") {|f|
  f.puts "Incoming call from #{agi.env['callerid']}"
}

As much as I love Batphone, it's almost as easy to do with a shell script:

#!/bin/sh

# sendxmpp config file needs to be mode 600 and owned by the asterisk user
CONFIG=~fugalh/telephony/.sendxmpprc 
RESOURCE=Asterisk
RECIPIENTS="hans@fugal.net erin@fugal.net"

# this snippet from http://yakko.cs.wmich.edu/~drclaw/asterisk/cidname/
declare -a array
while read -e ARG && [ "$ARG" ] ; do
    array=(` echo $ARG | sed -e 's/://'`)
    export ${array[0]}=${array[1]}
done

echo "Incoming call from $agi_callerid" | \
    sendxmpp -r $RESOURCE -f $CONFIG $RECIPIENTS

If you are lucky enough to get agi_calleridname set as well, you obviously might like to integrate that information into your message too. Here's the Asterisk dialplan snippet:

exten => s,1,AGI(/home/fugalh/telephony/jabber.agi)
exten => s,n,Dial(SIP/sipura&SIP/hans,30)
; ...

Thanks to serendipitous timing on tensai's part, I learned today that OpenSER version 1.2.0 was released just yesterday. It looks exciting; a few of the most fundamental problems with SER have finally been fixed. Number one on this list is script variables. Now you can do:

$ruri = "sip:" + $from.user + ":" + $to.domain;

Second on the list is that you can now "extend OpenSER functionality by writing Perl applications to be executed embedded in configuration files." Other wonderful things:

• Pseudo variables can be used directly in the script
• Pseudo variable transformations
• DNS NAPTR/SRV failover
• An XMLRPC management interface
• XMPP (Jabber) Gateway
• SNMP
• LCR database caching

All in all, it looks like an exciting release. If (Open)SER disappointed you in the past, now might be the time to look again at OpenSER.

I have extolled the virtues of (Open)SER+MediaProxy in the past, but I can no longer recommend it. Instead I recommend using Asterisk as a sort of pseudo media proxy. Why? Read on.

The problem is that SIP and NAT don't get along. The reason they don't get along (besides that NAT is evil) is that the RTP packets (the media stream) may follow a completely different, and more direct, path from endpoint to endpoint than the SIP packets do. This is a good thing, except in the face of NAT. If there's a NAT involved, the RTP packets often can't get through. SER+MediaProxy solves this by tricking the endpoints into thinking mediaproxy is the other endpoint for RTP data. MediaProxy, which is not behind a NAT, is able to receive data from both parties and forward it on to each party (via the NAT holes already punched by them talking to the media proxy).

It's an elegant setup. It's a wonderful solution to an evil problem. But it turns out, quite surprisingly, that MediaProxy doesn't scale as well as other solutions. That is, there's a too-low ceiling on the number of simultaneous calls that MediaProxy can handle before the CPU maxes out.

Here's a setup that scales much better. Instead of doing the MediaProxy dance when NAT is detected, instead have SER proxy the packets through Asterisk. Have Asterisk set up to accept calls from SER (who has already authenticated the user), or to do the user authentication (in addition to SER?), and it will happily insert itself in the middle of the call as a back-to-back user agent. If you do this you will want to take care that you avoid gratuitous transcoding because that will certainly not scale as well as MediaProxy. But if you can keep from transcoding, this scales more than 2x better than MediaProxy.

This is a surprising result, because Asterisk is more complicated than MediaProxy, and the whole back-to-back user agent scheme is a lot more complicated than simply forwarding packets which is in essence all MediaProxy has to do, and that Asterisk has generally done a poor job of supporting SIP. MediaProxy is written in Python, and Asterisk in C. That might be part of it. Jared Smith tells me that there are various new optimizations for bridging calls in Asterisk, such as header caching. That might be another part of it.

Since setting up Asterisk this way is no harder and I think quite a bit easier than setting up MediaProxy (esp. in light of the "don't use_mediaproxy() (and friends) more than once in a route" bug), and Asterisk scales better in the real world, I have to recommend Asterisk as your media "proxy".

The story doesn't end here, though. It continues to evolve. It remains to be seen whether another implementation of a media proxy than MediaProxy or RTPProxy (which is still not as scalable as Asterisk in our tests) can do the job more efficiently still. I think that it should be possible. I might do it as a learning project in Erlang, but no promises. At this point I think it is doubtful whether a naïve implementation in any language, even C, would outrun Asterisk with its bridging optimizations. On the other hand, forwarding RTP and RTCP packets unchanged isn't rocket science and it just might be easy to outrun Asterisk.

Thanks to Elliot at Arrivaltel and Jared for helping me with testing, pontificating, and discussion, and to Brent Thomson for casting a shadow of doubt over MediaProxy (for other reasons) in my mind in the first place.

SER (and OpenSER) is a SIP router. That is, it receives SIP messages and decides where to send them or how to reply to them, in order to connect the two endpoints of a SIP call. It doesn't do media, it doesn't do voicemail or IVR or PBX or any of these other things. It's focus is routing, and it does it well and fast, and can handle a whole boatload of traffic.

But SER is not perfect. In fact, I would classify SER as barely adequate. In my opinion, a SIP router should be four things:

• Robust
• Powerful
• Scalable
• Easy to Administer

By robust I mean it doesn't crash, it's fault-tolerant, and it doesn't require 'reboots' (of the software, not the OS). I like to call this lack of reboot the "living" property, following Steve Yegge's post The Pinocchio Problem. SER is fairly robust, in that if you have a good dialplan and no flaky modules it doesn't crash often, and it's tolerant of rogue packets and other external problems. It is most definitely not living software though, and the constant restarting of the server during development and debugging is a serious problem.

By powerful I mean you can do whatever you need or want to do. You want to hook it up to your database for user authentication. You want to send yourself a Jabber message when Such and Such calls So and So. You want to do least-cost routing or some kind of convoluted load balancing that nobody has ever tried before. This is power. SER has this power, ultimately, through writing modules and using the exec module, it can be difficult (writing a C module), expensive (spawning a new process for every message!), or both. SER is severly limited by its scripting language which has no concept of variables (except storing stuff in an RDBMS with AVPs) and almost no string processing abilities.

By scalable I don't mean speed (though speed can't hurt), but throughput. One SER server can handle on the order of 6000 SIP messages per second. (Your mileage may vary, depending on what you're doing and how you do it.) This is where SER really shines. You might even say it's SER's raison d'être. It's also important to be able to distribute load across multiple servers. You can do this with SER also.

By easy to administer I mean that a normal human being should be able to configure the daemon, write the routing scripts, and debug the routing scripts. Given, the person doing the routing scripts will need to have a solid understanding of SIP. But with SER, you have to practically be a SIP guru in order to set up even a basic route script. The scripting syntax is simple enough, but as mentioned above the limitations make doing anything nontrivial a real acrobatic programming trick. Don't even get me started on SER route script debugging. I'll just say it involves ngrep and either very terse and unhelpful logs, or very verbose and unhelpful logs.

Ok, so we see how SER measures up to my ideal. What can be done? Can the SER people step up to the plate and bridge the gap? Anything is possible, but I'm not holding my breath. Why not? Because of technical and architectural reasons, let alone community reasons. Technical because SER is written in C, and C makes things hard, and when things are hard niceties like living software often don't happen. Architectural because SER is locked into their scripting language and the design decisions that go along with that. They'd have to break compatibility (or walk on eggshells) to fix that. Even if they surprise us in these areas, there's still the difficulty of writing modules and extensions in C.

No, we can't expect the SER people to make this leap for us. They might, and that would be nice, but let's not count on it. So where does that leave us? A SER heir could be written with these four aspects in mind. I think it could be done and done well if the right tools are chosen. Although I'm not 100% sure yet, I think the most important tool will be Erlang. Erlang almost solves the robust and scalable aspects from the very start. Erlang makes living software almost by default. Erlang was designed by and for the telephony industry. Erlang has good performance. The only part I'm not sure about yet (because I'm not familiar enough with Erlang), is whether it will be easy or hard to do routing scripts. Obviously, I think, you wouldn't ask sysadmins to write them in Erlang. It's a language with a different paradigm and that would be a serious barrier to entry. However, it would be nice to allow scripting to be done in Erlang for those with that skill. I don't know whether it will be hard to make a sensible DSL in Erlang or not. Based on my experience with ejabberd the daemon running/interaction might need some TLC, but I'm confident that can be accomplished with a few good shell scripts.

Will I write it? Not while you're looking. But I do want to learn Erlang and I do like to do real programs when learning a language. So don't be surprised if an infant heir to the throne of SER comes out of left field some day.

In the meantime, please enjoy Erlang: The Movie.

Does the world need yet another Ruby Asterisk Gateway Interface library? Does the world need a guy who dresses up like a bat?

I was not satisfied with the complexity and learning curve of existing AGI libraries for Ruby, so rather than spend an hour learning one of them I spent several hours writing my own. I hope this pays off in the future when it only takes me 5 minutes to remember/relearn how to use batphone. It was fun, anyway. That's the point, isn't it? That's why Batman does it. Not for the citizens of Gotham, but because it's fun.

Here's a synopsis:

#!/usr/bin/ruby
require 'agi'

agi = AGI.new
agi.answer
agi.stream_file('batman_help_the_monkeys_are_everywhere', nil)
begin
  # press pound to make them stop!
  r = agi.stream_file('tt-monkeys', '#')
end while r.result != ?#
agi.stream_file('POW', nil)
agi.hangup

Here's the URLs: For Documentation press 1. For Download press 2. For direct access to the batphone press 3.

POW!

X-Lite is the best currently available free SIP softphone for OS X. Version 2 does not work on the Intel Mac though; it crashes constantly. So, with a little digging through Google I was able to find the X-Lite version 3 beta for OS X, which does work.

This is good because I have a softphone. But it's bad because now I have less motivation to port Twinkle to OS X. It looks like a not-too-difficult port and I'd love to have all the power of Twinkle at my fingertips. Maybe I'll get to it over the next couple of months. Maybe you will. If you do, be sure to let me know!

Yours truly will be presenting tomorrow (June 6, 2006) on (Open)SER and MediaProxy: Bugspray for NATs at the Utah Asterisk Users Group.

SIP is big, SIP is here to stay. Although this is an Asterisk users group, It's important to know the strengths and weaknesses of Asterisk and its open-source SIP router friend, SER. Especially because one of the most successful ways to solve the myriad firewall and NAT-related problems of SIP is with the MediaProxy module.

You may not believe it, but with the help of MediaProxy SIP can be used easily in all but the most insanely firewalled environments. Come to the SLC library tomorrow at 7pm and learn how you too can conquer NAT. Directions are given in this previous meeting announcement

Getting started with Asterisk just got easier. Well, it didn't just get easier, because the SPA-3000 isn't a new product. It's just that I just now realized how perfect this device is for the fledgling VOIPer.

The traditional advice is to get an Asterisk Developer's Kit which gives you one FXS port and one FXO port, expandable. In the old days, this wasn't bad advice because the old devkit was reasonably priced. The new devkit is better, but pricier. At $240 it can hardly be considered entry level for any but the most serious (or rich) Asterisk newcomers.

The Sipura SPA-3000 is an amazing bundle of joy that also provides an FXS and FXO port. What I didn't realize (but should have guessed) before now was that it can not only bridge its FXS and FXO ports, but act as a gateway with external SIP devices, like Asterisk. So this thing can do everything the Asterisk devkit can do except add FXS or FXO ports, but without all the headache or cost. You can find these for less than a hundred, and you don't have to install it in your PC, worry about interrupts, or set up zap*.conf. You can still use Asterisk, of course, but you can also pair it up directly with SER or any SIP provider you like. Yes, you could do that with Asterisk too, but frankly Asterisk isn't a stellar SIP implementation and if you're serious about SIP you'll find on occasion that you might like to bypass it. (Whether or not you want to be serious about SIP is another post. ;-)

If you want to get started with VOIP in the comfort of your own home, all you need is a Linux box and an SPA-3000.

What about all the phones scattered throughout the house? Plug the phone company directly into the SPA FXO, and the SPA FXS into the wall. (Or just go straight VOIP, in which case you don't technically need a 3000 but might be better served with a 1000 or 2000 family member)

What about Digium?! How will the Asterisk developers feed their children and pets?! You traitor! You capitalist! You... You... Microsoft-lover!

I am full of gratitude for the Asterisk developers and for Digium, but I frankly believe Digium's products are overpriced and I have no qualms in taking my business elsewhere when it makes sense to do so. I have a Sipura SPA-1001 and I am very impressed with the quality of it, so I am confident in my recommendation. Go with the SPA.

If you want to prefix the area code for 7-digit numbers with Asterisk, it usually looks like something like this:

ext => _NXXXXXX,1,goto(505${EXTEN},1)

But if you need to server all area codes dynamically, you can do something like this:

ext => _NXXXXXX,1,goto(${CALLERID(number):0:3}${EXTEN},1)

This assumes caller id is set, and that your callers are using standard PSTN-like DIDs.

If you ever see strange sdp content like the following:

...
c=IN IP4 68.142.145.14568.142.145.145
...
m=audio 3501835018 RTP/AVP 0 8 3 98 97 101
...

That is, the IP address and port are repeated back to back (which really throws off the far end), and you are using openser and mediaproxy, then you are probably calling use_media_proxy twice in your ser routing. Apparently, this is a bad idea.