APIs for SIP applications

There are three types of SIP APIs: (1) source code API such as ones defined by JAIN SIP, libsip++ and pjsip, (2) high-level API to control the per-call behavior such as CPL, SIP-CGI, SIP Servlet, LESS, or (3) pseudo-code style API to control the behavior of the server or client such as SER or sipp config files. They all serve different purposes. The source code API is needed to create new software applications using existing libraries, the high-level API creates easy to define services, and the config file allows creating server or client behavior/scenarios.

With Python as the programming language, it is possible to create single software to expose these different types of APIs. This is because Python source code looks like human understandable pseudo-code if written with care. It makes sense to expose the APIs in my P2P-SIP 39 peers project to support such behavior. In this post, I present some of my initial thoughts on how to implement the generic API.

Firstly, the existing SIP module (rfc3261.py) already has an easy to use source code API. This is further enhanced in additional modules such as voip.py for user-agent specific functions. In particular, the source code API exposes object-oriented classes such as Transaction, Dialog, Stack, etc., to perform the functions defined in various layers in RFC 3261.

Secondly, the high-level API such as CPL and SIP-CGI can be implemented using per-user scripts in python itself (instead of XML for CPL, for example). The SIP server or client can import the per-user script based on the request-URI of the request and handle the processing. An example script to perform redirect to voice mail is shown below:

def redirect_to_voicemail(event):
event.location = URI('sip:jones@voicemail.example.com')
event.action.redirect()

def proxy_then_voicemail(event):
if event['From'].uri.host.endswith('example.com'):
event.location = URI('sip:jones@example.com')
e = event.action.proxy(timeout=10)
if e in ('busy', 'noanswer', 'failure'): redirect_to_voicemail(event)
else: transfer_to_voicemail(event)

basic.addEventListener('incoming', proxy_then_voicemail)

Because of the un-safe nature of Python, we will lose a number of features provided by CPL, but nevertheless the programmable API is easy to read and write, and nicely integrates with the existing source code. Similar extensions can be built for user agent applications similar to the LESS programming API.

Thirdly, instead of having the SIP client or server import the per-user script, the client or server itself can be written in the script. The following example shows how to translate some parts of SER config script to create a new server scenario.

from sipapi import *
import re

_debug = True # enable debug trace
open(('67.93.12.18', 5060)) # listen on this ip:port for incoming packets

def route(event):
# sanity check section
if event['Max-Forwards'] and int(event['Max-Forwards'].value) <= 0:
return event.action.reject(code=483, reason='Too many hops')
if len(str(event)) > 8192:
return event.action.reject(code=513, reason='Message overflow')
# this is used by sipsak to monitor the health of server
if event.method == 'OPTIONS':
if event['From'].uri.user == 'sipsak' and not event.uri.user:
return event.action.accept()
...
basic.addEventListener('incoming', route)

run() # the loop to process the SIP listening point

Similar scripts can be written to create client scenarios similar to how sipp creates scenarios from configuration files.

I feel a generic SIP API will make the job of Python programmers easier, instead of having to learn new APIs and implement them in custom SIP servers and clients. In any case, the article just poses some ideas, and I will be happy to mentor any student who would like to work on this project!

Why Open Source?

In this article I present my view on Open Source Software. There are a number of articles elsewhere that talk about advantages of Open Source and why it works [1][2][3]. (I use the term Open Source to mean Free Software, instead of diving into OSS vs FS debate.)

Point 1: Most software organizations are driven more by business objectives, and less by technology.

If you look around you notice several types of software companies: most of them are commercial ones like Microsoft, there are some like Sun that are commercial but pretend to do open source work, very few like Red Hat that build business out of open source software, and finally, there are some true open source software like Linux and Apache. I think most of Sun's Open Source attempts are for business reasons to provide competition to other existing businesses.

Point 2: Software programming is an art.

It is like learning a new language or doing sculpture or painting. Initially a programmer is too involved with the syntax and semantics of the language constructs and application. Once he is proficient, the programming comes naturally. For example, a good Java developer will think in terms of high level modules and classes, but once he starts implementing them things move smoothly instead of worrying about where to put the open bracket or when to create a new method. After a while, the software development process becomes an art of making beautiful, modular and efficient software. Some people have a built-in talent for the particular art, but most people learn by practice, practice and practice.

Point 3: Good art requires personal motivation.

Doing a scientific experiment is different than painting a new picture. At the broad-level, given the set of input data, and experimental setup, one is likely to achieve the same result in an experiment. On the other hand, an art piece requires inherent motivation of the artist and his personal inspiration to do something new, something different. Sometimes the inspiration gets driven by commercial interest, in which case an artist may end up producing art work without much motivation -- e.g., in the commercial Indian film industry, a music director has to produce tens or hundreds of scores per year, affecting the quality of the art as well as causing plagiarism from western music. On the other hand, motivated musicians who record one album a year do generate quality and innovative work.

Considering the above three points, a software engineer who is paid to work on a particular technology or piece of software is less likely to be personally motivated to create that piece of software. On the other hand, an open source developer who is not paid for his work initially, starts the open source work because of his personal motivation to create that piece of software. Thus, an open source software is more likely to be of better quality compared to a commercial software with the same amount of testing. Hence commercial software requires quality assurance to make it competitive with open source. Although quality assurance can reduce software bugs, it doesn't make the 'art' as good as the open-source version. If you compare the design of the Apache web server or SIP express router with their commercial counter parts, you can understand what I mean by 'art' in this context.

When I look at an open source software, I assume it reflects ideas and vision of the innovative and inspired developers who were motivated to write that piece of software. When I look at a commercial software, I assume it is created by software engineers who got paid to write code, to write documents, to test code, and more than that it is sold by salesmen who are paid to sell those pieces of software. I don't see much personal motivation or inspiration in the loop, and I don't aspect the design or the implementation of the software to be a good piece of art. (In small companies where all the people have same vision or inspiration about the software, it is possible to create quality work. However, things change over a period of time as the company grows and new software engineers are paid to perform work on other people's innovations.)

In the context of P2P-SIP, we haven't seen much of commercial interest beyond the few initial contenders. The main reason is that P2P-SIP inherently is peer-to-peer, and against the business model of services that the modern web/phone industry is so accustomed to. In other words, the industry has not figured out a way to make money out of open P2P-SIP. On the other hand, there are a number of developers who created open source prototype applications out of personal inspiration. The next steps for the open source P2P-SIP developer community: to build a bigger community, advertise and publish their work, prove that it works better than existing solutions, and use it on a daily basis!

Documenting RFC implementations

Just wanted to document how I am using the existing documentation such as IETF RFCs and Internet-Drafts in my 39 Peers project. You can looks at some samples at here or here.

I wrote a htmlify.py script. The script uses the SilverCity python package to decorate the base python code. To include the documentation from RFCs, the script interprets the python source code to identify lines such as

# @implements RFC2617 (HTTP auth)

If found, it downloads the particular RFC from IETF website, removes any formatting empty lines such as near page breaks, and numbers all the pages and lines. It then stores the resulting file as a text document which you can lookup and use in your documentation.

Furthermore, the script identifies lines such as

# @implements RFC2617 P3L16-P3L25

If found, the line is replaced by a HTML DIV block with content of the documentation in RFC2617 text file from page-3 line 16 to page-3 line 25.

I find this technique pretty handy, and keeps my decorated source code with inline documentation extracted from RFCs and drafts, instead of I having to write extensive document.

Problems due to NATs and firewalls

Network Address Translators (NAT) and firewalls create problems for end-to-end connectivity on the Internet. This not only affects P2P-SIP but also client-server SIP. In this article I post some example numbers to illustrate the point.

These numbers are for example only: suppose there are 10% public Internet nodes, 30% nodes behind good (cone or address restricted) NAT, 30% nodes behind bad (symmetric) NAT and 30% nodes behind UDP blocking firewalls (F). Let's denote these as P=10%, G=30%, B=30%, F=30%. Here the public Internet nodes are typically from universities and research institutes, those behind good NAT are usually from residential DSL/Cable access, those behind bad NAT are partly from residential and partly from enterprise environment, and those behind UDP blocking firewalls are from enterprise and corporate networks. Suppose a call event between any two pair of nodes is independent of each other for the probability analysis purpose and nodes are equally likely to call any other node. Thus, percentage of calls between two public Internet nodes is (10%)^2 = 0.01 = 1%.

Now let us enumerate the NAT and firewall traversal techniques available to SIP. STUN helps with good NAT, whereas TURN relay is needed for bad NAT. ICE is used to negotiate the connectivity using STUN or TURN bindings. A TCP-based relay (or even HTTP relay) is needed for UDP blocking and very restricted firewalls. (what about TCP hole punching and other techniques?) A STUN server is light in terms of bandwidth utilization, whereas a TURN relay needs high network bandwidth and hence costs the service provider more money. Same is the case with TCP-based relay.

In a call if one participant is behind a UDP blocking firewall (F), then the call must use a TCP relay. This amounts to 1-(1-F)^2 = 51% calls going through TCP relay.

In a call if both participants are behind bad NAT, then we need a TURN relay. This amounts to B^2 = 9% of the calls.

If one participant in a call is either on public Internet or good NAT and other is on public Internet, good NAT or bad NAT, then the media can go end-to-end using STUN bindings. This amounts to 40% of the calls.

In conclusion, the VoIP provider will need to host UDP or TCP relays for 51+9=60% of the calls. This is not a good proposition.

In real world, the call events are not independent of each other: probability of a corporate user calling another corporate user within the same corporation is high. Also probability of a home user calling another home user is also high. For example, a SIP service targeted towards consumers can expect to have most of the calls among residential users. Thus, the percentage of calls that can be end-to-end is much higher than 40%. Similarly, an enterprise VoIP system can expect to have mostly internal intra-enterprise calls, which do not need to cross the enterprise firewall. Hence the percentage of calls needing the relay is not as high as 60%. Let us analyze these two use cases separately.

Suppose, for a consumer SIP service, the distribution of nodes is P=15%, G=50%, B=30%, F=5%, i.e., less number of users are from bad NAT or UDP blocking firewalls. In this scenario about 20% calls need a relay whereas 80% calls don't.

In an enterprise VoIP system, suppose 60% calls are intra-office and 40% are with outside the office network, then only those 40% calls need a relay whereas 60% calls don't. In a properly engineered enterprise VoIP system, appropriate ports are opened for UDP as well as appropriate media relays are installed in DMZ which facilitates smooth media path for inter-office communication.

While we can play with these numbers as much as we want, the fact remains that a significant percentage of calls need media relay, either UDP TURN relays or TCP relays. This puts unnecessary burden on the VoIP service provider to install and manage relays and buy network bandwidth for those relays, or simply disallow calls that require relay (in which case they may lose customers).

In a peer-to-peer system with super nodes such as Skype, these super nodes can act as media relays and hence save a lot of bandwidth and maintenance cost for the provider. There are some things to consider though: a node behind public Internet can become UDP as well as TCP relay for any call, whereas a node behind good NAT can become only UDP relay with some workaround, but not a TCP relay. This puts too much burden on nodes behind public Internet.

Let us consider the original example with P=10%, G=30%, B=30%, F=30%. In this case the 51% of calls that require TCP relay must use one of the 10% P nodes. When acting as a relay, the bandwidth requirement at the relay is twice that of when the node is in a call. Suppose each node makes N calls a day, and generally speaking needs bandwidth for N calls. However, a public Internet node not only needs bandwidth for its own N calls, but also for relaying 5xN calls of other users which amounts to total bandwidth for 11xN calls. Thus, while the super-node architecture is beneficial to the provider, it heavily punishes users on the public Internet. (My guess is that number of public nodes using VoIP are about 4-5%, which further burdens the public nodes).

A managed P2P-SIP infrastructure can be a good alternative, where corporations and universities donate hosts/bandwidth on high speed network to act as relays/super-nodes. Alternatively, one can have an incentive system to promote hosts to become relays and super-nodes.

RTMFP vs SIP

Adobe's RTMFP is not P2P-VoIP as exemplified by Skype. On the other hand, RTMFP is closer to client-server SIP or H.323 where signaling happens via a server and media path can be end-to-end between the endpoints. When people refer to RTMFP as P2P, it is more like 'end-to-end media' similar to client-server SIP.

Why is RTMFP important? The previous Adobe protocol RTMP is strictly client-server even for media path. This gives poor quality for real-time media communication because media packets go from client to server, that too over TCP, and then are redistributed to the other client, again on TCP. End-to-end media based VoIP systems existed before Adobe implemented RTMP. I suppose the difficulty of NAT and firewall traversal and lack of interactive video communication requirement in Flash Player resulted in RTMP. Adobe corrected this mistake in the new protocol RTMFP which allows NAT and firewall traversal (to some extent) and allows end-to-end media path without going through the server. Although, the signaling is still going via the central server.

Once we understand this difference between P2P-VoIP and RTMFP, lets enumerate the differences between an RTMFP-based and a client-server SIP-based communication system.

1. RTMFP is a closed protocol, although Adobe recently opened up the previous RTMP. On the other hand, SIP is an open standard from IETF. This means anyone can implement SIP whereas only Adobe can implement RTMFP. That also means that a bug in the RTMFP protocol or its implementation is outside the scope of public review such as for security experts.

2. RTMFP is an integrated protocol that has support for signaling, encryption, media flow (flow control and congestion control), NAT traversal. Whereas SIP is just one piece of the puzzle, that is used in conjunction with RTP/RTCP, SDP, STUN, TURN, ICE, SRTP, etc. to build a complete system. In that regard there is more scope for interoperability problems in SIP systems. The SIP interoperability test (SIPit) events have helped in solving interoperability problems among current products for over a decade. (see next point on why RTMFP alone may not be sufficient?)

3. Based on the available documentation, RTMFP works on UDP. Whereas SIP can work on UDP as well as TCP. In an RTMFP application, the client should fall back to TCP-based RTMP if for some reason UDP is blocked for the client-server communication. This also means that the client will lose some of the benefits such as encryption available in RTMFP. There are other protocols RTMPS and RTMPE to facilitate security and encryption over TCP-based RTMP.

4. Although RTMFP works on UDP, it implements additional flow control and TCP-friendly congestion control. This helps media traffic deal with network congestion and slow receivers. On the other hand most existing SIP system do not implement such mechanisms in the media path. While this looks like an advantage in RTMFP, it turns out to be a problem because of the way it is implemented. In particular, the network components are disconnected from the media source components such as camera and microphone. The rate control mechanisms are implemented in network components which internally slow down the media traffic by delaying or dropping the UDP media packets. On the other hand the encode quality settings on camera and microphone components are unaffected. This results in packet drops due to congestion and hence choppy video or audio drop-outs. A good application built on top of RTMFP is supposed to get feedback from network components and adjust the encode quality parameters (framerate, bitrate, quality) in the camera and microphone components so that the packet drops are reduced. Thus, unless the application is smart enough to deal with this, the disconnected implementation of rate control and media source causes quality problems in RTMFP.

5. Both RTMFP and SIP can use media relays to workaround NATs and firewalls. However, RTMFP does not use a super-node architecture where some clients (Flash Player instances) act as relays, whereas (P2P) SIP can use existing client nodes to act as media relays. This means that when using RTMFP, the service provider must bear all the bandwidth cost of the relays, whereas in (P2P) SIP the cost can be distributed among the users because of the peer-to-peer nature. I analyze the cost due to NAT and firewall traversal in my next post.

Why does client-server video conference fail?

I analyze some problems in client-server communication for multi-party video conferencing.

Audio communication differs from video in two important ways: (1) usually in a conference only one person is speaking at any time whereas everyone's video is on, (2) audio codecs are usually fixed bit-rate whereas video codecs adjust bit-rate based on various parameters such as available network bandwidth and desired frame-rate.

Problem 1:
In a client server mode, because video coming from one participant needs to be distributed to all the other participants, the bandwidth and processing requirement at the server can be higher; unlike audio where usually only one person is speaking. Secondly, the downstream video bandwidth requirement at the client increases with the number of participants in a conference. In an N-party conference, each client will have usually one outbound audio stream, one inbound audio stream, one outbound video stream and N-1 inbound video streams. Note that this problem is worse for peer-to-peer (P2P) video conference, where everyone is sending video stream to everyone else: in which case there are N-1 inbound and N-1 outbound video streams at each client. For asymmetric network access (ADSL or Cable), where upstream bandwidth is lower than downstream, this causes early saturation in outbound network bandwidth. Shutting down video stream or reducing the video quality while a person is not speaking saves some bandwidth especially for speaker mode conferences.

Problem 2:
Second point of difference is that audio is usually encoded using fixed bit-rate codec whereas video bit-rate is adjusted based on several parameters such as available network bandwidth, desired quality and frame-rate. In a client-server environment most implementations use the client-to-server network quality information to decide what bit-rate to use for client's video encoding. Consider a two party client-server conference, where first client is closer to the server hence has lower latency. The first client decides to use high quality high bitrate video encoding. On the other hand the second client decides to use low quality low bitrate video encoding. This asymmetry causes the first client to receive poor quality video whereas the second client's downstream link gets congested with high bitrate video. The problem is further aggravated if in a multi-party conference there is only one participant on poor quality network. The problem is caused because we use client-server network latency metric instead of end-to-end network latency metric in deciding the video encoding bitrate.

Problem 3:
Sometimes, the conference server imposes bitrate control to limit the traffic towards a low bandwidth client. However, for efficiency reason the server doesn't re-encode the video packets. Instead, it just drops non-Intra frames if there is not enough bandwidth. This causes marginal to no improvement primarily because Intra frames are several times bigger than other frames. Secondly, it causes choppy video which further degrades the experience. The layered encoding in MPEG solves this problem.

Problem 4:
Larger video packets may not traverse end-to-end over UDP. An encoded audio packet is usually small, of the order of 10-80 bytes per 20 ms. On the other hand an intra-frame video packet size can be much larger, say 1000-10000 bytes. When media packets are sent over UDP, and the packet size is large, there is high probability of getting the packet dropped. This is because of the MTU restriction and middle-boxes (NAT and firewall) in the media path. An UDP packet of size larger than MTU (typically approx 1300-1400 bytes) gets fragmented at the IP layer such that subsequent fragments after the first one do not have the UDP header information (such as source and destination port numbers). A port inspecting NAT or firewall that doesn't handle fragmentation correctly may drop such subsequent fragments, causing loss of the whole UDP packet at the receiver end. Thus, video over UDP has to take care of additional fragmentation and reassembly, and/or discovery of path MTU in the application layer.

Problem 5:
The server may allow video over UDP as well as TCP from the clients, typically to support NAT and firewall traversal. If some clients are over TCP and others over UDP, then the server also needs to proxy packets from one to other. If the client over TCP assumes ordered packet delivery, then the server will also need to do buffering, packet re-ordering and delay adjustment, which further adds to the implementation complexity of the server. The problem is not that visible for audio beyond a glitch in sound, whereas for video the view may get completely corrupted until the next Intra frame.

Problem 6:
A slightly related problem is when the conference server does audio mixing but video forwarding. In this case, the server must perform delay adjustment, packet re-ordering, and buffering for the audio path. However, for efficiency reason it may blindly forward the video packets among the participants. Thus the synchronization information between the audio and video gets lost, and performing lip synchronization at the receiving client becomes a challenge. A correct implementation of the server should act as an RTP mixer, i.e., include the contributing source information in the mixed audio stream, and distribute RTCP information to all that participants for synchronization. (How to do this if each audio call leg is a separate RTP session?)

Some of these problems (2,3,5,6) can be solved to some extent by using peer-to-peer video conferencing.

Asynchronous Internet Programming

Programming an Internet application requires several asynchronous events such as network events, file I/O, timers and user interactions. In this article I walk through some existing design patterns for asynchronous programming.

Most programmers think synchronously, i.e., in a single control flow. For example, in a web server implementation, a socket is opened for listening, and when an incoming connection is received it accepts the connection, receives the request, and responds to the client. There are several steps in this flow that can block, e.g., waiting for incoming connection or incoming request. If a resource such as thread or process is blocking, it causes inefficiency in the overall system design. For example, if one thread is blocked serving a request from one client, another thread needs to be created to serve request from another client. Creating a thread-per-request causes too many thread creation and deletion in the system when the request-per-minute load increases.

Event handlers:
The first approach is to break your program into smaller chunks such that each chuck is non-blocking. Then use a event queue to schedule these chunks. The earlier windows programming (WinProc) falls under this category. The main loop just listens for the events on an event queue. When a user input is received such as mouse movement, click or keyboard input, a event is dispatched to the event queue. The event handlers installed in the program handle the events, and may post additional events in the queue. External events such as socket input can also be linked to this event queue. One problem with this design is that several event handlers need to share state (hence global variables) which results in complex software.

while not queue.empty():
msg = queue.remove(0)
dispatch(msg) # calls the event handler for msg

Some event-driven programming languages such as Flash ActionScript enforces this event handler design practice coupled with object-oriented design. Typically, in ActionScript, individual object has event queue and can dispatch events, instead of exposing a global event queue to the programmer. An object can listen for events from another object, e.g., a controller program can listen for click event on the button, and handle it appropriately. Thus, there can be some data hiding, information separation, and better software design using event handlers. In other languages (C++) libraries such as Poco that facilitate event driven object oriented programming. Nevertheless, one problem with this design is that logically similar source code needs to be split across different functions and methods, and sometimes different classes. For example, a timer event handler defined as a separate function from the timer initialization code.

button.addEventListener('click', clickHandler);
...
function clickHandler(event:Event):void {
...
}


Inline Closures:
Java as well as ActionScript solve this problem using inline closures. In particular, you can define event handlers inline within a method definition, thus keeping the logically similar code together in your program file. However, this is not always possible, e.g., if the same timer needs to be started from several places in the code. In this case, it does make sense to keep the timer handler as a separate method.

timer.addEventListener('timer', function(event:TimerEvent):void {
... // process timer event
}


Deferred Object:
Twisted Framework solves this problem using the Deferred Object pattern. The idea is as follows: instead of breaking the control flow source code every time a blocking operation occurs, the source code is broken when the result of the blocking operation is needed. This makes a lot of difference in programming, and makes the software cleaner. For example, earlier when a socket connection was initiated we needed to install an event handler for the success and failure result and perform the rest of the processing in those event handlers. Now, using the Deferred Object pattern, the socket.connect returns a deferred object, which is used in the same control flow as if it were a result of the connect method. The result object can be passed around elsewhere like a regular object. Only when we need to do something on completion of the connection, we wait on the result object'. This can be done either synchronously using result.wait() or asynchronously by installing a success and error callbacks. Provisions are there to allow one deferred object to wait on result of another deferred object before proceeding. An example follows:

result = socket.connect(...)
result.wait()
if result:
... # connection successful


Co-operative multitasking:
Python's multitask module allows cleaner source code by taking advantage of co-operative multitasking. The idea is to build application level threads of control which co-operate during blocking using the built-in yield method. A global scheduler runs in the main application. I find this to be the most clean design in my programming. An example follows.

data, addr = yield multitask.recvfrom(socket, ...)
yield multitask.sendto('response', addr)


One major problem with these approaches is that because of the inherent underlying single event queue software architecture, we cannot take advantage of multi-processor CPU architecture easily. Thus, we must use multi-threaded design in our software. Most of the earlier designs can be extended to multi-threaded application with some care. For example, one can run multiple global event loops listening on a single shared and locked event queue or one event-queue per thread where the dispatcher schedules it to the appropriate thread's queue. Clearly, the first one is more efficient from a queuing theory perspective. Care must be taken to lock the critical section in event handlers, or control flow that may get accessed from different threads at the same time.

Consequently, we get several multi-threaded designs: thread-per-request, thread-pool, two-stage thread-pool, etc. The performance comparison of these designs in the context of a SIP server is shown in my paper titled Failover, Load Sharing and Server Architecture in SIP Telephony, Sec 6.

Programming languages for implementing SIP

The programming language used for the implementation can affect the software architecture. For example, Flash ActionScript is a pure event-based language with no way of implementing a blocking operation. Hence, when a connection is made on a socket object, the socket object will dispatch the success or failure event. The caller installs the appropriate event handlers to continue the processing after the connection is completed.

C
There are two main reasons for implementing SIP in C: ability to compile on several platforms and very high performance. The primary advantage of implementing a SIP stack in C is that it can be easily ported and compiled on variety of platforms especially embedded platforms. Usually a C compiler is available for a platform, whereas others such as Java interpreter or C++ compiler may not be. Secondly, because there is no overhead (e.g., in terms of run-time environment and code size), the performance is usually the best. The main problem with implementing a SIP stack in C is the development time and cost of maintenance of the software. Finding bugs and adding a feature in a C program is usually more challenging than other languages. However if the software is well designed then the problem can be alleviated to a large extent. In any case, the number of lines of code that needs to be written in C is usually much more than the other high level languages such as Java or Python.

The pjsip project presents an implementation of SIP and other related protocols. It has been used in a variety of real-world projects and has proven itself to be a good SIP implementation especially where performance matters or the resources are limited.

C++
The object oriented design allows better reusability and maintainability compared to programming in C. However, the number of lines of code is still large. If advanced C++ features such as standard template library (STL) are used then portability may become a concern for certain embedded platforms.

One of my earlier SIP implementation was in C++ (and C) at Columbia University. Another example implementation is reSIProcate, an open source SIP stack.

Java
The Java programming language is very popular among corporate world and enterprise application developers. The standards community has developed APIs that cover several aspects of SIP implementation and some of its extensions. This allows the application to be built against those APIs whereas the actual implementation of the SIP stack can be provided by several vendors. The main problem with an implementation written in Java is that it tends to be too verbose. Java on one hand gives the illusion of a very high level language, but on the other hand requires the programmer to write a lot of code even to do a small thing. Part of this is because of the way the language is defined -- in particular the exception handling and strict compiler enforced type checking. This requires the programmer to do a lot of typecasting and hence there is potential for run-time errors. Another problem with Java is that the run-time could become a memory hog.

NIST SIP Stack provides the reference implementation of the JAIN SIP API.

Tcl
The Tcl programming language is not as popular as the other high level languages such as perl, PHP or Java. However, because of the simplicity in the language construct it is very easy to learn. Unless the software is designed right, it becomes very difficult to maintain a large piece of software.

Columbia’s SIP user agent, sipc, is written in Tcl.

ActionScript
ActionScript (or ECMAScript 4), improves on the Java programming language by allowing much smaller source code size of the implementation and much shorter syntax for common operations. However, there are two major limitations: the platform is limited to Flash Player or AIR (Adobe integrated run-time) and the language is purely event-based. The limitation of Flash Player may not seem important at the beginning, but prevents certain features. For example, the current version of Flash Player doesn’t have UDP or TLS sockets that could be used for SIP. Even the functions of a TCP socket is limited in that you can only initiate connections but cannot receive incoming connections. This prevents us from implementing a complete SIP stack in ActionScript without support from Flash Player or an additional plugin. The Flash Player version for embedded devices is usually older than the current version which makes portability an issue. Because of the run-time overhead the performance is limited. The media codecs supplied by the Flash Player 9 or earlier are proprietary codecs that are usually not supported by any other implementation. This causes interoperability issues as well.

Python
The object oriented nature, the compact coding style and very small source code size of the implementation makes Python a very good choice for implementing application protocols such as SIP. There is some overhead because it is an interpreted language; however the overhead is comparable to that of Java run-time. The interpreter is now usually available for embedded platforms as well, making it more portable than other languages such as ActionScript or C++.
The biggest advantage of an implementation in Python is that the code size is drastically smaller than other languages. I have implemented the basic SIP stack in less than 2000 lines of Python source code. Compare this with the Java implementation of SIP which has more than 1000 files. The lower size means that not only the development time is smaller, but also the testing, code review and maintenance cost is much lower.

P2P API

In this article I describe several sets of APIs for P2P. For a software engineer, designing a good API is very important. A good abstraction of the underlying concept leads to a good API. For example, the data model of storing key-value pair in a P2P network is usually abstracted as a hash table. Programmers like seeing existing semantics in an API because they are already familiar with those semantics. This article uses API ideas from OpenDHT, JXTA, Adobe Flash Player, and IETF P2P-SIP draft.

A good API
  1. should take the form of use-cases
  2. is very general and concise: does one thing and does it well
  3. is self-explanatory and is similar to existing concepts, models, practices
  4. is independent of implementation details
At the high level, there are three abstractions for P2P APIs: data storage, peer connectivity and group membership. There are several special cases among these abstractions.

Data Storage
A distributed hash table (DHT) is a form of structured P2P network with data stored using hash table abstraction. In particular, it provides put, get and remove API methods. Programmers are familiar with container semantics of hash-tables. In Python, this looks like:

a = DHT()
a['key1'] = 'value1'
print a['key1']
del a['key1']

Let us apply this to a use-case of P2P-SIP user location storage. In particular, the key is the user identifier of the form 'kundan@example.net' and value is the user location of the form 'kns10@192.1.2.3:5062'. With this we see several problems in the existing container semantics of the API listed above. Firstly, a user can have several locations, in which case a call request is sent to all those locations as in SIP forking proxy behavior. The following modification to the API causes more confusion because set takes a single value whereas get returns multiple.
a['kundan@example.net'] = 'kns10@192.1.2.3:5062'
print a['henning@example.net'] # prints all contacts of henning

To solve this we can assume a 'set' semantics for a[k]
a['key1'] += 'value1'
a['key2'] += 'value2'
print a['key1'] # print a list of values
a['key1'] -= 'value1' # remove specific key1-value1
del a['key1'] # remove all values for key1

Another problem is that this API is not secure or authenticated. A secure DHT API based on public-key infrastructure can sign the stored value on set and del.
a = DHT(privatekey=...)   # supply the private key of the owner
print a['key1'] # print all values for given key
print a(owner=...)['key1'] # print values only by given owner
A third problem is that the API is not extensible to pure event-based languages such as Flash ActionScript. In ActionScript, there are no blocking operations, hence set and get much be defined asynchronously. ActionScript already has SharedObject abstraction to deal with remote shared storage of objects. This can be reused for the API as follows:
so = DistributedSharedObject.getRemote(privatekey=...)
so.setProperty('key1', 'value1') # sets the key1-value1 pair
so.retrieve('key2') # initiates get
so.addEventListener('sync', syncHandler)
so.addEventListener('propertyChange', changeHandler)
function syncHandler(event:SyncEvent):void {
... # put is completed
}
function changeHandler(event:PropertyChangeEvent):void {
if (event.property == 'key2') # get is completed
trace(so.data['key2'])
}

Another problem is that the API doesn't take into account the time-to-live of key-value pair. This can be solved by supplying a default timeout in the constructor.
d = DHT(privatekey=..., timeout=3600)   # default TTL of one hour

Note that a data storage API can be built on top of a routing and connectivity API. Since we would like to separate the abstractions (data storage and peer connectivity/routing), we will define separate sets of APIs for these.
d = DHT(net=..., .privatekey=..., timeout=...) # use the given connectivity (net) object

Peer connectivity and routing
The connectivity and routing layer deals with maintenance of P2P network. Programmers are familiar with socket abstraction and there has been effort to map P2P to socket abstraction [2].
s0 = ServerSocket(...)  # a P2P node is created
s0.bind(identity=..., credentials=....) # node joins the P2P network
The bind method is similar to the JXTA semantics, and similar to the attach method as proposed in IETF P2P-SIP work. Actual communication with a specific peer can happen over a connected socket. A connected socket is returned in the connect or accept API methods on the server-socket.
s1 = s0.connect(remote=...) # connect to the given remote identity
s2, remote = s0.accept() # receive an incoming connection from remote
The connection procedure takes care of negotiating connectivity checks using ICE (or similar algorithm) to allow NAT and firewall traversal.

Once connected, the socket can be used to send or receive any message to the peer.
s1.send(data=..., timeout=...)
data = s1.recv()
The original server-socket can be used to send or receive messages to specific peers without explicit connection.
s0.sendto(data=..., remote=...)  # send to specific peer
data, remote = s0.recvfrom(timeout=...)
Sometimes, a node needs to send a message to any available node close to a given identifier. This can be implemented using overloaded sendto method that takes either a remote address or a key identifier. The latter performs P2P routing to the given destination key identifier.
s0.sendto(data=..., key=...)
data, remote, local = s0.recvfrom(...)
if local == s0.sockname: # received for this node identifier
...
else: # received to the given key presumably close this node
key = local
A connected socket can be disconnected or a node removed from the P2P network using the close method.
s1.close()  # close the connection
s0.close() # remove the node
These abstractions allow building a distributed object location and routing APIs [1] using locality aware decentralized directory service.

Group membership
Group membership APIs are similar to the multicast and anycast socket APIs. A node can join or leave a group, and a message can be sent via multicast or anycast to one or more nodes in a group. Let us define a group identifier similar to that in JXTA. We extend the previous socket abstraction to create a new group.
s3 = s0.join(group=...)  # join a given group
The join method returns a semi-connected socket object which can be used to send or receive packets on the given group.
s3.send(data=...)        # send to every node in the group
s3.sendany(data=...) # send to at most one node in the group
data, remote, key = s3.recv() # receive a multicast or anycast data
The socket API gives an intuitive abstraction to understand the P2P concepts. The actual implementation of the group membership may be more complex than simple routing and connectivity. Closing a group socket leaves the group membership.

We have seen how a P2P API can evolve to accommodate various concepts into existing well known abstractions such as hash table and socket.

References:
1. Towards a common API for structured peer-to-peer overlays (2003) http://oceanstore.cs.berkeley.edu/publications/papers/pdf/iptps03-api.pdf
2. The Socket API in JXTA 2.0 http://java.sun.com/developer/technicalArticles/Networking/jxta2.0/

Welcome to 39 peers!

I have launched an open-source project named "39 peers". From the web site:

"The 39 Peers project aims at implementing an open-source peer-to-peer Internet telephony software using the Session Initiation Protocol (P2P-SIP) in the Python programming language. The software is still incomplete -- especially the P2P part.

Peer-to-peer systems inherently have high scalability, fault tolerance and robustness against catastrophic failures because there is no central server and the network self-organizes itself. Internet telephony can be an application of peer-to-peer architecture where the participants locate and communicate with each other without relying on expensive or managed service providers. 39 peers project is an attempt to provide a open source and free-for-all peer-to-peer network targeted towards open standards based real-time communication.

The 39 peers project is developed for student developers and researchers to experiment with new ideas. It is written in Python scripting language. It supports open protocols such as IETF SIP and RTP. It is licensed under GNU/GPL license."

memcached

Recently I got a chance to look at memcached a distributed in-memory cache.

There are two ways of scaling databases: (1) replicating with master-slave such that reads can be done from any slave database whereas writes needs to be done in master, and (2) distributing the data among segments such that accessing a data needs to access only a particular segment instead of the whole database. The first technique doesn't work well for scaling SIP-related contact data, because number of writes (login/logout, presence updates) are significant compared to number of reads (call routing, IM delivery). Distributing the data among segments has been done before, and my thesis also describes it in the context of SIP telephony.

The memcached system uses this second technique, hence has much better performance in certain cases. It is targeted towards web applications that generate dynamic web content by accessing the database at the back-end. One could run memcached on several servers, potentially co-located with the web servers in the server farm. The dynamic web service (e.g., written in PHP) is then modified to first look in the cache, and if not found then query the actual database (and update the cache as well). The cache provides a distribute hash-table (DHT) interface.

When the data is updated in the cache, it gets stored in the appropriate memcached instance based on some hash of the data key. When a query is done, that instance is queried for the data. Each cache instance can be configured with a limit of memory size, typically 1-3GB based on the available memory in the system. The distributed nature of the cache allows you to linearly scale the cache by just adding more instances of the cache on different machines.

At first glance this looks very promising for use in a P2P-SIP application. However, there are some serious limitation. Although memcached has built-in hashing mechanism for data storage, there is no automatic replication of data (hence no fail-over), the hashing needs to be implemented by the client itself (hence no recursive queries). In particular, memcached implements only one part of a full P2P data storage application such as OpenDHT. Because of this limitation, it cannot be used effectively for P2P-SIP client implementations without adding a lot of additional software. Nevertheless, memcached can potentially be used for P2P-SIP proxy server farm.

As a new project idea, it would be interesting to try to use memcached in openser SIP proxy server to readily build a P2P SIP server farm without much overhead! If you are project student willing to work on this, I will be happy to mentor you.

Business out of P2P-SIP

Here are some of my thoughts on how to make (or save) money with P2P-SIP.

There are two points to note: (1) an end-user doesn't know (or care) whether he communicates using client-server or peer-to-peer as long as he gets all the features (audio, video, text, conferencing, offline message, etc), security and speed. (2) unlike service infrastructure where the provider can make money out of user base (either charging per call or per minute as in Skype-out or via advertisements as in youtube?), there is no solid business model for P2P-SIP. Since it is open protocol, anyone can develop a client and break free from the provider. This of it as a email client technology -- how many vendors make money selling email clients?

There are some alternatives to this ideal view of P2P-SIP though. For example, one could control the user base -- keep the enrollment of the user controlled by the provider and allow communication among the clients of the same provider. This requires that the implementation be proprietary (at least the security and validation part), otherwise anyone may write an alternate implementation to bypass the validation. Thus the protocol doesn't remain open and is not P2P-SIP.

Alternatively, one could build a server-farm of SIP proxies using P2P technology so that the maintenance cost of the servers is reduced. However, this still requires dedicated set of servers (electricity, bandwidth, call centers, etc), hence the cost saving is very low. Even though you may be using P2P-SIP you won't save much money.

Thirdly, one could build and sell P2P-SIP software, e.g., for enterprises, that doesn't require expensive call managers or central servers. This works well, but has one problem. People have come to expect that communication, at least PC to PC, is free. Unless the client software provides tons of new features, which cannot be easily done by another vendor, selling such a client software is also difficult.

So, how do you make money? You don't. Instead, you can save money.

You can save money that you would otherwise be spend on maintaining the server infrastructure or you can enable new businesses that were earlier not possible because of close systems. Most of the current communication mechanisms have evolved in a tightly controlled environment, e.g., Mobile phones, Skype, Yahoo, etc. Instead what we need is an open mechanism similar to emails and web. An email client is not tied to a single server. A web client (browser) is not tied to a single server or web site. A communication client should not be tied to a single provider.

Consider an open system where anyone can call anyone else without paying to a provider beyond the Internet connection fees. This opens door for a whole new set of businesses. For example a small time vendor, consultant or topic expert can lend his service in real-time for a small fee without going through the hassle of setting up agreement with a provider. Existing systems such as social network sites or content distribution networks can have additional feature that allows a user to interact with his friends, or other users on the same site, or other users watching the same content, irrespective of the attachment of the user to the provider. There is a huge potential for innovations in the communication client space, similar to how web browsers have evolved.

So, who can do this? I don't think it will come from existing businesses who have large user base -- this includes the likes of Yahoo, MSN, AIM, or Skype. The reason is that they wouldn't want to open up their users to others. I also don't think it will come from existing businesses that are targeted for selling advertisements -- the likes of Youtube, Google, etc.

The question to ask is who benefits from ubiquitous P2P-SIP? Media companies and content owners should invest in making P2P-SIP ubiquitous so that their existing system is improved or becomes more appealing to end users. Such companies can probably sponsor open source projects for P2P-SIP. If there is a better and free communication infrastructure, all sorts of new innovations are possible. For example, people would like to talk with their friends while watching a TV series online, a cricket match online or a movie online.

In conclusion, I feel that big companies that can benefit from real-time communication, but don't have an existing solution to work with, should invest in open-source P2P-SIP technologies. If you are interested in talking about it, feel free to drop me an email.

Data format and interface for external P2P mode

Last month, Henning and I submitted an internet-draft on "Data format and interface to an external peer-to-peer network for SIP location service". The text and HTML version are available. The idea is to use an interoperable XML format for accessing the DHT (which is based on OpenDHT's interface) and an XML format for storing user location data such as contact address, cryptographic keys and certificates, offline messages, presence watcher information, etc.

Structured vs unstructured P2P or Why we chose DHT for P2P-SIP?

One of the questions that people ask me is 'whether structured or unstructured P2P works better for SIP-based Internet telephony?'. My answer is that structured P2P such as distributed hash table (DHT) works better because of the following reason.

Unstructured P2P networks usually rely on caching and replication to improve the lookup performance and reliability. In the case of file-sharing application, once the file named "matrix-II.mpg" is uploaded in a P2P network such as Kazaa, the content of the file remains the same (un-mutable). Thus, for reliability the file may get replicated at multiple places. Intermediate nodes in the download path may cache the file locally to improve download performance by other downstream nodes. On the other hand, a phone's contact location or IP address uploaded under the resource named "bob@home.com" may change often (i.e., mutable), e.g., if bob changes his device or the device's IP address is changed. This makes all the randomly replicated and cached copy of the resource content useless. Thus, random caching and replication is not useful.

In unstructured P2P network, any replica of the file "matrix-II.mpg" is good enough, whereas in Internet telephony you want to reach only "bob@home.com" but not some replica of Bob. You can not replicate Bob's phone. The only thing that can be replicated is the mapping between Bob's identifier and his phone's IP address. However, to allow frequent updates and consistent view of the mapping, it is desired that the mapping is replicated in a structured manner so that both Bob can easily update the mapping and any prospective caller can easily query the up-to-date copy of this mapping. This means given the resource name, we can know where the resource should be present in the P2P network. Thus, it is structured.

Unstructured P2P is suitable for random or blind search and allows wild-card search. For example, you can search for a file name containing "matrix" keyword. On the other hand, in Internet telephony most calls are made to the known destination (phone number or destination user identifier). Hence, hash table interface that allows lookup based on a key is good enough. Distributed hash table (DHT)-style P2P algorithms such as Pastry, Chord, Bamboo provide ideal platform for such lookups.

Skype is believed to be unstructured among the super-node, but I think there may be some form of structure in how the data is stored among the super-nodes. Plus, there is no guarantee on upper bound on lookup cost which means it might be falling back to some centralized lookup if the number of hops in search exceed some limit. Since Skype is proprietary, we do not know how exactly it does lookups.

New internet-drafts

Last week, there were some new I-Ds on p2p-sip.

draft-matthews-p2psip-nats-and-overlays-00.txt
The draft analyzes various issues with NAT in the context of P2P-SIP and presents some important conclusions: (1) partial mesh of peers is more suited than ring or full mesh connections, for example; (2) connections can remain mostly static for maintenance of p2p; (3) structured p2p is more suited because of bounded connection count; (4) symmetric P2P (such as Kademlia?) is more suited because the connections across NAT are bidirectional; and (5) use of superpeers where all peers behind a common NAT can elect a small number of superpeers which handle connections across the NAT. The basic idea is to understand the NAT limitations such as number of pinholes allowed and timeout, and apply these for a deployable architecture.

draft-shim-sipping-p2p-arch-00.txt

The basic idea is to use the SIP-using-P2P architecture. There is some overlap with my 'external DHT' technical report. Some points of contentions are: the draft proposes P2P lookup first, and then fall back to DNS (RFC3263) which has a negative effect on performance since P2P lookup is slower. The format of the record can reuse the SIP Contact header instead of specifying individual terms such as TCP80, UDP5060, etc. The login server is clearly undesirable. But the draft is a good starting point. We will need more maturity in areas of security, authentication, and data format.

draft-mayrhofer-enum-domainkeys-00.txt
It proposes to use telephone numbers instead of email like user@domain format because one can validate that the number owner actually owns the telephone. The main problem is that phone numbers are too rigid and tied to physical port or line, thus we cannot have identifiers for virtual users or devices, e.g., sip:lamp@cs.columbia.edu belongs to an electric X10 controlled lamp in our lab. Secondly, phone numbers are hard to remember, thus needs address book. Thus it doesn't make much difference whether we use phone number or email as long as we know the remote's public key. The main problem to be solved is the distribution of public key in p2p. A certificate-based approach can be used instead.

draft-quittek-p2p-sip-middlebox-00.txt
It tries to list various requirements for NAT traversal in P2P-SIP. Not much new value. Ignores some of the existing NAT traversal schemes such as ICE.

In conclusion, there are still a lot of open issues in P2P-SIP, most notably in security, authentication and NAT/firewall traversal. Any solution for P2P-SIP must reuse the SIP's NAT traversal schemes such as ICE. There is a new version of best practices draft for NAT traversal in SIP.