Saturday, May 23, 2009

APIs for SIP applications

There are three types of SIP APIs: (1) source code API such as ones defined by JAIN SIP, libsip++ and pjsip, (2) high-level API to control the per-call behavior such as CPL, SIP-CGI, SIP Servlet, LESS, or (3) pseudo-code style API to control the behavior of the server or client such as SER or sipp config files. They all serve different purposes. The source code API is needed to create new software applications using existing libraries, the high-level API creates easy to define services, and the config file allows creating server or client behavior/scenarios.

With Python as the programming language, it is possible to create single software to expose these different types of APIs. This is because Python source code looks like human understandable pseudo-code if written with care. It makes sense to expose the APIs in my P2P-SIP 39 peers project to support such behavior. In this post, I present some of my initial thoughts on how to implement the generic API.

Firstly, the existing SIP module (rfc3261.py) already has an easy to use source code API. This is further enhanced in additional modules such as voip.py for user-agent specific functions. In particular, the source code API exposes object-oriented classes such as Transaction, Dialog, Stack, etc., to perform the functions defined in various layers in RFC 3261.

Secondly, the high-level API such as CPL and SIP-CGI can be implemented using per-user scripts in python itself (instead of XML for CPL, for example). The SIP server or client can import the per-user script based on the request-URI of the request and handle the processing. An example script to perform redirect to voice mail is shown below:

def redirect_to_voicemail(event):
event.location = URI('sip:jones@voicemail.example.com')
event.action.redirect()

def proxy_then_voicemail(event):
if event['From'].uri.host.endswith('example.com'):
event.location = URI('sip:jones@example.com')
e = event.action.proxy(timeout=10)
if e in ('busy', 'noanswer', 'failure'): redirect_to_voicemail(event)
else: transfer_to_voicemail(event)

basic.addEventListener('incoming', proxy_then_voicemail)

Because of the un-safe nature of Python, we will lose a number of features provided by CPL, but nevertheless the programmable API is easy to read and write, and nicely integrates with the existing source code. Similar extensions can be built for user agent applications similar to the LESS programming API.

Thirdly, instead of having the SIP client or server import the per-user script, the client or server itself can be written in the script. The following example shows how to translate some parts of SER config script to create a new server scenario.

from sipapi import *
import re

_debug = True # enable debug trace
open(('67.93.12.18', 5060)) # listen on this ip:port for incoming packets

def route(event):
# sanity check section
if event['Max-Forwards'] and int(event['Max-Forwards'].value) <= 0:
return event.action.reject(code=483, reason='Too many hops')
if len(str(event)) > 8192:
return event.action.reject(code=513, reason='Message overflow')
# this is used by sipsak to monitor the health of server
if event.method == 'OPTIONS':
if event['From'].uri.user == 'sipsak' and not event.uri.user:
return event.action.accept()
...
basic.addEventListener('incoming', route)

run() # the loop to process the SIP listening point

Similar scripts can be written to create client scenarios similar to how sipp creates scenarios from configuration files.

I feel a generic SIP API will make the job of Python programmers easier, instead of having to learn new APIs and implement them in custom SIP servers and clients. In any case, the article just poses some ideas, and I will be happy to mentor any student who would like to work on this project!

Sunday, May 17, 2009

Why Open Source?

In this article I present my view on Open Source Software. There are a number of articles elsewhere that talk about advantages of Open Source and why it works [1][2][3]. (I use the term Open Source to mean Free Software, instead of diving into OSS vs FS debate.)

Point 1: Most software organizations are driven more by business objectives, and less by technology.

If you look around you notice several types of software companies: most of them are commercial ones like Microsoft, there are some like Sun that are commercial but pretend to do open source work, very few like Red Hat that build business out of open source software, and finally, there are some true open source software like Linux and Apache. I think most of Sun's Open Source attempts are for business reasons to provide competition to other existing businesses.

Point 2: Software programming is an art.

It is like learning a new language or doing sculpture or painting. Initially a programmer is too involved with the syntax and semantics of the language constructs and application. Once he is proficient, the programming comes naturally. For example, a good Java developer will think in terms of high level modules and classes, but once he starts implementing them things move smoothly instead of worrying about where to put the open bracket or when to create a new method. After a while, the software development process becomes an art of making beautiful, modular and efficient software. Some people have a built-in talent for the particular art, but most people learn by practice, practice and practice.

Point 3: Good art requires personal motivation.

Doing a scientific experiment is different than painting a new picture. At the broad-level, given the set of input data, and experimental setup, one is likely to achieve the same result in an experiment. On the other hand, an art piece requires inherent motivation of the artist and his personal inspiration to do something new, something different. Sometimes the inspiration gets driven by commercial interest, in which case an artist may end up producing art work without much motivation -- e.g., in the commercial Indian film industry, a music director has to produce tens or hundreds of scores per year, affecting the quality of the art as well as causing plagiarism from western music. On the other hand, motivated musicians who record one album a year do generate quality and innovative work.

Considering the above three points, a software engineer who is paid to work on a particular technology or piece of software is less likely to be personally motivated to create that piece of software. On the other hand, an open source developer who is not paid for his work initially, starts the open source work because of his personal motivation to create that piece of software. Thus, an open source software is more likely to be of better quality compared to a commercial software with the same amount of testing. Hence commercial software requires quality assurance to make it competitive with open source. Although quality assurance can reduce software bugs, it doesn't make the 'art' as good as the open-source version. If you compare the design of the Apache web server or SIP express router with their commercial counter parts, you can understand what I mean by 'art' in this context.

When I look at an open source software, I assume it reflects ideas and vision of the innovative and inspired developers who were motivated to write that piece of software. When I look at a commercial software, I assume it is created by software engineers who got paid to write code, to write documents, to test code, and more than that it is sold by salesmen who are paid to sell those pieces of software. I don't see much personal motivation or inspiration in the loop, and I don't aspect the design or the implementation of the software to be a good piece of art. (In small companies where all the people have same vision or inspiration about the software, it is possible to create quality work. However, things change over a period of time as the company grows and new software engineers are paid to perform work on other people's innovations.)

In the context of P2P-SIP, we haven't seen much of commercial interest beyond the few initial contenders. The main reason is that P2P-SIP inherently is peer-to-peer, and against the business model of services that the modern web/phone industry is so accustomed to. In other words, the industry has not figured out a way to make money out of open P2P-SIP. On the other hand, there are a number of developers who created open source prototype applications out of personal inspiration. The next steps for the open source P2P-SIP developer community: to build a bigger community, advertise and publish their work, prove that it works better than existing solutions, and use it on a daily basis!

Tuesday, May 05, 2009

Documenting RFC implementations

Just wanted to document how I am using the existing documentation such as IETF RFCs and Internet-Drafts in my 39 Peers project. You can looks at some samples at here or here.

I wrote a htmlify.py script. The script uses the SilverCity python package to decorate the base python code. To include the documentation from RFCs, the script interprets the python source code to identify lines such as

# @implements RFC2617 (HTTP auth)

If found, it downloads the particular RFC from IETF website, removes any formatting empty lines such as near page breaks, and numbers all the pages and lines. It then stores the resulting file as a text document which you can lookup and use in your documentation.

Furthermore, the script identifies lines such as

# @implements RFC2617 P3L16-P3L25

If found, the line is replaced by a HTML DIV block with content of the documentation in RFC2617 text file from page-3 line 16 to page-3 line 25.

I find this technique pretty handy, and keeps my decorated source code with inline documentation extracted from RFCs and drafts, instead of I having to write extensive document.