Articles on technology, Internet, Protocols, Web, Open Source, VoIP, P2P, SIP, RTMP and WebRTC.
WebRTC notification system and signaling paradigms
Writing coherent dialogs using Twilio
dialog package to create such a dialog as a
single and simple coherent script in Tcl. The script performs programmed interactive voice or
messaging interaction using the popular cloud telephony platform, Twilio.It should also work, after trivial modifications, with other similar systems such as Restcomm or Somleng that use TwiML or similar markups. The package can further be extended to support other popular dialog languages such as VoiceXML. The core idea can be re-implemented using other scripting languages such as Python.
Scripting VoiceXML and TwiML using Tcl
- How easy is a programming language?
- What is Tcl?
- What are XML-based documents?
- How do I get started?
Introducing rtclite
"Light-weight implementations of real-time communication protocols and application in Python


Getting started with WebRTC
The first step is to download a browser that supports WebRTC, e.g., Google Chrome Canary. In your browser address bar, go to chrome://flags, search for Enable Media Stream and click to enable it. This enables the WebRTC APIs for your web applications running in the browser. Use this browser for getting started with the following two projects.
1. The first project is voice and video on web that enables web-based audio and video conferencing among multiple participants. It uses resource-oriented data access to move all the application logic running to the client application in HTML/Javascript, whereas the server merely acts as a data store interface. The architecture enables scalable and robust systems because of the thin server. The source code of WebRTC integration is available in the SVN branch, webrtc. Follow the instructions on the project web page to install and configure the system, but use the link to the webrtc branch instead of trunk in SVN. After visiting the main page of the installed project, see the help tab to create a new conference. On the preferences page, make this conference persistent, check to enable WebRTC and uncheck to disable Flash Player for this conference. Visit the conference from multiple tabs in your browser, using different user names. Let the moderator participant enable the video checkboxes for various participants to see the audio and video flowing via WebRTC among the participants.
2. The second project is SIP in Javascript that creates a complete SIP endpoint running in your browser. It contains a SIP/SDP stack implemented in Javascript along with a SIP soft-phone demonstration application. It has two modes: Flash and WebRTC. In the WebRTC mode, it uses a variation of SIP over WebSocket and secure media over WebRTC. The default mode is to use Flash, which you can change to WebRTC as follows. Checkout the source code from SVN and put it on a web server. Install and run a SIP proxy server that supports SIP over WebSocket as mentioned on the project page. Now visit the page on your web server phone.html?network_type=WebRTC on two tabs of your browser. Follow the getting started instructions on that page to register on the first tab's phone with one name, and on the second with another, and try a call from the first to the second. This is an example program trace at the proxy server from a past experiment on how the WebRTC capabilities are negotiated between the two phones using SIP over WebSocket. In addition to the network_type, you can also supply additional parameters on the page URL to pre-configure the phone, e.g., username, displayname, outbound_proxy, target_value. Example links are available to configure as alice and bob to connect to the web and SIP server on localhost, assuming your webserver is running locally on port 8080 and SIP server on websocket port 5080. Just open the two URLs in the two tabs, click on Register on each, make a call from one to another by clicking on the Call button, accept the call from another by clicking on the Call button, and see the audio and video flowing via WebRTC between the two phones.
The Philosophy of Open Source
Flash-based audio and video communications in the cloud
Voice and Video Communications on Web
I co-authored and presented a paper on "SIP APIs for voice and video communications on the web" at IPTcomm 2011. The paper compares various alternative architectures, and presents the components of our ongoing project at IIT, Chicago. We are open to sponsorship of the project to further continue its R&D work. Please feel free to get in touch with me or Prof. Davids if you are interested in sponsoring student projects in her lab related to this technology.
A Proposal for Reference Implementation Repository of SIP-related RFCs
Clearly the text in RFCs and drafts cannot be as unambiguous as real source code of a computer program. So many programmers may read and implement some features differently, resulting in non-interoperable implementations. Having a readily available pseudo-code for SIP and many of its extensions relieves the programmer of error-prone step (2) above, and resolves any misinterpretation at an early stage. There is a huge cost paid by the vendor or provider for this programmer's misinterpretation of the specification.
This project proposal is to keep an open and public repository of reference implementation of RFC 3261 and other SIP-related extensions. If this repository is maintained by public bodies such as SIPForum and open source community, it will enable easy access to developers and enable better interoperability of new extensions.
The goal of this effort will be to encourage submission of reference implementations by RFC and Internet Draft authors . In case of any ambiguity, the clarification will not only be applied to specification but also to the reference implementation.
If we use a very high level language such as Python then the reference implementation essentially also serves as a pseudo code, which can be ported to other programming languages. The goal is not to get involved in the syntax of a particular programming language, but just express the ideas more formally to prevent misinterpretation of the specification. Perhaps if Python is not suitable, then a similar high level language syntax can be defined.
This will greatly simplify the job of a programmer, and in the long term, will result in more interoperable and robust products seamlessly supporting new SIP extensions and features. The programmers will have fewer things to worry about; hence can write more accurate code in the short time. From an specification author's point of view, it will encourage him/her to write more solid and implementable specification without ambiguity, and encourage him/her to provide the pseudo-code in the draft. From a reviewer's point of view, one can easily judge the complexity of various alternatives or features, e.g., one can say that adding the extension 'foo' is just 10 lines of pseudo-code to the base SIP reference implementation.
It will help RFC and draft authors in seeing the complexity and implementation aspects of their proposal. Sometimes an internet-draft proposes multiple solutions without any details on them. This is partially due to the lack of implementation and complexity evaluation of the various approaches. With reference implementation and pseudo-code repository, the author can provide a patch to the existing code to evaluate the complexity of the proposal.
A few years ago I wrote a tool to annotate software source code with RFC/draft, so that when you are reading a class or method in a source code file, you can quickly know which part of the RFC/draft it implements. Please see an example here and here. Such annotations in reference implementation will help in co-relating the RFC/draft with the actual implementation.
If there is wide support for this proposal, we can raise it to SIPForum or other bodies, we can help get started and bootstrap the repository of reference implementations of a few SIP-related RFCs. Then we can invite contributions from the community and RFC/draft authors towards completing the implementations. Please post your comment to let us know what you think.
Implementing video conferencing and text chat using Channel API
"Random-Face [4]: This is a chatroulette-type application built using the Flash VideoIO component on Adobe Stratus service and Python-based Google App Engine. ... You can view the source code of two files, index.html that renders the front end user interface and main.py that forms the back-end service."
"Public-Chat [5]: This is a multi-party audio, video and text chat application built on top of Python-based Google App Engine and using Channel API for asynchronous instant messaging and presence. ... Developers can see the source code files: index.html is the front-end user interface, webtalk.js is the client side Javascript to do signaling, and main.py is the back-end service code."
The Channel API essentially implements an XMPP-style asynchronous communication from your server to the Javascript client. I use this to implement notifications for new messages, change in user list, and update of user video session to other participants in the system.
Lessons in starting a software project
- Brainstorm often: During the initial phases of software growth or even before starting to write a single line of code, you should do several sessions of brain storming. It could be on validating your idea, figuring out competition, predicting the future, picking a programming language, potential learning, etc. This is the difference between carefully planned birth versus unexpected pregnancy. Just because you can write some software, should you? Especially if better alternatives exist?
- Use good version control system: Even for the most trivial projects, you should try to use version control system. I like SVN (subversion) for my open-source projects, but if you can afford git, it works better for complex project management. If you are starting an open source project, consider code.google.com for hosting your SVN repository -- it is fast, simple and hassle free. It is like a good home for your baby software.
- Document all ideas: When the software is evolving you will have many ideas for new features, doing things differently, or incorporating competing features. Obviously due to lack of resources and time, you won't be able to incorporate all these. But you must document all the ideas, and if possible prioritize them. Keep a single list of ideas. Usually the software will evolve on its own to attract new features. Implement only the most crucial ideas and features, and resist the temptation to add many features.
- Few developers during growth: Keep the core set of excellent developers to one, two or at most three when the project is growing. Every major piece of software should have only one excellent developer. This avoid unnecessary friction and induces feeling of ownership. Software is like a baby, which needs a good parent to raise and grow, before it can mature and face the world. You wouldn't want to raise your software in a foster house where nobody feels ownership, i.e., in an organization with an engineering "team".
- Pick the right language and tools: Every programming language has some strengths and weaknesses. Make sure you select the right language, that is quick to develop with and maintain, and works well for your target application. For example, with low-level C/C++ you get performance, and with high-level Java, Python, you get portability. Over the years I have liked Python for most of my applications. Unfortunately, in corporate environment, Java is the pet-child because there are many fold more software developers and managers who know Java well. For modern Internet and web applications, Python, Ruby, Erlang and ActionScript are becoming more popular.
- Include testing and defensive programming: To be successful, sooner or later your software project will need to get out of the demo-mode and face the real world. It might become too late at that time to worry about scalability or glaring bugs if those involve redesigning your software. It saves a lot of time and energy to use common techniques such as good logging, unit testing, performance best practices, and defensive programming from day 1. Also maintain an issue tracker and log even the tiniest of issues with your software. Sooner or later you will need to address them.
- Don't procrastinate: If you have an idea to work on, don't procrastinate. Just get started, write something up, try to get a prototype going. Most successful projects need a complete re-write at least once. So don't be afraid to write throwaway code.
- Don't document before coding: While software engineering people will say that you should follow good software process -- writing requirements specification, design document, test cases, etc. -- those can be written later too! Source code is what makes or breaks a software. You can write detailed specification and design documents, after you already have a prototype and want to document it or propose a change. In my experience, any design document written before writing the code is incorrect, and needs to change drastically after the source code is written.
- Don't spend time on one-off items: For your software, there are some items which are directly related, and then there are one-off items. For example, for a VoIP client, the protocol implementation, good voice quality, etc., are directly related. On the other hand, having a user signup page, instant messaging text chat, file sharing, etc., are one or two-off items, which are not directly related, but indirectly assist users in VoIP. When you start a project, do not spend time doing one-off items, but work on directly related items first.
- Don't wait too long for 1.0 release: There is 80% difference between an 80% complete software and a released software. When you formally release your software, you have to take care of user manual, getting started guide, installer as well as finish those last annoying bugs. In the case of software projects, it is very easy to get started but very difficult to put an end. There is always an endless list of features which needs to be completed before the release, and hence your release never happens. Unless, you make it happen. You will have to make a firm decision about what bugs are important and what can remain as known issues for version 1.0.
REST, RESTful and restlite
restlite = REST + Python + JSON + XML + SQLite + authentication
Features
- Very lightweight module with single file in pure Python and no other dependencies hence ideal for quick prototyping.
- Two levels of API: one is not intrusive (for low level WSGI) and other is intrusive (for high level @resource).
- High level API can conveniently use sqlite3 database for resource storage.
- Common list and tuple-based representation that is converted to JSON and/or XML.
- Supports pure REST as well as allows browser and Flash Player access (with GET, POST only).
- Integrates unit testing using doctest module.
- Handles HTTP cookies and authentication.
- Integrates well with WSGI compliant applications.
Motivation: As you may have noticed, the software provides tools such as (1) regular expression based request matching and dispatching WSGI compliant router, (2) high-level resource representation using a decorator and variable binding, (3) functions for converting from unified list representation to JSON and XML, and (3) data model and authentication classes. These tools can be used independent of each other. For example, you just need the router function to implement RESTful web services. If you also want to do high-level definitions of your resources you can use the @resource decorator, or bind functions to convert your function or object to WSGI compliant application that can be given to the router. You can return any representation from your application. However, if you want to support multiple consistent representations of XML and JSON, you can use the represent function of request.response method to do so. Finally, you can have any data model you like, but implementations of common SQL style data model and HTTP basic and cookie based authentication are provided for you to use if needed.
This software is provided with a hope to help you quickly realize RESTful services in your application without having to deal with the burden of large and complex frameworks. Any feedback is appreciated. If you have trouble using the software or want to learn more on how to use, feel free to send me a note!
Beauty of open source
Beauty | Software project |
Natural beauty is, well, natural, whereas cosmetics give artificial sense of beauty. | Open source software are built when some motivated developer feels like building something, where as commercial software are mostly built by engineers who are paid and forced to build. |
Natural beauty is long lasting, whereas makeup wear down after few hours or days. | Open source software lives longer, whereas commercial projects tend to get out beaten by competition sooner or later. |
You are born beautiful and don't have to pay for natural beauty, whereas cosmetics cost money, big money. | Open source software are mostly free, whereas you have to pay for commercial software. |
It is hard for salesman to sell fruits and water to enhance your beauty, whereas it is easy for salesman to sell cream and nail-polish. Sometimes these advertisements are deceptive. | Usually you don't find people advertising their open source work much, whereas companies have dedicated sales team to sell the commercial product. Sometimes these sales pitch are deceptive. |
Natural beauty is usually open and does not hide scars, whereas cosmetics are meant to hide scars, marks, etc., to give a (false) sense of beauty. | Open source software are open source code, where developers can jump right in the code and see things. Commercial software hides the source code, and instead presents documentation, power points, sales pitch, etc., to give a (false) sense of what is inside and actually hiding what is inside (source code). |
Natural beauty does not require support. On the other hand if you are putting a nail polish, you will need a remover; if you are putting on make up, you will need to remove it before bed. | Open source software usually comes with no support. If you have it, you own it, and you put up with it. Commercial software usually comes with (expensive) support system. If you have it, you need to keep paying for bug fixes and upgrades. Otherwise it will harm you sooner or later. |
Why Open Source?
Point 1: Most software organizations are driven more by business objectives, and less by technology.
If you look around you notice several types of software companies: most of them are commercial ones like Microsoft, there are some like Sun that are commercial but pretend to do open source work, very few like Red Hat that build business out of open source software, and finally, there are some true open source software like Linux and Apache. I think most of Sun's Open Source attempts are for business reasons to provide competition to other existing businesses.
Point 2: Software programming is an art.
It is like learning a new language or doing sculpture or painting. Initially a programmer is too involved with the syntax and semantics of the language constructs and application. Once he is proficient, the programming comes naturally. For example, a good Java developer will think in terms of high level modules and classes, but once he starts implementing them things move smoothly instead of worrying about where to put the open bracket or when to create a new method. After a while, the software development process becomes an art of making beautiful, modular and efficient software. Some people have a built-in talent for the particular art, but most people learn by practice, practice and practice.
Point 3: Good art requires personal motivation.
Doing a scientific experiment is different than painting a new picture. At the broad-level, given the set of input data, and experimental setup, one is likely to achieve the same result in an experiment. On the other hand, an art piece requires inherent motivation of the artist and his personal inspiration to do something new, something different. Sometimes the inspiration gets driven by commercial interest, in which case an artist may end up producing art work without much motivation -- e.g., in the commercial Indian film industry, a music director has to produce tens or hundreds of scores per year, affecting the quality of the art as well as causing plagiarism from western music. On the other hand, motivated musicians who record one album a year do generate quality and innovative work.
Considering the above three points, a software engineer who is paid to work on a particular technology or piece of software is less likely to be personally motivated to create that piece of software. On the other hand, an open source developer who is not paid for his work initially, starts the open source work because of his personal motivation to create that piece of software. Thus, an open source software is more likely to be of better quality compared to a commercial software with the same amount of testing. Hence commercial software requires quality assurance to make it competitive with open source. Although quality assurance can reduce software bugs, it doesn't make the 'art' as good as the open-source version. If you compare the design of the Apache web server or SIP express router with their commercial counter parts, you can understand what I mean by 'art' in this context.
When I look at an open source software, I assume it reflects ideas and vision of the innovative and inspired developers who were motivated to write that piece of software. When I look at a commercial software, I assume it is created by software engineers who got paid to write code, to write documents, to test code, and more than that it is sold by salesmen who are paid to sell those pieces of software. I don't see much personal motivation or inspiration in the loop, and I don't aspect the design or the implementation of the software to be a good piece of art. (In small companies where all the people have same vision or inspiration about the software, it is possible to create quality work. However, things change over a period of time as the company grows and new software engineers are paid to perform work on other people's innovations.)
In the context of P2P-SIP, we haven't seen much of commercial interest beyond the few initial contenders. The main reason is that P2P-SIP inherently is peer-to-peer, and against the business model of services that the modern web/phone industry is so accustomed to. In other words, the industry has not figured out a way to make money out of open P2P-SIP. On the other hand, there are a number of developers who created open source prototype applications out of personal inspiration. The next steps for the open source P2P-SIP developer community: to build a bigger community, advertise and publish their work, prove that it works better than existing solutions, and use it on a daily basis!
Welcome to 39 peers!
"The 39 Peers project aims at implementing an open-source peer-to-peer Internet telephony software using the Session Initiation Protocol (P2P-SIP) in the Python programming language. The software is still incomplete -- especially the P2P part.
Peer-to-peer systems inherently have high scalability, fault tolerance and robustness against catastrophic failures because there is no central server and the network self-organizes itself. Internet telephony can be an application of peer-to-peer architecture where the participants locate and communicate with each other without relying on expensive or managed service providers. 39 peers project is an attempt to provide a open source and free-for-all peer-to-peer network targeted towards open standards based real-time communication.
The 39 peers project is developed for student developers and researchers to experiment with new ideas. It is written in Python scripting language. It supports open protocols such as IETF SIP and RTP. It is licensed under GNU/GPL license."
Business out of P2P-SIP
There are two points to note: (1) an end-user doesn't know (or care) whether he communicates using client-server or peer-to-peer as long as he gets all the features (audio, video, text, conferencing, offline message, etc), security and speed. (2) unlike service infrastructure where the provider can make money out of user base (either charging per call or per minute as in Skype-out or via advertisements as in youtube?), there is no solid business model for P2P-SIP. Since it is open protocol, anyone can develop a client and break free from the provider. This of it as a email client technology -- how many vendors make money selling email clients?
There are some alternatives to this ideal view of P2P-SIP though. For example, one could control the user base -- keep the enrollment of the user controlled by the provider and allow communication among the clients of the same provider. This requires that the implementation be proprietary (at least the security and validation part), otherwise anyone may write an alternate implementation to bypass the validation. Thus the protocol doesn't remain open and is not P2P-SIP.
Alternatively, one could build a server-farm of SIP proxies using P2P technology so that the maintenance cost of the servers is reduced. However, this still requires dedicated set of servers (electricity, bandwidth, call centers, etc), hence the cost saving is very low. Even though you may be using P2P-SIP you won't save much money.
Thirdly, one could build and sell P2P-SIP software, e.g., for enterprises, that doesn't require expensive call managers or central servers. This works well, but has one problem. People have come to expect that communication, at least PC to PC, is free. Unless the client software provides tons of new features, which cannot be easily done by another vendor, selling such a client software is also difficult.
So, how do you make money? You don't. Instead, you can save money.
You can save money that you would otherwise be spend on maintaining the server infrastructure or you can enable new businesses that were earlier not possible because of close systems. Most of the current communication mechanisms have evolved in a tightly controlled environment, e.g., Mobile phones, Skype, Yahoo, etc. Instead what we need is an open mechanism similar to emails and web. An email client is not tied to a single server. A web client (browser) is not tied to a single server or web site. A communication client should not be tied to a single provider.
Consider an open system where anyone can call anyone else without paying to a provider beyond the Internet connection fees. This opens door for a whole new set of businesses. For example a small time vendor, consultant or topic expert can lend his service in real-time for a small fee without going through the hassle of setting up agreement with a provider. Existing systems such as social network sites or content distribution networks can have additional feature that allows a user to interact with his friends, or other users on the same site, or other users watching the same content, irrespective of the attachment of the user to the provider. There is a huge potential for innovations in the communication client space, similar to how web browsers have evolved.
So, who can do this? I don't think it will come from existing businesses who have large user base -- this includes the likes of Yahoo, MSN, AIM, or Skype. The reason is that they wouldn't want to open up their users to others. I also don't think it will come from existing businesses that are targeted for selling advertisements -- the likes of Youtube, Google, etc.
The question to ask is who benefits from ubiquitous P2P-SIP? Media companies and content owners should invest in making P2P-SIP ubiquitous so that their existing system is improved or becomes more appealing to end users. Such companies can probably sponsor open source projects for P2P-SIP. If there is a better and free communication infrastructure, all sorts of new innovations are possible. For example, people would like to talk with their friends while watching a TV series online, a cricket match online or a movie online.
In conclusion, I feel that big companies that can benefit from real-time communication, but don't have an existing solution to work with, should invest in open-source P2P-SIP technologies. If you are interested in talking about it, feel free to drop me an email.