7 Project ideas on the topic of Web Apps and/or Multimedia Communication

Here are some project ideas largely related to multimedia communication, WebRTC and related technologies. Please get in touch with me if you would like to explore further, contribute to, or sponsor one or more of these projects. I will be happy to provide insights, guidance, provide partial code, and/or connect with the right folks. As I make more progress in any of these, I will create separate articles describing them.


  1. Application builder: An application builder for web, mobile and desktop apps, that uses drag-and-drop, flow charts, and state machines to quickly create applications, both the application logic and the user interface design and implementation. The focus is on web components based development, and the ability to mash up web components to create apps or other web components. Use the web components from the RTC Bricks project to start with, including its inline-code based documentation style, to discover component methods, attributes/properties, events and styles. This project goes beyond my previous idea in Artisy, to include any application, not just communication related. It should also automatically capture state transitions, and inter-component and client-server communication for logging and debugging during run-time of the built applications. This project is the natural next step of the RTC Bricks project. 
  2. Web apps with zero leaks: This could be a silver bullet to tackling data leaks in web apps. The basic premise is to decouple any user or app data from the app logic, move the ownership of the data to the end user, and prevent the app logic from sending any such data back to its hosted service. Although this applies to many web applications including many standalone productivity tools, communication and collaboration apps, and it does not apply to cases were the data is owned in principle by the service itself such as banks or libraries. In such cases, the project can still help, e.g., to consolidate transaction history and money management across all credit cards and banks for each a single user, such that such consolidate data is owned by the user. The project is inspired by unhosted.org, inline with my earlier attempts related to collaboration apps, but in this case, the system enforces the policies such that the end user is in complete control of her data, and the system prevents the app developer or any hosted service owner from accessing any raw user or app data, not even for logging or debugging. If this type of web app model is successful, it could create a different app data economy where the end user can benefit monetarily by leasing her data that were collected by using such zero-leak apps. 
  3. Auto-retry and concurrent WebRTC: The existing WebRTC and related APIs, the client side processes, and the client-server negotiation for establishing peer connections are quite low level. Many apps end up repeating similar app logic in the application code to handle retries and concurrency issues. For example, if getUserMedia fails or is unresponsive, a retry is desired with a different device or a curated media constraint, or if it succeeds after the user gave up or re-attempted, then the previously obtained stream should be cleaned up, or if the peer connections are started, stopped or manipulated in an abnormal order, the app needs to sequence the signaling messages using queue and wait. Some of these problems happen because a JavaScript Promise is not abortable by default. This project provides a shim layer, similar to the popular webrtc-adaptor.js, but allows abortable functions or methods where possible, allow sequencing and queuing API steps in the client side to ensure correct final state, and provide generic retries and timer handling of the various APIs using well known retry algorithms. It should also detect and log any real or potential concurrency issues during run time.
  4. Standalone video phone: A standalone generic multi-protocol video phone/app for multi-party audio, video, and text-based call and collaboration. The protocols include popular WebRTC, SIP, and even XMPP, as applicable. The app be can invoked from the web, on existing third-party video conferencing web services, in which case the user interface of the app is used for the video conference experience, whereas the third party service is used for signaling and control, using some ideas from the RTC Helper project. The app can be invoked from the web on any web site, in which case it can use the app's own signaling service, to allow virtual presence of and collaboration among participants or visitors of that web page or web site, and allows storing context dependent digital trail on that web page or web site, similar to LivingContent. The focus on this project is on the separate and independent video phone app, and not on the web page overlays.
  5. Auto-create family newsletter: Use AI to automatically create family newsletters using pictures and videos taken from family member's phone, text transcribed by members for various events, and non-work calendar, or non-work email conversations. The creation engine runs privately within the family's home network, such as on a Raspberry-Pi based self-hosted internal service, to avoid data leak of sensitive information, and receives any content proactively (not automatically) by the family members, e.g., by marking certain photos or videos on the phone, by bcc'ing some emails, or by using the mobile app to record or transcribe some events. The goal is to make it very user friendly, with verbal or natural language prompts, without requiring any technical acumen. The newsletter should be Canva-style editable, printable and structured content, but may include animated images or short videos for online viewing. The creator can decide the format, template, or whether to adhere to a fixed template, or use new innovative ones for each issue of the newsletter.
  6. Enhanced recording of conferences: Existing recordings in video conferencing systems are boring. Although attempts to summarize or auto-create meeting notes improve productivity, it remains boring. The goal of this project is to make such recordings exciting. For example, use AI to create a comic book style snippet based on the content, with speech bubbles for important conversations, and Gibli-style cartoons for certain steps. Or create video snippets combining not just the participant or shared content video, but also pictures and videos from external sources based on the context and content of the recording. When recordings are played, allow interaction with the recorded content, such as to expand individual participant or content share video, translate in real-time, create funny or catchy phrase, animated gif, or sound effects, or to link to any related content from earlier meeting recordings of the same series or of similar context.  The recording could be tailored to individual viewer for a more personalized experience, so that the notes or video snippet is different when a developer versus a manager views it. The goal is to make the recording watchable, similar to reading an exciting book, or watching an exciting Youtube video.
  7. Video ad-blockers for mobile and TV: Ad-blockers don't work the same everywhere. For example, recent popular browser changes are moving towards "allowable" advertisement, and deprecating technologies that actually can block ads (e.g., manifest v2). Moreover ad-blockers don't work on mobile browsers or smart TV. Ads on videos are harder to block in some cases, and not even possible in others. The goal of this project is two fold. First, use image, video and audio analysis of live stream to detect when an advertisement starts so that it can be blocked, silenced, removed or replaced. And second, use external system running outside the mobile phone or TV, that actually detects and removes video ads, and generates or forwards the video stream using WebRTC, Screencast, or related technologies, to display on the mobile phone or TV.


No comments: