The Web: a Video Platform for everybody

html5 video book
  • Video on the Web is still evolving
  • Video element since 2007
  • Lack of Adobe Flash on iOS since 2010
  • YouTube on Chrome using <video> since May 2014
  • Are you using <video> yet?
@silviapfeiffer

There's a new kid in town: WebRTC

Web Real-time Communications

Peer-to-Peer opportunities in the Browser

Your typical applications:

Think outside the box:

But: Is it ready yet?

Commercial Users

HipChat : video chat

Commercial Users

MailChimp : capture selfie for user setup

Commercial Users

Amazon Mayday : customer support

Commercial Users

SnapChat : capture photos

Commercial Users

Chromecast : screen sharing with your TV

Customer Service

Get your own "Mayday" button

Let's write some code

The relevant APIs

Camera Access in the Browser

What does that mean?

MediaStream API: Video Input Example

<video id="sourcevid" autoplay muted></video>
        
var video =
    document.getElementById('sourcevid');
var constraints = {
  "video": true,
  "audio": true
};
navigator.getUserMedia(constraints, success, error);
function success(stream) {
    video.src = window.URL.createObjectURL(stream);
}
        

Photo Booth: Grab a selfie

The architecture for camera access

camera access uses only a webserver

Requirements: a Web page, a USB camera, and

a Web server = Apache

Mailchimp are using it during setup of a user profile

Face detection

Face substitution

Responsive Typography

Augmented Reality

RTCPeerConnection in the Browser

Connect video to others

JS libraries you can use

Coding with rtc.io

WebRTC Peer Connected Video Example

<script src="//path/to/rtc.js"></script>
<div id="videos"></div>
        
var rtcOpts = {
  room: 'wdcnz',
  signaller: '//switchboard.rtc.io'
};
var rtc = RTC(rtcOpts);
var videos = document.getElementById('videos');
videos.appendChild(rtc.local);
videos.appendChild(rtc.remote);
        

RTC supports multi-peer

Mesh topology, unless you use an MCU

Open Source MCU: Janus

full mesh for multipeer connections

The architecture of a peer connection

peerconnection needs signalling server

Requirements: a Web page, a Web server, a USB camera on both ends

Additionally: a signalling server = rtc-switchboard

Message exchange via Signalling Server

Payload: SDP packet (Session Description Protocol)

Options for a Signalling Server:

  • XHR / Comet - Web signalling
  • SIP (Session Initiation Protocol) - VoIP and Telcos anyone?
  • XMPP/Jingle (Extensible Messaging and Presence Protocol) - Telepresence anyone?
  • email
  • carrier pidgeon
  • Websocket Server - socket.io or Primus.js

Let's set up a rtc-switchboard signaller

...based on Primus.js on node.js

Primus is a universal wrapper for real-time frameworks

Setup:

git clone https://github.com/rtc-io/rtc-switchboard.git
cd rtc-switchboard
npm install
SERVER_PORT=8997 node server.js

...and use it in our example

Diagnostics tools

Node Developers Shout-out!

Moving towards Deployment

What to deal with in the real world

NAT in a corporate network

Network Address Translation

introduce a STUN server

Requirements: a Web page, a Web server, a USB camera on both ends, a signalling server

Additionally: a STUN server (Session Traversal Utilities for NAT)

Firewalls in a corporate network

Deal with packet filtering

introduce a TURN server

Requirements: a Web page, a Web server, a USB camera on both ends, a Signalling Server

Additionally: a STUN/TURN server (Traversal Using Relay NAT)

How to specify STUN and TURN in rtc.io

Add to the setup of the RTC object:

var rtcOpts = {
  room: 'wdcnz',
  signaller: '//switchboard.rtc.io',
  iceServers: [
        { url: 'stun:rtcjs.io' },
        { url: 'stun:stun.l.google.com:19302'},
        { url: 'turn:turn.server:3478', credential: 'passwd', username: 'user'}
  ]
};
var rtc = RTC(rtcOpts);

Google's Test App in the Cloud

Remote Text Recognition on a paper document

Mobile to Desktop video conference

The RTCDataChannel API

DataChannels with RTC

RTCDataChannel API similar to WebSocket API

function bindDataChannelEvents(id, channel, attributes, connection) {
  channel.onmessage = function (evt) {
      messages.innerHTML = evt.data;
  };
  messages.onkeyup = function () {
    channel.send(this.innerHTML);
  };
}

rtc.on('ready', function(session) {
  session.createDataChannel('chat');
  session.on('channel:opened:chat', bindDataChannelEvents);
});

DataChannel setup with RTC

For pure DataChannel use, remove audio/video capture:

<script src="path/to/rtc.min.js"></script>
<div id="messages" contenteditable></div>
var rtcOpts = {
  room: 'wdcnz-data',
  signaller: '//switchboard.rtc.io',
  capture: false
};

var rtc = RTC(rtcOpts);

Example: Collaboration using a shared whiteboard

Telehealth application with shared whiteboard and shared pointers

Example: WebTorrent

Project by Feross Aboukhadijeh (PeerCDN - sold to Yahoo)

Develop a BitTorrent client for the Web using RTCDataChannel

webtorrent screenshot

Wrap-Up

Advantages over legacy Video Conferencing

  • New codecs: High audio and video quality
  • Security: Mandatory encryption
  • Low cost: Commodity HW with HD Webcams
  • Web native: simplicity of integration with other services
  • Flexibility: adapt to interface needs
  • Cloud-based: automatic updates and maintenance
  • Availability: 1B+ endpoints (6B by 2018)
  • Chrome, Firefox, Opera; browsers, mobile and desktop

Learn the raw API: WebRTC Codelab

NICTA WebRTC Meetup