Preflight

Webkit
If using Google Chrome, you will likely need the Dev channel to see all of the functionality in this presentation. If you are using Safari, you will likely need a nightly build of WebKit in order to see all of the functionality in this presentation.
Mozilla
You are running a Mozilla browser. While such browsers generally have excellent support for HTML5 features, this presentation has only been tested using WebKit browsers such as Google Chrome or Safari. You should still be able to navigate the slides by using left/right arrow keys, but will currently see display errors with regard to the 3d rendering of the slides and some demo content.
Other browser
You are running a browser that has not been tested with this presentation. You may not be able to run some or all of the samples listed here. While we want to add support for as many browsers as possible, currently we only support WebKit-based browsers such as Google Chrome or Safari.
Media Source API: supported
Media Source API: not supported
Your browser does not support the Media Source API. If using Google Chrome 17+, you need to enable the API in about:flags.
If things look good, press → to move on.
Welcome! (This field is for presenter notes and commentary.)
Press:
  • Space or ← / → to move around
  • Ctrl/Command / – or + to zoom in and out if slides don’t fit
  • N to show/hide speaker notes
  • H to highlight important elements in code snippets

New HTML5 video technologies for the future of TV

Silvia Pfeiffer
3rd International Workshop on Future Television
3rd July 2012
14:45 - 15:30

Slide Template by HTML5 WOW
Who am I?

@silviapfeiffer
  • W3C and WHATWG video standards
  • HTML5 Accessibility Task Force
  • Google video accessibility contractor
  • Mozilla video accessibility contractor
  • Video research at CSIRO

The future of TV is the Internet

Young generation does not watch TV
  • discovery: by search, recommended by friends, similarity
  • interactivity: subscription, pause/rewind
  • preference: short-form
  • sharing: run skype in parallel, IM
  • multi-tasking: watch during homework
  • publish: anyone can be a producer
Example web series: SYNC

Episodes of 10min duration, released once a month.

Example web series: Video Game High School

Episodes of 10min duration, released once a week.

HTML5 is catching up to where TV was 20 years ago

HTML5 video technologies
  • publishing: audio and video element
  • accessibility: captions, audio descriptions, sign language video
  • navigation: chapters, media fragment URIs, interactive transcripts
  • live streaming: HTTP adaptive streaming
  • device/network adaptability: HTTP adaptive streaming
  • internationalization: multitrack audio
  • multi-angle views: multitrack video
  • live interviews: WebRTC real-time communication
  • content protection: DRM APIs

Publishing

Anyone can be a publisher
<video controls poster="nyan_cat.png">
  <source src="nyan_cat.mp4" type="video/mp4">
  <source src="nyan_cat.webm" type="video/webm">
</video>
Control: Media and the JavaScript API
Producing: anyone can create a channel
<video controls poster="image.png"></video>

var playlist = ["video1", "video2", "video3"];

video.addEventListener("ended", loadNext, false);
function loadNext() {
  video.src = nextVideo();
  video.load();
  video.play();
}

Accessibility

Accessibility: captions and subtitles
<video controls poster="video.png">
  <source src="video.mp4" type="video/mp4">
  <source src="video.webm" type="video/webm">

  <track kind="captions"  srclang="en" src="video_cc_en.vtt">

  <track kind="subtitles" srclang="de" src="video_sub_de.vtt">

</video>

Associate caption files with a video through markup.

Ally: WebVTT for caption authoring

Example: arduino-en.vtt

WEBVTT

1
00:00:13.000 --> 00:00:16.100
I heard about this <c.red.caps>arduino</c> project,
and I saw it online - 

2
00:00:16.100 --> 00:00:20.100
- and I said '<b>Wow!</b> a lot of people are starting to talk about this.
I should <i>check it out</i>!'
Ally: captions and subtitles
Ally: sign language video
<video controls mediagroup="elephant_sign">
	<source src="videos/elephant.webm" type="video/webm">
	<source src="videos/elephant.mp4" type="video/mp4">
</video>
<video mediagroup="elephant_sign">
	<source src="videos/elephant.sasl.webm" type="video/webm">
	<source src="videos/elephant.sasl.mp4" type="video/mp4">
</video>

Also works for dubbing and multi-view video.

Ally: audio descriptions
<video controls poster="video.png">
  <source src="video.mp4" type="video/mp4">
  <source src="video.webm" type="video/webm">

  <track kind="descriptions" srclang="en" src="video_desc_en.vtt">

</video>
WebVTT descriptions in HTML5

Turn on ChromeVox.

Navigation

Chapters and interactive transcripts
Navigation: Media fragment URIs
<video controls>
	<source src="videos/Big_Buck_Bunny_extract.webm#t=30">
	<source src="videos/Big_Buck_Bunny_extract.mp4#t=30">
</video>

HTTP adaptive streaming

Live streaming with HTTP Adaptive Streaming

Extends the audio and video interfaces

partial interface HTMLMediaElement {
  // URL passed to src attribute to enable the media source logic.
  readonly attribute [URL] DOMString webkitMediaSourceURL;

  // append bytes from webkitMediaSourceURL
  bool webkitSourceAppend(in Uint8Array data);
};
Example byte range file
{"total_size": 2348204,
 "height"    : 270, 
 "width"     : 480, 
 "duration"  : 33003.0,
 "clusters"  :
     [{"timecode":     0, "offset":    4444},
      {"timecode":   315, "offset":   10989},
      {"timecode":  4309, "offset":  222621},
      ...
      {"timecode": 32298, "offset": 2336319}], 
  "filename" : "test.webm"}

More details

Experimental Implementation in Chrome 17+

Activate through chrome://flags/

Video interviews / conferencing

Requirements for peer-to-peer video
  • access to local devices: camera, microphone
    • navigator.getUserMedia()
  • display a/v from local or remote peer
    • createObjectURL
  • connect to remote peers
    • PeerConnection API

Plugin-free video conferencing in the browser!

Access local device and display: getUserMedia(), createObjectURL
<video id="sourcevid" autoplay></video>
<script>
  var video = document.getElementById('sourcevid');
  navigator.getUserMedia('video', success, error);
  function success(stream) {
      video.src = window.URL.createObjectURL(stream);
  }
</script>
Cool things to do with getUserMedia()

Live video effects

Exploding Video

Xylophone

Add the remote stream
<video id="remotevid" autoplay></video>
var remotevid = document.getElementById('remotevid');

peerConn = new webkitPeerConnection("TURN localhost:12345", onSignal);
peerConn.onaddstream = onRemoteStreamAdded;
peerConn.addstream = localStream;

function onSignal(message) {
    sendSignalling(message);
}
function onRemoteStreamAdded(event) {
    remoteVideo.src = window.URL.createObjectURL(event.stream);
}

Demo: WebSocket Server, 2 Clients

Cool things to do with PeerConnection

Video Conference

WebMeeting

We Blend

Puzzlible

HTML5 video shines with other HTML5 features

Enhanced application example: GPS video tracking

Opportunities in interactivity
  • context-aware advertising
  • more information, e.g. news
  • immediate purchase
  • user choice of story line
  • new content: games

The technology is there - how will we map the new world?

Go out and play!

Silvia Pfeiffer
blog.gingertech.net
silviapfeiffer1@gmail.com
@silviapfeiffer
Slides: TBA