HbbTV – 2-IMMERSE

MediaSync: a new handbook

PABLO CESAR (above, right, with co-editor Jack Jensen) from 2-IMMERSE partner CWI, Amsterdam, celebrates the publication of a new edited collection from Springer:

In March 2015, Springer approached my colleagues and me because of our workshop on Media Synchronization at the ACM International Conference on Interactive Experiences for TV and Online Video (ACM TVX):

I do hope you will forgive me for contacting you out of the blue. The topic area of the workshop looks really interesting and I wonder whether you might want to consider using the material discussed as the basis for an edited book.”

I thought that it was the right time, and the right publisher, to edit such a book. A scholar or practitioner interested in media synchronization needed to find their way from a set of scattered works. Given the number of disciplines considering this research area (from human factors to networking and from multimedia systems to document engineering), not all the works were consistent in terms of concepts, definitions and assumptions. I believed an approachable overview was needed, covering the broad media synchronization area from different angles and perspectives.

After three years, tons of hours, thousands of emails, several delays, and some frustration, the book MediaSync: Handbook on Multimedia Synchronization was in my mailbox. The result: over 60 authors, 23 chapters, 679 pages, 241 illustrations, 193 colour illustrations, 60 authors; a foreword by Ralf Steinmetz and an afterword by Dick Bulterman.

Some of the benefits of the volume include:

A unique reference and perfect companion book about the fascinating media synchronization (mediasync) research area
Provides a comprehensive overview of the mediasync research area from complementary perspectives, such as theoretical foundations, available technologies, use cases, standardization efforts and experimental studies
Paves the way for overcoming remaining challenges towards the deployment of next-generation multimedia systems
Includes chapters from the most representative and influential experts in this area

Photograph of cover of MediaSync Handbook

Summary

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users’ perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences).

Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives.

Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences.

Open-source DVB CSS libraries available

MATT HAMMOND, Lead Research Engineer, BBC R&D writes:

2-IMMERSE is seeking to enable the easy creation of multi-screen experiences. In doing so, one of the aims is to leave a legacy of re-usable technology solutions to the technical challenges encountered.

One such technical challenge is synchronising videos playing on different devices. Keeping those videos in-sync helps create the impression of a single unified experience. 2-IMMERSE has adopted a solution defined by DVB, known as DVB Companion Screens and streams (DVB CSS). This same solution has also been adopted by the HbbTV association which defines the interactive capabilities of TVs in many countries in Europe and beyond.

The BBC has previously developed an implementation of DVB-CSS in the Python programming language (pydvbcss). Now, working as part of 2-IMMERSE, they have developed a new version in the JavaScript programming language that is used across the 2-IMMERSE service prototypes.

This new version has now also been open-sourced, making it available for others to use. It takes the form of two software libraries that are available on GitHub and via the popular npm package manager:

The dvbcss-clocks library (on GitHub and via npm) is a powerful way to represent timing and timing relationships.
The dvbcss-protocols library (on GitHub and via npm) implements the protocols defined by DVB CSS to synchronise. It uses dvbcss-clocks to model the timing relationships between the devices being synchronised.

2-IMMERSE hopes that these libraries will be of use to anyone wanting to understand and use DVB CSS technologies in the future.

HbbTV 2: a note on the state of play

MICHAEL PROBST from IRT writes:

Hybrid Broadcast Broadband TV (HbbTV), as Wikipedia details, is both an industry standard (ETSI TS 102 796^[1]) and promotional initiative for hybrid digital TV to harmonise the broadcast, IPTV, and broadband delivery of entertainment to the end consumer through connected TVs (smart TVs) and set-top boxes.

The latest release of the HbbTV specification was released about a year and a half ago. From a public perspective, not much has happened since then, as it is still not possible to purchase a HbbTV 2-enabled TV. But in fact, a great deal has been happening “behind the curtains” and HbbTV 2 is evolving steadily. In this post we highlight some of the latest developments in terms of implementations and services.

Let’s have a short look at the new elements in HbbTV 2:

Support for HTML5 including the Media Elements replacing the “unloved” XHTML 1.0
Fancy UIs with CSS3 Animations and Transitions, but also downloadable fonts, e.g. to support languages with exotic characters.
Enabling closed subtitles for all broadband delivered media.
Companion screen: discovery of TVs and launching HbbTV apps; discovery of special (manufacturer specific) launcher apps and launch of mobile apps; communication between HbbTV apps and mobile apps
Media Synchronisation: The most complex feature allows to play broadcast on TV with content from the Internet in sync either also on TV or on the companion device.

HbbTV 1.5 was a relatively small update to the first release and included only features with urgent market needs, and hence it was implemented and deployed rather quickly. In Germany we can assume that all new TVs sold in 2017 implement 1.5. The market share of these devices, in relation to all HbbTV TVs, depends to a degree on who you ask, but now in summer 2017 it is quite likely 50%.

Once HbbTV 2 is deployed Germany’s broadcasters will have to deal with a large legacy of HbbTV 1.X devices. Other countries have the advantage of starting with HbbTV 2 but will have to migrate from former platforms like MHEG-5 (UK) and DVB-MHP (Italy). The UK industry has chosen to launch HbbTV with a sub-profile (Freeview Play), that does not include companion screen and media sync features, but Italy could be the first country where deployed TVs have a full HbbTV 2 implementation.

The HbbTV consortium is still very active and working on a number of different topics to allow HbbTV to find new markets:

The test suite already supports a first set of the HbbTV 2 test cases, and the HbbTV testing group is working hard to complete it.
ADB (Application discovery via broadband) is a specification that will make HbbTV services available in broadcast networks where operators block or do not care about HbbTV. A second version of this spec will allow scenarios were people to have to use STBs of the network operator, e.g. by employing watermarks.
IPTV: this spec addresses specific issues to use HbbTV in IPTV networks, i.e. where they differ from classical broadcast networks
Operator apps: not yet released, but this specification will define a complete new class of applications suitable for broadcast network operators.

HbbTV 2 in action

Several companies have shown prototypes over the last years including the BBC and IRT who are partners in the 2-IMMERSE project.

In 2015 IRT presented a first HbbTV 2 showcase – live streaming of MPEG DASH with EBU-TT-D subtitles – with several partners including both manufacturers of streaming encoders and TVs, as well as a CDN provider and content provider. As a result of the success of this activity we now see first live streaming services with MPEG-DASH and subtitles offered by ARD broadcasters. More information on this can be found here.

Also in 2015 IRT cooperated with ARD Mediathek – a service providing catch-up TV – to enable their mobile application to cast videos on HbbTV 2 TVs. The application has been very useful for testing HbbTV 2.0 features for automatic discovery and application launch with a large number of TV manufacturers and the next step will be that the function is integrated into the end-user version of the application. Further information here.

The concept of HbbTV 2 Media Synchronisation is largely based on contributions from the BBC. TNO and IRT also supported it as part of their cooperation in the EU-funded hbb-next project. The BBC have done some early HbbTV 2 implementations using their own TV emulator and have released libraries and tools as open source on github.com. At TV Connect 2017 they showcased synchronised playback of broadband media on the companion device with a broadcast service on TV in cooperation with Opera TV.

If you have not had the chance to see any of these demos, there is another one at IFA Berlin and IBC Amsterdam, both taking place in September.

The HbbTV 2 demos of IRT this year will focus on media synchronisation for broadcast services, like broadband delivered foreign language audio tracks. As a study for new types of devices emerging, IRT has implemented a companion application for Microsoft’s Hololens, that will show additional synchronised video feeds alongside the real picture of an HbbTV 2 TV set.

At the IFA IRT demos can be seen at the booth of ARD digital in hall 2.2/103 (look for the IRT table at the back of the booth). And at IBC IRT’s stand is in hall 10.F51 (in a corner of hall 10). We look forward to seeing you there.