UX Engine: Timeline

Functional Description

The timeline service is responsible for the temporal orchestration of a DMApp (within a single household). Think: the director of an orchestra that reads the score and cues the individual musicians to play their parts. At service startup it needs to be given the parameters needed to contact the layout service for this DMApp, the parameters it needs to get the score document (or possibly the document itself) and some way to subscribe to the timeline synchronisation service for this DMApp.

Verbs

createContext

Sent by the layout service to indicate a new context has been created. The layout service needs to at least pass two parameters to indicate how the timeline service can access the layout service and the synchronisation service for this context, for example layoutServiceUrl and syncServiceUrl.

loadDMApp

Sent by the layout service to indicate a new Distributed Media Application is starting.
A parameter (for example timelineDocumentUrl) needs to be passed to indicate the document describing the orchestration for this DMApp.

Issue: either the timeline starts playing straight away, or a second call startTimeline is needed.

It will now interpret the score (according to the current time updates provided to it by the sync service) and emit calls to the layout service to control starting and stopping of DMApp components (where media items such as specific images and such are considered a special case of a DMApp component).

stopTimeline

Informs the timeline service that this timeline has stopped playing. Note: unclear whether this call is needed.

timelineEvent

Informs the timeline service that eventIdentifier has occurred, so it can adjust its timeline accordingly. The intention of this call is to allow alternate paths through the timeline (think: interactive quizzes, or secondary content on different levels such as beginner/intermediate/advanced that the user can switch between, or that the DMApp can switch between based on a users answers to previous quiz questions or something), and there must be a way for the DMApp to signal to the timeline service that such an alternative path has to be selected.

Maybe this call needs a globalTimeSpecification if it can be scheduled for the future.

clockChanged

This is called to inform the timeline server of the current mapping between the per-context global clock and the wall clock. It has two parameters, wallClock (suggestion: let’s use a floating point number specifying seconds since the Unix Epoch, in UTC) and globalTimeSpecification.

Discussion

It is not yet clear whether we also need pause and resume (but if we need the functionlaity maybe clockChanged can be used for that, by telling the timeline service that the clock speed is now zero). It is also not yet clear whether we need seek (and, again, these may be implementable using clockChanged).

Verbs Needed from Components

Note: this section needs to go elsewhere (possibly both to layout service and app?) after editing. It is not about verbs provided by the timeline service, but about verbs needed by the timeline service. Either the verbs are provided by the components themselves, or they are provided by the layout service. In the first case, the components need to be addressable, in the second case the verbs need some componentIdentifier parameter.

initComponent

Initialize a component and assign parameters (media URL, etc) needed by a component.

Note: the need for the parameters is based on a model where the DMApp generally contains abstract components, such as “a video renderer”, and the actual video to use is encoded in the timeline document. An alternative model is that the DMApp itself already contains all the needed information, but this limits reuse of the DMApp (think: translating it into a different language).

Even if the component already knows all its parameters beforehand an initialize call is still needed: the intention is that it allows a component to do preparation work before it becomes visible and/or active. Think of things like prefetching media items and such.

startComponent

Starts the component at a given time globalTimeSpecification, at media position localTimeSpecification. Ideally, this time will be in the future when the call is issued, so the timeline service can issue a couple of these calls to different components and know they will all start at exactly the same time. If the time given is already in the past the component should start as soon as possible.

The localTimeSpecification specifies, for continuous media items, the position at which the media item should start playing back. This is needed so things like subtitles can be dynamically added and start playing back with the right synchronisation.

stopComponent

Stops the component at a given time globalTimeSpecification. See startComponent for timing issues.

modifyComponent

Modifies the internal (and possibly the external) state of a component at a given time globalTimeSpecification. Which bits of state can be modified remains to be seen (and probably depends on the component type), but one can think of media position, playback speed, etc. This call is needed so orchestrated changes (like a seek on the timeline, or jumping to alternative content) can be done in an orchestrated manner, under control of the timeline service.

Discussion

The order of the first three commands is always initComponent-startComponent-stopComponent.
This set of verbs presumes that componentIdentifiers are like xml-ids and long lived. The initComponent parameters may not be needed for static DMApps, but will be needed when dynamically inserted content is used (for later use cases). The intention of including the globalTimeSpecification in the calls is that these calls can be issues prematurely, so the receivers have a chance to prepare (for instance by loading the media items). If a globalTimeSpecification specifies a time in the past the action should be executed as soon as possible.

We may want to add an uninitComponent later, then the order of commands would be initComponent-startComponent-stopComponent-startComponent-stopComponent-…-uninitComponent. This would have the advantage that a component can be stopped but remain in their initialized state (with connections to media servers open, decoders allocated, etc) so that when things are dynamically hidden and shown again later there is less of a performance penalty. But whether we need this and whether it is useful depends on the API we select for migrating components in the layout service.

API Specification

To include specific data, communication substrate etc.

Here is an example of an API definition – feel free to use similar formatting:

dump

The receiver of this message is expected to dump (part of) its internal state to its logging output, for debugging purposes.

• Publisher: debugging tools
• Subscriber: all

Example:
{ "message" : "dump" ,
"what" : "random string that may influence what is dumped",
};