Cuepoint functionality

[pablo]

Proposal to include the ability to add cuepoints to playlist items, which would be visible in the controlbar, and would result in an event being fired when they are played.

If possible, we should conform this mechanism to the  track element (should that move forward swiftly).

Cases

Here's a couple of applications for cuepoints:

• subtitles, captions, karaoke (like DVD)
• chapter or slide markers (like Hulu)
• annotations/descriptions (like Youtube)
• metadata (programmatical 'custom' data)

Note we are only talking about the mechanism to add these cuepoints to the player. The logic and UI to process and/or display these cues is not part of this mechanism. However, on top of this mechanism one could build something like:

• A multilanguage subtitles plugin. This plugin would listen to cuepoint events and display the description of the event at the bottom of the screen.
• A powerpoint slide-sync plugin. This plugin would listen to cuepoint events and display the thumbnail and description of the event in a separate area, next to the video. Markers in the controlbar show the slide transitions. When hovering a marker, a thumb of the slide is shown.
• A mid-roll advertisement plugin. This plugin would listen to cuepoint events to pause the main video and play a midroll advertisement. When the ad is completed, the video is resumed. Markers in the controlbar show the advertisement. When hovering a marker, the description (e.g. Old Spice Advertisement) is displayed.
• A user annotation plugin. This plugin would listen to cuepoint events to display info bullets on top of the video.

Data

A cuepoint would contain the following data:

• type: description of the type of event (e.g. caption or powerpoint). Has no programmatical consequences - just for keeping track.
• position: position of the cuepoint in the mediafile, in seconds.
• marker: Add marker to the controlbar. Can be true or false (the default)
• text: description of the cuepoint, as (HTML) text. Displayed in the controlbar if marker==true.
• thumb: URL to a thumbnail of this cuepoint. Displayed in the controlbar if marker==true.
• data: Data of the cuepoint, for the plugin that uses it. For example, the text for a caption, or the position, color and contents of an annotation balloon.

API

The API for managing cuepoints is:

• player.addCuepoint(playlistItem, {cuepoint});
• player.removeCuepoint(playlistItem, {cuepoint});
• player.getCuepoints(playlistItem);

Cuepoints can also be added to a video (playlistitem) by means of an XML file, loaded with the flashvar or playlist item element cuepoints. For example:

<cuepoints>
  <cuepoint>
    <type>chapter</type>
    <position>23</position>
    <marker>true</marker>
    <text>Chapter 1: Takeoff</text>
    <thumb>http://www.mysitecom/chapter1.png</thumb>
  <cuepoint>
</cuepoints>

Event

Plugins and tools can get cuepoint data to listen to a new event. For example:

CuepointEvent..JWPLAYER_CUEPOINT::

Note that, when a user jumps beyond a cuepoint, the cue will be fired. For example:

• Cuepoint is set at 05:00.
• User starts video.
• User jumps to 6:00.
• Cuepoint event is fired.
• User jumps to 5:30.
• User jumps to 4:30.
• User jumps to 5:30.
• Cuepoint event is fired.

[pablo]

This could be an important improvement for the player, but we should probably put some more thought into how this should work before implementing.

[jeroen]

Why this more extensive description? Just to get a clearer sense of what such a cuepoint API would comprise of. It's immedately clear that this API is not easily merged with a mechanism for parsing / displaying captions (the captions plugin), since:

• Captions XML file formats are insufficient for cuepoints.
• Captions work different from cuepoints (in + out point)
• The cuepoint interface would not contain any UI for rendering captions (think all the stuff around multilanguage, styling etc).

[jeroen]

Feedback from Yannick:

• Have an icon per cue point. Something 16x16 for instance, to be displayed on the control bar.
• Cue point types: I can think of 3 types: a region, a chapter delimiter and an instant event. A chapter would separate the whole stream into different parts - like in your JS demo. A region would be a sub-part of the video - to highlight something that lasts for a while. And an instant event would be a simple point in the stream - to highlight something like a goal in a football match, or a crash in a race for instance. Only the region would have a duration.

[pablo]

Milestone Player 5.7 deleted