[Ogg a11y] Fwd: [whatwg] Captions, Subtitles and the Video Element

Sun Feb 22 16:23:14 PST 2009

On Mon, Feb 23, 2009 at 11:05 AM, Conrad Parker <conrad at metadecks.org> wrote:
> 2009/2/22 Silvia Pfeiffer <silviapfeiffer1 at gmail.com>:
>> Yes, I have noticed that problem with subtitles. Fortunately, mplayer
>> and xine allow you to delay the subtitles in reference to video. I am
>> not sure yet how to support that in a Web browser... any ideas
>> welcome! Might need an additional javascript API function...
>
> If the video and subtitles are just data which may have skewed timing,
> then I don't see why javascript would be required to sync -- surely
> the browser could just offer delay/resync functions, just like mplayer
> and xine already do?

It is this kind of interface that I am talking about. The browser
needs to provide to javascript an API to allow it to delay the text in
relation to the video. Then the user interface can have a button or
two through which the user can call this API and get the browser to
adjust the synchronisation.

I have just added a set of example functions to the <itext> design at
http://wiki.xiph.org/index.php/Timed_Divs_HTML#Direct_linking_on_a_HTML5_page
.

There is also a function called delay(itextelement, seconds), which
would come in useful for resynching.

Please make further suggestions of necessary functions!

> What may be required is to treat the timelines of
> caption/transcription data separately from the timeline for
> presentational video overlays; ie. you'd want to fix the skew between
> the video and the captions, but you don't want to change the timing of
> other text like sports scores, weather info etc. that is part of the
> underlying presentation.

If it all comes within a Ogg file, the synchronisation should not be a
problem, since it is crafted to be in sync. The real problem is
out-of-band data, i.e. your srt files that are referenced through a
URL in an <itext> element.

Cheers,
Silvia.

>
> Conrad.
>
>> On Sun, Feb 22, 2009 at 3:34 PM, Danielle Clarneaux <danclar1 at msn.com> wrote:
>>> As a hearing impaired person advocating closed captioning on the Internet, I
>>> would like to offer one point that somone with normal hearing might not take
>>> into consideration. I'm encountering more and more CC on television that is
>>> not in sync with the voice of the person speaking, and this is highly
>>> distracting for a hearing impaired person since we rely so much on lip
>>> reading.
>>>
>>> I know that you all are working on much more technical aspects, but I just
>>> wanted to put this out there as you develop much awaited CC for the
>>> Internet!
>>>
>>>
>>> Danielle Clarneaux
>>> danclar1 at msn.com
>>>
>>>
>>>
>>>> Date: Sun, 22 Feb 2009 13:22:48 +1100
>>>> From: silviapfeiffer1 at gmail.com
>>>> To: accessibility at xiph.org
>>>> Subject: [Ogg a11y] Fwd: [whatwg] Captions, Subtitles and the Video
>>>> Element
>>>>
>>>> Hi all,
>>>>
>>>> Greg Millam from the Google Chrome team sent an extremely interesting
>>>> email to the WHATWG this week. It is quoted below FYI. We agree in
>>>> most aspects of the solution, though his naming and approach are
>>>> somewhat different. Have a read. Maybe we can encourage him to join
>>>> our discussions here and we can develop solution that Chrome, Firefox
>>>> and Opera can all agree on.
>>>>
>>>> Cheers,
>>>> Silvia.
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Greg Millam <millam at google.com>
>>>> Date: Fri, Feb 20, 2009 at 9:37 AM
>>>> Subject: [whatwg] Captions, Subtitles and the Video Element
>>>> To: whatwg at whatwg.org
>>>> Cc: Ken Harrenstien <klh at google.com>
>>>>
>>>>
>>>> Hi guys -
>>>>
>>>> I'm one of the main engineers responsible for captioning support on
>>>> YouTube, and I've joined the Chrome team at Google to attempt to help
>>>> drive video captions and subtitling forward: Both to implement support
>>>> in Chrome for it, and to push for HTML5 support for captions.
>>>>
>>>> In my following statements, I am working off of a search through the
>>>> mailing list and reading of the HTML 5 spec. Particularly where the
>>>> Video tag is concerned. If there are any factual errors, or I'm way
>>>> off, just point my way. All this is as far as I can discover.
>>>>
>>>> The current state of accessibility and captions in HTML5 has been
>>>> relegated to http://wiki.whatwg.org/wiki/Video_accessibility - a wiki
>>>> page with use cases, requirements, existing solutions, and an empty
>>>> "Proposed Solutions" category. I aim to fix that. My main goal here is
>>>> to prevent captioning from missing out on HTML5 and being dropped
>>>> "because we never got around to it". (a la HDMI)
>>>>
>>>> Here is my proposal:
>>>>
>>>> Use cases:
>>>> * Accessibility.
>>>> * Ability to audiences in other languages.
>>>>
>>>> Goals:
>>>> * Allow movie formats to include captioning support.
>>>> * Make it simple for an author to create and publish transcripts,
>>>> without requiring them to embed it into the movie.
>>>> * Make it simple for caption or subtitle tracks to be accessible.
>>>> * Allow full javascript control: List, add, delete, and create caption
>>>> tracks.
>>>> * Provide a required format to act as a baseline across all browsers.
>>>>
>>>> The current state of the <video> element includes support for defining
>>>> a source video file, local or remote. There is no method to define a
>>>> caption source or track.
>>>>
>>>> Proposed Solution:
>>>>
>>>> HTML5 / Page Author:
>>>> * Each video will have a list of zero or more Timed Text tracks.
>>>> * A track has three variables for selection: Type, Language, Name.
>>>> These can be null, except for name.
>>>> * Type is a string, and may be (but is not limited to): "Caption"
>>>> "Transcript" "Translation" "Subtitles", etc. Others can be defined by
>>>> the user (e.g: "Commentary" "User Comments").
>>>> * Language is a language code (en, es, pt_BR, etc)
>>>> * Name is a freeform text identifier. By default, "default" or
>>>> "caption". If a video file has multiple tracks, they are added as
>>>> "caption1" "caption2", etc.
>>>> * <video> . . . </video> is not necessarily a standalone tag. If the
>>>> author desires, they can add more elements to define tracks. Whether
>>>> this should be <caption type="format" src="..." media="caption"> or
>>>> <source type="timedtext/format" src="..."> can vary. (I prefer
>>>> <caption> as it's more explicit).
>>>> * <caption src="foo.srt" type="caption" language="en" name="default"
>>>> /> adds a new caption. <caption> is standalone.
>>>> * All timed text tracks encoded in the video file are added to the
>>>> list, as an implicit caption element.
>>>> * Caption tags, when displayed, count as <span
>>>> class="caption">...</span> unless they have style associated with them
>>>> (uncommon). So they can be tweaked via CSS. Whether by the author or
>>>> overridden by useragent.
>>>>
>>>> User Agent:
>>>> * Implements support for <caption> tag.
>>>> interface MediaCaptionElement : HTMLElement {
>>>> attribute DOMString src;
>>>> attribute DOMString format; // default: "auto".
>>>> attribute DOMString type;
>>>> attribute DOMString language;
>>>> attribute DOMString name;
>>>> attribute DOMBoolean enabled;
>>>> };
>>>> * Media elements now have a list of Captions associated with it.
>>>>
>>>> * Support for (at minimum) "Subrip" format. Subrip I choose here for
>>>> the same reason we picked it for YouTube: It's readable,
>>>> understandable, and simple. You can create one with your favorite
>>>> editor. Subrip has no style associated with individual captions, so
>>>> can be subject to CSS caption rules for "SPAN.caption"
>>>> * Support for other formats (608, 708, .ass, dfxp, etc) up to the
>>>> user agent. (But preferred!)
>>>>
>>>> Javascript:
>>>> * Media or Video elements now have additional features exposed via
>>>> javascript.
>>>> * getCaptionList(): returns an array of caption elements.
>>>> * addCaption({src:'',name:'',language:'',type:''}) - Adds a Caption
>>>> element.
>>>> * enableCaption(captionElement) - Enables a CaptionElement for
>>>> display. If captionElement is null, enable the first track in the
>>>> list.
>>>> * disableCaption(captionElement) - Disables CaptionElement for
>>>> display. If captionElement is null, disabled all tracks in the list.
>>>>
>>>> User Agent UI: (Only relevant if User Agent adds its own controls for
>>>> media):
>>>> * Must be able to enable caption Elements.
>>>> * Preferably by a button on the UI with either "CC" or a double
>>>> underscore (preferred).
>>>>
>>>> User Agent Context Menu:
>>>> * Must have captions, with a list to enable/disable.
>>>>
>>>> ...
>>>>
>>>> Well - That's a start, and that's what I'd like to see and implement
>>>> over the next several months. Input and discussion would be much
>>>> appreciated! If anybody here has worked on it, I'd also like to talk
>>>> to you.
>>>>
>>>> Thanks,
>>>>
>>>> - Greg Millam
>>>>
>>>> --
>>>>
>>>> Every time you give up a dream, a chicken stays on its side of the road.
>>>> _______________________________________________
>>>> Accessibility mailing list
>>>> Accessibility at xiph.org
>>>> http://lists.xiph.org/mailman/listinfo/accessibility
>>>
>>> ________________________________
>>> Windows Live™ Hotmail(R):…more than just e-mail. Check it out.
>> _______________________________________________
>> Accessibility mailing list
>> Accessibility at xiph.org
>> http://lists.xiph.org/mailman/listinfo/accessibility
>>
>>
>