Should there be advisory markup attributes in order to describe
content even more precisely, e.g. dataRate?
It would be an optional attribute that, if present, would specify the minimum network connection
speed (in bits per second) required to use the media resource. The mechanism used to measure
bandwidth could determined by the user agent; so some implementations might wish to use a static setting
controlled by the user, and some might wish to use the average network throughput to the server hosting the media
resource. It would only be used for static fallback.
Retrieving a HTMLImageElement for the frame at a specific time from a
video would be very useful. Would this be API, e.g.
HTMLImageElement getFrameImage(DOMString time)?
or a special URI scheme?
It may be desirable to sync multiple video and audio elements, and have
them keep time with each other.
Wherever times appear in APIs or CSS properties, it may be convenient to get and set
times in other time formats. Formats that may be of particular interest include:
"HH:MM:SS.fraction" with hours and the fractional seconds optional
frame count index relative to start chapter name and/or index
In CSS, time formats could have their own function syntax (such as
clock("HH:MM:SS.fraction"), etc. In APIs, it would be possible to just accept strings, but
that doesn't let you get times in your format of choice. A possible solution is a MediaTime
interface which can convert to or from various time formats, and which is passed to and
returned from API methods that deal in times.
What does playbackRate do for audio? Should it be pitch-preserving? Should it
skip sections if you are going fast enough?
A movie is just a description of the multimedia presentation, the actual media sample
data is in tracks (or streams). Each track represents a sequence of renderable elements of a
uniform type, e.g. audio, video, text, etc. We should expose track information via the API
so developers can determine exactly what a media file contains and so they can enable and
disable individual tracks to control which take part in the presentation. Tracks contain
both metadata (eg. sample rate, sample size, bit depth, etc) and annotations (see notes on
media annotations below). We haven't designed an API for this yet. Perhaps track info and
metadata should be specified separately.
API to provide access to embedded metadata. In addition to metadata about media
characteristic (frame rate, data size, bit rate, etc), we should provide access to
annotation metadata. Annotations in MPEG-4 files can be in several different formats (eg.
3GPP, iTunes, QuickTime, ID3v2, etc), need not have unique names, and can be tagged with
language, so an API must allow all of them to be specified. Another form of interesting
metadata is chapter names and time ranges.
Should there be policies expressed on how to deal with graceful degradation if an
implementation is unable to support some aspect of a feature (e.g., negative playback
rates)? This might be a characteristic of the implementation as a whole or the
characteristic only when dealing with a particular media format (e.g., negative playback on
a RTP stream doesn't work even though for a .mov it could). Or should this (in)ability be
discoverable? Some clients that implement video decoding in hardware may not be able to
scale for instance.
How can a client discover the capabilities of the user agent (e.g., what codecs and
profiles/levels are supported)?
It might be useful in the API to get the current frame index, or go to a specific frame
(see time format discussion).
CSS Open Issues
<time> values only have second and millisecond unit identifiers. This will
make it inconvenient to represent time values in long duration media files because all
times must be converted to seconds. It may make more sense to use a time value
with a syntax like the SMIL
<clock-value> type, which allows units of hours, minutes, seconds, and milliseconds,
so longer times can be expressed without conversion.
It would be useful to extend CSS Media
Queries with a new media query feature to make
rules that are matched based on bandwidth. This will allow a content author to build
pages that select different CSS files based on the user's bandwidth as well as existing
media query features like screen size.