Synchronized Multimedia Modules
based upon SMIL 1.0
W3C Note 23 February 1999
CÔNG TY TNHH ĐẦU TƯ THƯƠNG MẠI VÀ DỊCH VỤ HỮU QUYẾT
Số 2, ngách 1/1 Phố Thiên Hiền, Phường Mỹ Đình 1, Quận Nam Từ Liêm, Thành phố Hà Nội.
MST: 0110557925
Status of this document
This Note was produced by the W3C SYMM Interest Group. It describes a snapshot of certain aspects of the discussions on future work on SYMM and SMIL. Following the approach spearheaded by the W3C HTML Working group, the Note describes how SMIL 1.0 functionality and future SYMM functionality can be represented as a set of markup modules. The intention was to study the feasibility of such an approach, and the details of the proposed modularisation are preliminary only. Please send comments to the public mailing list [email protected].
This document is a NOTE made available by W3C for discussion only. This indicates no endorsement of its content, nor that W3C has, is, or will be allocating any resources to the issues addressed by the NOTE.
Table of Contents
- Abstract
- 1. Introduction
- 2. Synchronized Multimedia Modules
- 3. Module Integration
- 4. Multimedia Profiles
- 5. Issues
- References
This document describes a proposed modularization of synchronized multimedia functionality, based upon the SMIL 1.0 Recommendation. The proposed modules complement HTML/CSS modules to support the definition of multimedia profiles that combine SYMM and HTML/CSS modules, as well as other XML-based languages. A set of example profiles for multimedia use is described. The document is intended to form a basis for work by a future SYMM Working Group.
The first W3C Working Group on Synchronized Multimedia (SYMM) developed SMIL - Synchronized Multimedia Integration Language. This XML-based language is used to express synchronization relationships among media elements. SMIL 1.0 documents describe multimedia presentations that can be played in SMIL-conformant viewers.
SMIL 1.0 provides limited means for integrating with HTML, CSS or other XML languages. This integration has been an area of considerable interest within the W3C as well as other external organizations (such as the ATSC T3/S17 specialist group). The W3C SYMM Interest Group has been tracking these issues in preparation for future work on the SMIL language..
The W3C HTML Working Group is working towards a modularization of HTML. The W3C CSS Working Group has had similar discussions. One reason that the HTML Working Group is pursuing modularization is to support the use of HTML in a wider range of applications. In support of this activity, the HTML Working Group and the SYMM Interest Group have been discussing strategies for integrating multimedia functionality into HTML. Several proposals have been submitted and are being discussed (HTML+TIME and BHTML). One approach to solving this problem is to modularize synchronized multimedia functionality, and make these modules available to other languages.
1.1. Goals and Requirements
There are four goals for modularizing synchronized multimedia functionality:
- G1: provide a means for the W3C and third parties to integrate timing models into other languages (such as HTML).
- G2: provide a means for the W3C to extend SMIL with new or optional features.
- G3: provide a means for third parties to extend SMIL with domain-specific features.
- G4: provide a means for third parties to integrate other languages into SMIL applications.
In essence, modularizing SMIL is the first step towards the integration of timing and other key aspects of SMIL with other Web languages. This modularization provides a means for other languages, such as HTML and an XML-based vector-graphics format, to benefit from timing and synchronization functionality while reusing the lessons learned and work embodied in SMIL. This becomes especially important for the development of cross-language tools which can readily transcode syntax but generally have a difficult time porting content to different timing and content models.
This Note describes the current approach to this problem by the SYMM Interest Group. The modules described herein satisfy the following design requirements:
- R1: modularize SMIL 1.0 without meaningful changes to the SMIL 1.0 content model.
- R2: modularize SMIL 1.0 without meaningful changes to the SMIL 1.0 timing model.
- R3: specify SMIL 1.0 modules that complement HTML modules.
- R4: adopt new W3C recommendations when appropriate and not in conflict with other requirements.
Several additional requirements will inform the work of a future SYMM working group:
- R5: All of the modules should have integrated support for the document object model. This facilitates additional control via script and custom user agents (e.g. for playback controls). Support for the respective object model is optional for all profiles. In designing the object model for facilitating script control, it should be investigated whether declarative solutions can also be provided, and, where possible, be specified.
- R6: Interfaces should be defined that provide means for integration between 3rd party player/renderers and browser engines. Time-based behaviors of browsers should also be available to 3rd party player/renderers. For example, applications may interface with a HTML renderer and with an independent audio and video renderer. The result should be tightly integrated multimedia presentations.
- R7: Where practical, modules should be defined isomorphic with modules from other W3C recommendations. This will assist module sharing across profiles. Targetting for isomorphic modules requires bilateral cooperation. Alignment of overlap in functionality concerns in particular the HTML module definitions.
- R8: It must be possible to define a profile that is 100% backwards compatible with SMIL 1.0.
- R9: For the purpose of integration with other profiles, modules may take another syntax. However, it is required that the semantics are maintained.
The purpose of the requirements is to ensure that the modularization makes no significant changes to the SMIL 1.0 language while laying a foundation for integration with HTML, CSS, and other Web Languages. This Note provides a discussion of current integration strategies and proposals.
The SYMM Interest Group recommends that the results presented in this Note form the basis for some of the work of a future SYMM Working Group.
1.2. Modules and Profiles
Modularization of synchronized multimedia functionality provides a building block for language, application, and platform developers. This Note recognizes that in addition to functional modules, there exists the notion of profiles and platforms.
For the purposes of this Note, we define the terms module, profile, and platform:
- module
- A module is a collection of elements. Each element is in one and only one module.
- profile
- A profile is a collection of modules particular to an application domain or language. For example: The SMIL profile corresponds to the collection of modules that make up the SMIL language. A enhanced television profile would correspond to the collection of modules for media-enhancement of broadcast television.
- platform
- A platform is a collection of profiles, non-Web technologies, policies, resource requirements, etc.
This Note focuses on modules and profiles, and is not concerned with specifying or characterizing platforms. Nevertheless, it is worth noting the distinction between a profile and a platform: a profile generally describes a document type and its usage, whereas a platform encompasses the entire environment. (The term "profile" as defined by the HTML WG is likely to bear a wider scope than used in this Note.)
This Note proposes a set of modules for SYMM functionality. The first section describes the modules associated with functionality defined in SMIL 1.0. A second section describes proposed extensions to SMIL 1.0.
2.1. SMIL 1.0 Modules
- Structure
- Includes the <smil>, <head> and <body> elements. These elements will be used in documents based upon traditional SMIL profiles.
-
- Meta
- Includes the <meta> element. The elements contain information describing the document, either to inform the human user or to assist some automation. The element will be used in documents based upon traditional SMIL profiles, possibly in concert with meta data modules available from other efforts.
-
- Layout
- Includes the <layout>, <root-layout> and <region> elements, and related attributes. This module supports the layout mechanism described in SMIL 1.0. These elements will all be used in documents based upon the SMIL profile.
-
- Timing and Synchronization markup
- This module includes timing structure (e.g. sequence and parallel timelines) and timing control properties (e.g. begin, end, and repeat) that describe the timing relationships among elements. This module also includes mechanisms to control the performed synchronization of the document (described in SMIL 1.0 as soft and hard).
This is separated from the media so that it can be applied to HTML and XML applications independent of support for media.
Note that in some integration profiles, the specific syntax for the markup may vary slightly with the profile (e.g. the SMIL 1.0 <seq> element may be represented as a sequence attribute if a CSS style-based integration is used). These differences notwithstanding, the module semantics and the timing model will be consistent across all profiles.
The specific syntax and mechanism of integration with HTML and XML applications is under review. -
- Media Elements
- This includes the media declaration elements <ref>, <animation>, <audio>, <img>, <video>, <text> and <textstream>. The media declaration may relate to the object element in HTML (this needs attention).
-
- Linking
- Includes the <a> and <anchor> elements.
The relation of this module to other linking modules, particularly XLink, XPointer and HTML linking, is under review. For example when integrating with HTML, timing applied to the (HTML) <a> and <area> elements could provide much or all of the SMIL 1.0 defined functionality. -
- Content control
- This includes the <switch> element and test attributes introduced in SMIL 1.0. This is of general utility to authors, independent of the multimedia functionality.
2.2. Proposed Extensions
- Layout (extensions)
- Additional functionality will likely be added to this module as part of ongoing work. Relationship of this Layout module to other Layout mechanisms, including CSS, is to be considered.
-
- Timing and Synchronization markup (extensions)
- Additional functionality will likely be added to the module as part of ongoing work. Proposed extensions include event-based timing constructs, e.g. to support user-interaction and stream events, and more detailed mechanisms to control the performed synchronization of the document.
-
- Transitions
- This module includes new functionality to support transition effects on elements. These may be media elements, or they may be traditional HTML (or XML) elements. A CSS-style syntax under consideration might represent transitions as in this HTML-based example:
<html> <head> <title>Powerpoint Animation in HTML</title> <style> p.caption { transition-style: wipe; font-size: big } li { transition-style: dissolve } </style> </head> <body> <p class="caption">Multimedia Requirements</p> <ul> <li>Timing the Display of Bullet Items</li> <li>Transition Effects</li> </ul> </body> </html> Specifics of the module syntax is under review. There is a desire to present a simple syntax for typical use-cases, and also to support more complex functionality by modeling transitions as filters with timing support. The various proposals will be considered and refined as work proceeds.
- Animation
- This module includes functionality to support common simple animation like motion paths. Where Transitions apply rendering filters, Animation behaviors manipulate Object Model properties like position, color, size or rotation (e.g. in an XML-based vector-graphics format).
-
- Resource Management
- This module includes support for resource (e.g. bandwidth) management hints. These can facilitate user agent optimizations, and can control the degradation strategy of the user agents. For example, the author may assign priorities to elements to inform a user-agent that will dynamically balance network bandwidth. Other proposed markup includes hints for cueing or preloading media and a means of specifying that an individual media element is delivered as a substream within an aggregated media stream (for large streamed presentations).
The HTML Working Group is still reviewing formal mechanisms to define or specify profiles. Several approaches are under consideration including DTDs, XML Schemas, and RDF-based mechanisms. The SYMM Activity will provide associated requirements and use-cases to the HTML Working Group, and will follow the recommendation of the HTML Working Group for profile specification.
Several mechanisms for integrating SYMM functionality with other languages (especially HTML) have been discussed. It is possible that a given module may be represented with different syntax depending upon the profile and integration. Nevertheless, the SYMM modules will define consistent semantics for all profiles, regardless of syntax variants.
The current variants and issues for integrating synchronized multimedia functionality with HTML and XML languages are:
- Defining tagsets for SMIL elements. This is a yes/no choice, and combines with the others. I.e. A profile can either:
- Define XML tags for SMIL elements together with SMIL attributes (as specified with one of the mechanisms below), or
- Define only attributes. All SMIL elements would be represented as attributes (of some sort) applied to HTML elements.
At issue in particular are the SMIL timing structure elements <par> and <seq>, which could be expressed as an attribute on HTML (or XML) container elements. While this is expressed as a dichotomy, it really defines a range of possibilities. Profiles may map some elements to attributes and leave others as elements.
- Attribute Syntax variants
- XML Namespace qualified attributes
- This has the advantage of general applicability to any XML language. It has the disadvantage that it is impossible to set the value of an XMLNS attribute via CSS stylesheets without defining a CSS property that corresponds to the XMLNS attribute.
- CSS properties
- This has the advantage that stylesheets can be used to define timing, transitions, etc. It the disadvantage that it requires CSS, which may not be appropriate for some XML languages or for some profiles and platforms.
- XSL properties
- This is a variant on CSS properties, but would support a broader range of XML applications.
A range of profiles have been discussed that would make use of different sets of modules. Below is an initial list of profiles that could integrate SYMM and HTML functionality in different ways. This is not intended to be a final statement, but rather an indication of how modules would be used. It provides a backdrop for the discussion of modularization.
In addition, some specific proposals can be described in terms of the relevant modules:
- SMIL 1.0 would be defined as the 1.0 versions of the Structure, Meta, Layout , Timing, Media, Linking, and Content control modules. SMIL 2.0 would include all of the new versions of these modules, as well as some or all of the new modules proposed.
- HTML+TIME proposes a profile that extends an HTML 4.0 profile to include Timing, Media, Linking and Content control modules.
- BHTML proposes a profile that similarly extends HTML to include Timing, Media and Linking, as well as Transition support.
4.1. Example Profiles
- Leightweight presentations
- A basic profile would handle simple presentations, supporting timing of text content. The simplest version of this could be used to sequence stock quotes or headlines on restricted platforms like palmtop devices or smart phones. The Mobile Access group is looking at related profiles. An HTML-based profile would combine HTML modules (probably a subset of HTML) with the timing module. It might include the transitions as well.
-
- Timeline-Centric Multimedia presentations
- This describes the profile associated most closely with SMIL 1.0. This does not integrate HTML, and uses SMIL layout rather than the flow layout model associated with HTML . Typical use-cases for this profile are multimedia presentations that include time-based and streaming media. This would include most or all of the SMIL modules.
-
- HTML integrated Multimedia presentations
- This is a variant on the Traditional Multimedia profile that mixes multimedia and HTML. It mixes HTML layout (i.e. flow layout) and CSS positioning. Use-cases for this profile are presentations that use HTML as a media type in concert with time-based and streaming media (e.g. PowerPoint-like presentations with time-based media). This profile would include (most or all of) the HTML modules, the Timing, Media, Transition, Animation, Resource Management and Content control modules (i.e. all but the Structure, Meta and Layout modules). The linking functionality would overlap between the Linking module and a similar HTML module.
-
- Web Enhancement of Broadcast, On-Demand Video and DVD
- This describes support for web presentations tied to broadcast and/or on demand media. The primary media will often define the main timeline. The main additional requirements here are for stream event support and the ability to tie the page clock to the primary media. Stream event support requires event-based timing of elements (and timelines), and a means of associating events with document-external (stream) triggers. E.g. where the stream could be a TV-broadcast, it would be possible to refer to the events by referencing the TV broadcast stream or player-object. It must be possible to ensure that the multimedia (enhancement) timeline is strictly synced to or driven by the primary (broadcast) media.
Note that where "webcasting" may associate Web-pages to a TV-broadcast, in "enhanced television" additional media form an integral part with the main video presentation, providing enhancement of and interaction with the program.
While the typical use-cases may be dominated with the primary media (e.g. streaming video broadcast or DVD video), the authoring model is still closely tied to the Web and HTML. This profile would likely use the same modules as the HTML integrated Multimedia presentations profile, but with the additional timing and synchronization requirements described.
Syntax
The details of the elements and properties still need to be worked out. In particular, there are open issues on the syntax for timing markup, and the mechanism for integration with HTML and XML applications, and stylesheet tools like CSS and XSL.
Accessibility syntax
An additional issue that must be coordinated with representatives of the Web Accessibility Initiative (WAI) concerns accessibility support. This includes elements and attributes which assist the accessibility of content declared in the presentation. In SMIL 1.0, this concerns the title, alt and longdesc attributes.
It is not clear whether this should be an additional module, or additional support in specific modules. In particular, the intent is not to define a new set of Accessibility mechanisms, but to integrate current work of expert groups in this area.
- Broadcast HTML
- A Modular Hypertext Markup Language for Broadcast Applications. HTML working group contributed paper. Ted Wugofski. This is available at: http://toocan.philabs.research.philips.com/misc/atsc/bhtml/
- HTML+TIME
- Timed Interactive Multimedia Extensions for HTML (HTML+TIME) Note 18 September 1998, Patrick Schmitz, Jin Yu, Peter Santangeli. This is available at: http://www.w3.org/TR/NOTE-HTMLplusTIME
- Reformulating HTML in XML
- W3C Working Draft 5 December 1998, Dave Raggett, Frank Boumphrey, Murray Altheim, Ted Wugofski. This is available at: http://www.w3.org/TR/1998/WD-html-in-xml-19981205
- SMIL 1.0
- Synchronized Multimedia Integration Language (SMIL) 1.0 Specification 15 June 1998, Philipp Hoschka. This is available at: http://www.w3.org/TR/REC-smil