Draft V2 Working Document:
Architecture of the
Universal Remote Console Specification (AIAP-URC) of
the Alternate Interface Access Protocol (AIAP)
Version 1.14, 1011/2321/2002
Version 1.1 4 describes
has
been approved by INCITS V2 for external releasethe decisions of the
V2a telecon 10/22/2002.
WARNING
This document is a draft working paper of INCITS/V2, the Information Technology Access Interfaces Technical Committee of the InterNational Committee for Information Technology Standards. It does not represent an official position, recommendation, specification, or standard of INCITS/V2, INCITS, or ANSI. It is subject to change or withdrawal, at any time, without notice. Any implementation, rendition, or model based on the content of this document is done at the risk of the implementer.
This document gives an overview of the architecture developed to meet the requirements of the ‘Universal Remote Console’ (URC) scenario within the Alternate Interface Access Protocol (AIAP). V2 is currently developing a set of detailed specification documents for this architecture, which will replace version 5.0 and all earlier versions of the working documents titled ”V2 Working Document Universal Remote Console Specification (AIAP-URC) of the Alternate Interface Access Protocol (AIAP)”, and the associated open and closed issue lists.
ALL RIGHTS RESERVED. This document is provided for information and comment only. Comments on this document should be sent to info-v2@nist.gov. Reproduction or redistribution of this document for information or comment is authorized, only if the entire document, including this notice, is provided. No other use of this document is authorized without written permission from INCITS/V2.
© 1998, 1999, 2000, 2001, 2002 INCITS
Preface
AIAP
The Alternative Interface Access Protocol (AIAP) is a set of standards for the discovery, selection, configuration, and operation of user interfaces and options. These standards could apply to personal devices, stand-alone or networked systems, and services or devices (targets). The AIAP conveys information about user interface content and functionality; user preferences; target capabilities; and user commands. It allows alternative interfaces to be accommodated or constructed, in real-time if necessary, to provide fundamental access to computing services and information regardless of any limitation of the user.
Among the ways that AIAP is currently envisioned to achieve this, are:
1. By using an alternate user interface component instead of the native user interface component.
2. By allowing a person to use a complete alternate user interface (which includes its own alternate input, control and display mechanisms) instead of the native input, control and display mechanisms on the target (a ‘Universal Remote Console’).
3. By allowing the user to cause their characteristics or user interface preferences to be communicated to the target (either directly or by providing a code which the target uses to look up the user preference or characteristics) where the target changes its own user interface behavior based on the user preferences or needs.
4. By allowing the user to cause new user interface software to be determined and downloaded onto the target directly or indirectly.
AIAP-URC
The purpose of the Universal Remote Console Specification of the Alternative Interface Access Protocol (AIAP-URC) is to address the second of these four scenarios. That is, to define a standard interconnection protocol that allows users to control a mass-market device/service (target) from another device. A key purpose of this specification is to address the needs of people with disabilities to control mass-market products from special Assistive Technologies (AT) (such as is called for in recent government and industry initiatives).
The user interacts with the target through the Universal Remote Console (URC). The URC may be a dedicated device, but will more often be a feature running on a computer, a cell phone, an Assistive Technology, or other device. (This functionality is also associated with the term “Accessor”, although accessor is a more general term that also includes alternate interface components).
The key to this approach is that the target projects a presentation-independent version of its user interface using the AIAP-URC specified XML-based language to URC devices connected to it via a network protocol. This presentation-independent version of the user interface must contain no assumptions as to the presentation mode (visual, auditory, or tactile) of the user interface. The AIAP standard defines the requirements for the underlying network to facilitate the interaction between the URC and the target. The URC software takes the presentation-independent user interface (UI) and renders it as a concrete UI on the user’s remote console device. The concrete UI may be visual, speech-based, braille-based, or in some other form.
The burden on the target manufacturer is relatively light if their product is network controllable: The manufacturer just has to commission a presentation-independent user interface for their product. This one UI can then accommodate the needs of a wide range of users with a variety of devices chosen to meet their preferences and abilities. Examples include someone watching television and using a URC as a “remote control” to their entertainment center; or someone turning the home security system on or off from their bedroom or from their office; or someone with a disability using a URC as an alternate interface to a product they cannot otherwise access.
If the manufacturer wishes to provide customized concrete UIs to optimize aesthetics and usability for certain classes of users or devices, the manufacturer may provide a concrete UI in addition to the presentation-independent UI.
The “Universal” in “Universal Remote Console” refers to the fact that the user’s URC will work with all targets whose abstract UI complies with the AIAP-URC specification.
The structure of a V2 user interface definition is intended
to facilitate the use of intelligent software agents to control services and
devices in the future.
Table of Contents
Table of Contents
Terms, Definitions, Acronyms and Abbreviated Terms
Alternate/Abstract Interface Markup Language (AAIML)
Presentation-Independent Template
Universal Remote Console (URC)
User Interface Implementation Description (UIID)
Components of a Target’s Abstract User Interface
User Interface Implementation Descriptions (UIIDs)
Presentation-Independent Template
User Interface Instances and Instantiations
Terms, Definitions, Acronyms and Abbreviated Terms
AAIML
Acronym for Alternate/Abstract Interface Markup Language.
Abstract interactor
An interactor that describes the selection, input, or output for a user interaction, without constraining the concrete form of the interaction.
AIAP
Acronym for Alternate Interface Access Protocol.
AIAP-URC
Acronym for Alternate Interface Access Protocol Universal Remote Console.
Alternate/Abstract Interface Markup Language (AAIML)
The Alternate & Abstract Interface Markup Language (AAIML) is a vehicle by which a target conveys an abstract user interface description to a URC in the control phase, i.e. after a session has been opened between the URC and the target. The abstract UI description is presentation independent and must include all features and functions the target provides via its default (built-in) user interface.
Concrete
interactor
An interactor that describes the selection, input, or output for a user interaction, and includes information on the visual or non-visual realization of that interaction, for example a listbox or a particular speech grammar.
Control Phase
The control phase is the time period in the URC-target communication exchange when the URC controls the target via AAIML.
Discovery Phase
The discovery phase initializes the URC to locate and identify all available targets.
Interactor
An abstract or concrete user interface element that describes a choice for the user to make, some input to obtain from the user, or some output to convey to the user.
Presentation-Independent Template
A form of UIID. It describes a mapping from a user interface socket to a structured set of abstract interactors. This mapping provides access to all of the commands and readable data points within the user interface socket.
Session
A continuous period over which a user is engaged with the target.
Supplemental Resources
Interpretation and translation resources that may be used in building a user interface. These resources include text for labeling interface elements, help text, translations into other languages, and icons, graphics or other multi-media elements.
Target
The target is a device (e.g. VCR) or service (e.g. online phone directory) that the user wishes to use.
Target-Class Template
A UIID that can be mapped to the user interface socket of any target of a certain class such as microwave ovens or televisions.
UI
Acronym for User Interface.
UIID
Acronym for User Interface Implementation Description.
Universal Remote Console (URC)
The URC is a device or software through which the user accesses a target. The URC complies with the AIAP-URC specification and is capable of rendering any AAIML specified user interface. It is “universal” in the sense that it can be used to control any AIAP-URC compliant target. It is assumed that users will choose a URC capable of meeting their personal interaction requirements.
URC
Acronym for Universal Remote Console.
User
Interface Implementation Description (UIID)
A UIID is a user-oriented representation of a target that maps user interface socket elements to interaction (presentation and user input) mechanisms. It provides a structure into which the elements of the presentation are embedded. The presentation mechanisms may be modality-independent, i.e. capable of adaptation to any URC, or specifically designed for a given class of URC device and/or presentation modality.
User Interface Instance
A UIID that completely describes a user interface and has been built and made available in advance of the user’s session with the target.
User Interface Instantiation
A UIID that completely describes a user interface and has been dynamically derived from a presentation-independent template during the user interface construction phase of a user’s session with a target.
User Interface Socket
A low level description of a specific target. It describes the functionality and state of the target as a set of data points and commands.
Overview
AIAP-URC provides a standard way for manufacturers
and other interested parties to describe the information necessary for building
a user interface to a target product or service (hereinafter referred to as a
‘target’). The intention is that this
description structure allows interfaces for any interaction modality to be
constructed, either automatically or by hand.
This document focuses on description of targets. The AIAP will also ultimately include
specification of requirements and preferences pertaining to users, user devices
and contexts.
Figure 1: Overview of the
primary components of a user interface description in AIAP-URC.
Overview
AIAP-URC
provides a standard way for manufacturers and other interested parties to
describe the information necessary for building a user interface to a target
product or service (hereinafter referred to as a ‘target’). Figure 1 illustrates the primary components of such
a description. The intention is that
this description structure allows interfaces for any interaction modality to be
constructed, either automatically or by hand.
This document focuses on description of targets. The AIAP will also ultimately include
specification of requirements and preferences pertaining to users, user devices
and contexts.
The architecture consists of three classes of component: the user interface socket, supplemental resources, and user interface implementation descriptions (UIIDs). The user interface socket is a description of the functionality of a specific target at a low level, i.e. specifying input and output requirements. It is provided by the target manufacturer or provider, and may be located with the target (built into target devices, or co-located with target services) or in some other location specified by the manufacturer. UIIDs provide information on how to present the user interface socket to a user. UIIDs may be contributed by any source, including target manufacturers/providers, URC manufacturers, third party organizations, user groups and individuals, and multiple UIIDs may be available for a particular target. One particular UIID is distinguished: a presentation-independent template. Manufacturers and providers of targets are required to provide a default presentation-independent template. This UIID gives general information that can be used to generate a user interface for any presentation mechanism, and any URC. It is envisioned as being similar to the AAIML language described in previous V2 working documents. The presentation-independent template is a standard format that URC manufacturers will be required to support, thus enabling any URC to be used to control any target. It is anticipated to provide basic control at a reasonable level of efficiency, while alternate UIIDs can provide more highly designed user interfaces. In general, a UIID may be specific to a user device or presentation-independent. It may be specific to a target device or cover a class of targets (eg VCRs).
UIIDs may optionally draw on supplemental resources. These resources include alternative labels, help text, translations into other languages, and graphics or other multi-media elements. The presentation-independent template is an exception in that it must be provided with all labels and other supplemental user interface elements in at least one language without referring to non-local resources. As with UIIDs, supplemental resources may be provided by any source, and may be located at the target, the URC, elsewhere on the network, or be dynamically generated by a resource service. Table 1 summarizes the contents, requirements, providers and number per target of these components, while the following sections describe each in more detail.
|
|
Contents |
Essential or optional |
Provider |
Number per target |
|
User interface socket |
Low-level description of target functionality (input and output) |
Essential |
Target mfr/provider |
1 |
|
Supplemental Resources |
Alternate and language-specific contents (typically static) |
Optional |
Anyone |
0 or more (unlimited) |
|
Default Presentation-Independent Template[1] |
Mapping of user interface socket to abstract interactors (user interface components) |
Essential |
Target mfr/provider |
1 |
|
Other UIIDs |
Mapping of some or all of user interface socket to a user interface, may draw user interface elements from supplemental resources. Includes presentation-independent templates. |
Optional |
Anyone |
0 or more (unlimited) |
Table 1: Some properties of the architectural components.
Note: The terminology used to label these components, and specific details of the contents of each component are subject to change.
Components of a Target’s Abstract User Interface
User interface socket
The user interface socket is a low-level interface
provided by the target manufacturer/provider.
It describes the functionality and state of the target as a set of data
points and commands. The data points
must include all of the data a user can view and/or manipulate, and may also
include additional supporting data that is not presented to the user. Example data points include the volume of a
television, or the current floor of an elevator. The commands must include all of the target functions that can be
called by users. Examples include the
‘search’ command of an airline reservation system or the ‘seek’ command of a CD
player.
Figure 2: Overview of the
contents of a User Interface Socket.
User interface socket
The user interface
socket is a low-level interface provided by the target
manufacturer/provider. Figure 2 gives
an overview of the contents of a user interface socket. It describes the functionality and state of
the target as a set of data points and commands. The data points must include all of the data a user can view
and/or manipulate, and may also include additional supporting data that is not
presented to the user. Example data
points include the volume of a television, or the current floor of an elevator. The commands must include all of the target
functions that can be called by users.
Examples include the ‘search’ command of an airline reservation system
or the ‘seek’ command of a CD player.
Element descriptions
Elements in the user interface socket have additional associated information. This may include constraints such as maximum and minimum values, whether the element is readable and writable by the user, semantic tagging, and information on validity checking and exception handling. For example, a core data point representing a social security number could be declared as type string, and given a maximum and minimum length of 9 characters. It would be both readable and writable by the user. It could have associated validity checks to ensure that all characters are digits, and have a tag indicating that it is a social security number. This tagging would allow a software agent to automatically fill in this field of the form with the user’ social security number. It could also be used by a graphical user interface generator to format the field into blocks of 3, 2 and 4 characters.
Dependencies
Dependencies are used to express relationships among the elements of the user interface socket. Commands may be connected to specific data points with dependency links indicating the relationship between the command and the data point. For example, an airline reservation service may have a ‘search’ command that has a dependency on the ‘destination’ data point indicating that the command cannot be fulfilled until a destination has been entered. There may also be dependencies between data points that determine when a data point can be read or written. For example, a TV may have a data point for the channel number that can only be written if the TV’s power is on. In the airline reservation system example, a ‘return date’ data point may be inactive (i.e. cannot be read nor written) if the trip type is ‘one-way’.
Grouping
The user interface
socket will also include provision for grouping of elements. This is intended to capture semantic
grouping in which a set of elements are logically related. For example, a hi-fi system may have a group
of tuner elements, a group of CD elements, and a group of audio control
elements such as balance and volume.
Supplemental Resources
Figure 3: Overview of
supplemental resources.
Supplemental Resources
The architecture allows supplemental resources for interpretation and translation to be provided separately, and referenced by UIIDs. Figure 3 summarizes some of the supplemental resources envisioned. These resources may include alternative labels, help text, and graphics or other multi-media elements. They may also include translations into different languages, including symbol-based languages. These resources may be provided by any source, and may be located at the target, the URC, elsewhere on the network, or be dynamically generated by a resource service. Hence translations of user interfaces may be provided completely by third parties, even after the target has been manufactured.
Supplemental resources are different from data points (user interface socket) in that they do not represent a target’s state.
The supplemental resources may supplement the user interface socket with additional or alternative semantic tagging. This information can be useful for software agents that prefer semantic models that were not (sufficiently) provided by the target manufacturer through the user interface socket.
A control such as the volume of a television set may have resources providing labels for this control in different languages. For each language there may be several text versions. For example “Volume”, “Vol.”, and “Sound level” in English. There may also be icons suitable for graphical user interfaces, and sound files suitable for audio interfaces. Resources may also include video clips. For example, an instruction for assembling a piece of furniture may have video, graphical and text equivalents of the information.
The supplemental resources may
relate to elements in the user interface socket and/or to elements within
UIIDs. Their structure and organization
are yet to be specified.
Figure 4: Overview of
features of User Interface Implementation Descriptions
User
Interface Implementation Descriptions (UIIDs)
A UIID is a user-oriented representation of a target that maps user interface socket elements to interaction (presentation and user input) mechanisms. Figure 4 illustrates some of the features of UIIDs. It provides a structure into which the elements of the presentation are embedded. The presentation mechanisms may be modality-independent, i.e. capable of adaptation to any URC), or specifically designed for a given class of URC device and/or presentation modality.
Any number of UIIDs could be defined for a given target. These may include all of the elements of a target’s user interface socket, or only a subset of them. For example, a complex home entertainment system could offer a UIID specifically designed for controlling the television set. This UIID could set the hi-fi and vcr to ‘off’ without presenting these options to the user. Only the options relevant to controlling the TV would be included in the user interface.
Partial UIIDs may be used to build more complex UIIDs.
Presentation-Independent Template
One special form of UIID is distinguished: a presentation-independent template that includes all of the elements of the user interface socket. This UIID is self-contained and does not refer to supplemental resources. It is language-specific, but can be translated to a different language by substituting its text components for equivalent text components stored as resources for the target language. Target manufacturers or providers would be required to provide one default presentation-independent template. Additional presentation-independent templates may be provided by any source.
Target-Class Template
Another form of UIID is the target-class template. This UIID can be mapped to the user interface socket of any target of a certain class. For example, there may be a ‘microwave oven’ UIID, or a ‘TV’ UIID. Using such UIIDs, a user can operate any target of the general class in exactly the same way. This is of particular benefit to those who find it difficult to learn new interfaces. For example, target-class templates for UPnP device classes could be created. These could be explicitly mapped to the elements in a specific device’s user interface socket, or standards for user interface socket naming schemes could be adopted (e.g. the volume control always has a particular standard id). For maximum application these target-class templates would adopt the standard format of presentation-independent templates.
Partial/Hint UIIDs
Incomplete UIIDs may
also be defined and used as building blocks for dynamic generation of a user
interface for a target. These may take
the form of presentation hints, such as layout recommendations. They may also be fragments of UIID for
performing particular common tasks, such as setting the time on a device, so
that a user could perform these operations the same way on every device they
use.
User Interface Instances and Instantiations
UIIDs may be so
detailed that they are effectively user interfaces for the target. Target providers may create highly branded,
optimized user interfaces for specific URCs and make these available through
the AIAP standards. A user interface
provided in advance may be referred to as a user interface instance,
while one that is derived from a presentation-independent template is a user
interface instantiation. These can
be viewed as two ends of a spectrum rather than distinct categories.
UIIDs that are
complete user interfaces may be provided by target manufacturers/providers, URC
manufacturers, third party organizations, user groups or individual users. They may even be built for a particular
user, or group of users. For example,
an organization supporting non-literate people could provide a UIID for the
local supermarket’s online shopping service, based on pictures of products
instead of text descriptions. An
individual’s caregiver could further customize that UIID by deleting all items
that the individual never purchases.
Communication Phases
A communication session between a URC and a target involves three major stages. In the discovery phase, information describing available targets is communicated to the URC, and the user chooses a target to interact with. Users may initiate simultaneous sessions with more than one target. This description focuses on one such session. Once a user has indicated that they wish to operate a specific target, a concrete user interface for the target must be rendered. The user interface construction phase finds or generates this user interface. The third phase is the control phase, in which the user operates the target using the constructed user interface on their URC. Each of these phases is described in greater detail below.
Discovery
In the discovery phase the URC locates and identifies all available targets. The AIAP-URC standard defines target properties that can be accessed by any URC. The discovery phase uses discovery and object dissemination functionalities implemented in an underlying networking environment layer.
User Interface Construction
User interface
construction takes place when the user selects a target to operate. Construction of a user interface may take
place on the target, on the URC, or be performed as a network service, or any
combination of these. Construction of
an appropriate interface is essentially the task of constructing a set of
structured, concrete user interface elements appropriate for the capabilities
of the URC, the user’s interaction requirements and preferences (e.g. English
language, non-visual, icon-based), and the task the user has in mind. The AIAP-URC standard does not specify how
such interfaces should be constructed, where the building blocks are to come
from, or the format the final interface should take.
AIAP-URC does specify
the form of a presentation-independent template, which will form the basis of a
user interface where no more suitable UIID or set of partial UIIDs exists. The standard will also specify the mechanism
by which equivalencies and translation resources can be provided and referred
to in UIIDs. More broadly, the AIAP
standard will specify the nature and format of descriptions of user and URC
capabilities and preferences, which will provide further input to this phase.
User interface
construction may be performed more than once for a given control session. It may be revisited if the target changes
any aspect of the user interface socket other than the values of the
elements. For example, new elements may
be added, or the constraints on elements altered. This may be necessary if some aspects of the interface are
dynamically generated. If the target
is providing a UIID, it may also change this UIID during a session, which will
cause the user interface construction phase to be repeated.
Control
The
control phase is the time
period in the URC-target communication exchange when the URC controls the
target via a concrete user interface on the URC. It requires fine-grained, two-way communication between the URC
and target. This communication is
required to synchronize the values of the data points of the user interface
socket between the target and the URC.
If a user’s action causes a change in a data point within the user
interface socket or activates a command in the user interface socket, then this
change or command is immediately transmitted to the target. Similarly, if a
target changes the value of a data point, this change is immediately
transmitted to the URC. A coarser synchronization model can be applied to
individual data points, but the underlying communications protocol is required
to support the fine-grained model.
Fine-grained synchronization would be used for data points such as the
volume of a television, where the user’s changes must be immediately reflected
on the television. Coarse-grained
synchronization can be used for data points such as the fields of a form, which
are only required by the target when the ‘submit’ command is issued.
Required Functionality
Target
This section is incomplete. It will describe the minimum required functionality of a target, in order for it to be an AIAP-compliant target, operable from any AIAP-compliant URC.
A target is required to provide access to a user interface socket describing its functions and data, and a -presentation-independent template, describing an abstract user interface for those functions.
Universal Remote Console
This section is incomplete. It will describe the minimum required functionality of a Universal Remote Console, in order for it to be an AIAP-compliant URC, able to operate any AIAP-compliant target.
A URC is required to be capable of constructing a concrete user interface for any target from either the target’s user interface socket, or the user interface socket plus presentation-independent template.