Summary of the MMI TPAC face to face meeting and related meetings

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Summary of the MMI TPAC face to face meeting and related meetings

Deborah Dahl

W3C Technical Plenary and Advisory Council Meeting



Multimodal Interaction Working Group Meeting

Group page


This is a summary of the Multimodal Interaction Working Group’s meetings at the 2015 W3C Technical Plenary and Advisory Council Meeting (TPAC) in Sapporo, Japan, October 26-30. We had a number of productive discussions with other Working Groups, including Geolocation, Spatial Data and Timed Text (getting feedback on new EMMA 2.0 features), Automotive (discussion of the MMI Architecture) as well as discussions with the Web of Things Interest Group, on discovery and thing description.

The minutes of the meeting will be published soon.


October 26


1. We held a meeting with the Geolocation WG [1]  to review the <emma:location> element [2]. Most of the attributes in <emma:location> are based on attributes in the Geolocation API; however there are two additional higher level attributes in <emma:location> (<emma:address> and <emma:description>) that aren't included in the Geolocation API. These are related to the work of the Spatial Data on the Web WG, so we scheduled a meeting with them on Tuesday. We were also referred to some non-W3C specs which might be relevant to locations in EMMA, specifically, "A Presence-based GEOPRIV Location Object Format" [3] and "Civic Addresses" [4].


October 27


1. Meeting with Spatial Data on the Web WG [5]. This group is just getting started, but they may have specs relevant to the <emma:description> attribute of <emma:location> in a few months.

2. Meeting with the Automotive Working Group [6] to review the Multimodal Architecture [7] for automotive applications, and to talk about location in EMMA 2.0.


Technical Plenary Day

October 28


1. The Wednesday TPAC agenda consisted primarily of breakout sessions. We held a session on "Standards for Personal Assistants". One interesting suggestion that came out of this session was that virtual personal assistants should be more knowledgeable -- that is, they should be able to intelligently analyze web pages or other information on the web, and present the information to the user according to the user's preferences. Typically, the "intelligence" of intelligent personal assistants is thought of as having to do with the ability to engage in a natural spoken conversation, but this discussion also considered the issue of how the assistant actually acquires knowledge and determines how to present it to the user. See the summary and minutes [8] for details of the session.


October 29


MMI Meeting

1. Review of EMMA 1.0 in preparation for EMMA 2.0 discussion in the afternoon [9].

2. We joined the Web of Things Interest Group [10], specifically the task forces on API's and Discovery. Helena Rodriguez (invited Expert)  presented her work on discovery of multimodal components to the WoT discovery task force.

3. Review of EMMA 2.0 features [9] and discussion of a JSON version of EMMA. From the discussions with the meeting observers, there did not seem to be much value in officially standardizing a JSON format for EMMA, since there are many tools that can automatically generate JSON from XML.

4. We reviewed several MMI use cases [11], specifically UC-1, 2, 3, 7 and 8. We also discussed two new EMMA use cases. One was using EMMA to represent user input from gestures in a Virtual Reality enviroment, and the second was to record user input, including speech and gesture, in order to be able to replay interactions for testing. Deborah Dahl (Invited Expert) also presented a demo of EMMA 2.0 including the <emma:output> element.


October 30


MMI Meeting

1. Discovery: Helena Rodriguez (Invited Expert) presented initial work on Discovery Vocabulary, which provides a vocabulary for describing Modality Components. This work follows on from Discovery Use Cases and Requirements [12] and Discovery State Handling [13] This will form the basis of the next MMI WG publication on Discovery. The overall approach is based on concepts from the Web Services Description Language (WSDL) 2.0 [14] except that in this case the description is of modality components rather than Web Services. The vocabularies are used to describe aspects of the component such as its type, interaction modalities available, and operations.

It was pointed out that some work in accessibility, such as Universal Remote Console (ISO 24752) [15] and also some work on individual needs and preferences for accessibility is relevant to this topic.

2. Joint meeting with the Timed Text WG [16]  to present our example of EMMA outputing Timed Text [17]:

We presented the example of EMMA 2.0 containing Timed Text as an output format. Use cases include showing caption text for a video in a browser, presenting machine-translated captions, or video descriptions. The main comment we received was that if we wanted to show the final rendering for the user, the timed text should be in a container, for example, an HTML page. However, the timed text within EMMA would not need to be in a container if the EMMA document represents an intermediate stage prior to final rendering, and the exact presentation for the user hasn't yet been specified. Other feedback was that the section would be clearer if we added discussion of specific use cases.

3.  Use cases:

Our final agenda topic was continuing to discuss use cases and brainstorm new use cases. We added  UC-17: User interface as a sensor, UC-18: smart remote for appliances and UC-19: Handling millions of components [11].



[1] Geolocation Working Group

[2] <emma:location> element

[3] A Presence-based GEOPRIV Location Object Format

[4] Civic Addresses:

[5] Spatial Data on the Web WG

[6] Automotive Working Group

[7] Multimodal Architecture and Interfaces:

[8] Standards for Personal Assistants

[9] EMMA slides (prepared by Michael Johnston, Ivited Expert): (to be replaced with public link)

[10] Web of Things Interest Group

[11] MMI use cases

[12] Discovery Use cases and Requirements

[13] Discovery State Handling

[14] Web Services Description Language (WSDL .20)

[15] Universal Remote Console

[16] Timed Text Working Group

[17] Timed Text in EMMA Timed Text in EMMA 2.0