• Interaction Framework
Skip to end of metadata
Go to start of metadata

Why build an Interaction Framework (or Semantic Framework)

The goal for this section is to begin to describe the kinds of interactions that take place in a Smart Classroom to begin so we can begin to paint a picture about the how this can help in the designing of curriculum and also working with techies to further build out the types of interaction possible in a smart classroom - one of the advantages of approaching the development of a semantic framework from the research and pedagogical design side of things is that it can more easily be technology independent, that is to say if a new technology comes along that does a better job (from a techie point of view) it can replace the current one without impacting the overall pedagogical design.


For now I'm thinking about structuring the different affordances of the smart classroom in terms of Dichotomies, however it might be better to think of them as sliders where there are often things not only at the extremes but also along the continuum.

Coordination of people and groups

This part of the framework discussed specifically the people (researchers, students, teachers, groups) within the space.

Synchonous versus asynchonous communication

(From: A taxonomy for Multimedia Service Composition - Nahrstedt & Balke)

  • The idea here is whether or not communication (or development of content) happens (or needs to happen) in real time or if the communication is asynchonous - This is a bit different that the notion of Time below in that this assumes the system is live at the time and looks at the interactions taking place when the room is at this state (looking more at the interaction of users)

Service Composition
(From: A taxonomy for Multimedia Service Composition - Nahrstedt & Balke)

A good description of how the different kinds of composition take place; it's also a good example of the notion of sliding along the continuum where

  1. Successive composition, (asynchonous)
  2. Concurrent composition, and (synchronous)
  3. Hybrid composition (a blend)
    • This would be a great way to describe how content is produced in the curriculum - does one thing naturally flow into the next or do things happen simultaneously... requiring "on-the-fly" aggregation and sorting of the content.


(From: A taxonomy of ubiquitous computing applications - Jeon et al., 2006)

  • Generality - Characteristic in which most society constituent uses ubiquitous computing through computing environment that have simple interface and autonomy
  • Specialty - Characteristic in which specific individual or group uses ubiquitous computing to achieve specific purpose (Taken from Jeon et. al.)

I like the general idea of this definition but I think it could be adjusted a bit to suit a classroom better - the way I see it you could think about the users as general or specific reading this way we could begin to think about whether or not it is important for the room to "know" who is in the room or if anyone can be in and interact with the room.

An example of Generality might be something like a museum where users are undefined and are free to enter or exit the space without needing to authenticate themselves or to sign in to the room.

On the other hand Specialty would refer to a specifically define set of users who are "known" to the room. An example would be something like our Physics scenario where students log into the room.

Note: I'm still playing with this one a bit but it's starting to take shape, I'm not sure if logging in is required for notions of Specialty or if the people in the room are "known" in a broader sense - one particular class for example

Coordination of Space

  • This details how students are organized throughout the room from
    • Assigned - describes a state students are set into a space by the teacher, or agents in the room
    • free movement - is where no conditions are placed on where the student needs to be to interact with the space.
      check out some literature on coordination of learners in a collaborative working environment, there must be plenty on this

      Technology Affordances

This section begins to discuss the kinds of technological affordances of the space, such as physical ones (displays etc) and perhaps more complex ones like when the room is accessible


A crucial element to designing the curriculum within these contexts is also what kind of display the information will be presented on and who is using the display. If only one user is interacting with the space or a specific device an "Individual" representation is adequate; however, if it is multiple students sharing or interacting with the displays a "Combined Aggregate" would be required.

States of Data Interaction

(From Phenomenon Server Documentation - Gnoli, A., working draft 2009, will attach paper with permission)

This section essentially describes the different states that information will be presented to the user in. The key difference shown below is that in some case the information cannot be edited by the users of the space but can merely be observed.

  • Static - "The user is external to unfolding of the phenomenon"
    • I would add in this case that the users role is not to change the information in the space rather to observe or collect information within the space - an example of this would be the exploded visualization of physics problems across the room where students can move throughout the room and collect physics problems on their handheld but they don't actually change the questions in any way.
  • Interactive - "These 'phenomena' can be modified by the users"
    • "This means the evolution of the phenomena can't be completely known before the execution because the user actions influence how the phenomena evolves"
    • Above is a nice way of putting this, the landscape essentially changes as users interact with it, which may signal the need for more active agents or decision points in the curriculum as the users interact with it.

This bring up some interesting questions about what it means for information to be static... if students more the information around the screen, sort it in different ways, or make connections between one piece of information and another but the individual pieces themselves remain unchanged can the environment still be considered static?

Collecting Implicit Information - Information Gathering Practices

This section describes how information is collected or sent to the room.

There are two distinct methods of this and I'm still playing with the wording on how to describe them, but in essence there are two methods of the room collecting information: one where the room is constantly collecting data from the participants or the environment (intentional); VERSUS the room simply waits until the users input information to the room (passive)

  • Types of inputs:
    • Intentional: RFID, Sensors, etc.
    • Passive: Keyboards, Smartphones, touch walls etc.

The kinds of inputs used go a long way in determining what kinds of interactions take place - Intentional inputs would make one assume that the room itself is more along the Active Situation Recognition dichotomy since the room would be reading what's going on in the room without students actively serving it up to the room.

Again this begins to move towards the active passive paradigm for information gathering and processing

Design of the Space

This section begins to describe the design of the space and the activity within the space - issues such as the Temporality (when it's "on") the location (ubiquitous, single space) etc. Basically all the elements that make up where and when the room is accessible.


(no current sources for this section yet)

Unlike Time which looks at when an environment is available or live this area looks more at the duration of the activity in the room. There are many cases where a room may be used for a single interaction and then that particular curriculum is finished, in other cases it may be an ongoing project.

  • Single Use - In this case the room and the curriculum is enacted a single time and then ends, in most cases aside from some reflection or small follow up work the information gathered in the room is not used again. An example of this might be a simple sorting or solving exercise that is being used for a class to begin a discussion around a topic but the information has no use outside the single class
  • Persistent - In this case the interactions and the data collected continue over a prolonged period of time (could be over a fixed period of time or truly persistent and unending). Here the information gathered is added to existing data to create a richer database for analysis by students, teachers, and researchers. Our proposed associative web for the physics class and Wallcology are both examples of this.


(From: A taxonomy of ubiquitous computing applications - Jeon et al., 2006)

Here we have the ideas of where a user can connect to the smartroom (in a sense how far the room or learning is extended)

  • Mobility - Characteristic in which anywhere connectivity of network is guaranteed to use user's ubiquitous computing
  • Fixation - Characteristic in which ubiquitous computing environment can be used in specific place

In the case of Mobility projects such as Tim Zimmerman's where students can go out with mobile devices and capture data to be examined back in the class would be a good example - the "room" in this case is extended and augmented by the mobile device; Another example would be our proposed extension of the Physics curriculum to allow students to log in from their home computers, although the room itself isn't present the students are still interacting and modifying the contents of the room.

Some good examples of Fixation would be Wallcology and Roomquake as the students must be in the room to take part.

Access to the Environment

(From: A taxonomy of ubiquitous computing applications - Jeon et al., 2006)

This section talks more about users' ability to access the smartroom, or when it's live

  • Always - Characteristic in which anytime network connectivity is guaranteed

Think about Tom and Gugo's Wallcology project - it is always running, and students could in theory check on what's going on at any time of the day assuming they are in the room

  • Temporary - Characteristic which is used at specific time by temporal request of user according to construction of ubiquitous computing

In this case the room is only live when people are in the room and using it - like our current Physics application; however, some of our designs including making a version of the Physics curriculum available from students' home computers would change this attribute.

Recognition and Software

This section talks about the kinds of software that manages the room and the students - this might be agents or other types of controls to the decision making and scripting of the environment.

Situation recognition

(From: An Agent-Based Framework for Context-Aware Services - Burkle et al., 2007)

The idea here is how the system goes about producing a recognition of what is going on in the room.

Note: Burkle et al. don't specifically define this notion of active versus passive but really only describe their own system which is clearly an active one as per my definition

  • Active Situation Recognition - The room actively monitors the situation continually pulling cues from the environment (using something along the lines of XMPP, or monitoring agents)

Things such as how long students have been in the room, the number of questions they have answers or other factors that would actively make a room react or decide without prompting from an outside command would qualify as Active.

A short quote from Burkle et al. describing Active Situation Recognition "To be able to "fuse" many of these perceptual components together in order to determine more sophisticated and meaningful states, additional middleware interfaces have been developed to facilitate the intercommunication of these components, as well as higher layer agents which act as receptors of the whole range of the elementary context cues and process them in order to map the resulting contextual information into a complicated situation."

  • Passive Situation Recognition - The room sits passively waiting for users or
    teachers to activate a check to see the state of the system and to respond accordingly

In the Pasive case the room itself doesn't react (or change states) because of what is going on in the room until prompted by someone (either the users or a teacher) - the room could collect data but the conditions of the room would stay the same

Context Awareness

(From: Intelligent Agents Meet the Semantic Web in Smart Spaces - Chen et al, 2004)

Is this basicaly the same as situation recognition? Could be - Might be worth noting both

DEF: In particular, it acquires and maintains consistent knowledge about the meeting participants' individual locations, scheduled events and presentations, speaker pro?les, and the state of the meeting. The broker discovers and downloads some contextual information expressed in OWL directly from the Web or from data provided by sensing agents. It uses logical reasoning to infer contextual information that it can't directly acquire.

Sending information to users

(From: Semantic web technologies for ubiquitous computing resource management in smart spaces - Soldatos et al (nd)

A little different from the collection of information, the sending of information describes how and when students are given new information - this could be in the form of new questions, next steps, of updates about what others are doing or states in the room.

  • User Requested - The user has to request new information, by sending a request to the room directly for updates
    • Student requests a new question to solve
    • Student requests and update as to the status and location of their bug (Wallcology)
    • Student actively selects to be notified when other students complete tasks, add new content etc.
  • Context Determined (need better name ?)
    • Students are sent new information when they (or the rest of the class) complete something or reach a certain state.

Can be correlated with context awareness, but I feel there is something very distinct between these two factors, I need to push the edges of these two a bit but I think the distinction is important

Thoughts on the need for this classification

"Both middleware and hardware components are characterized by extreme diversity in terms of functionality, underlying technologies and vendors"
(From: An Otology-based Framework for Dynamic Resource Management - Pandis et al. 2005)

  • Despite this variety there is little attempt to classify how these different technologies intersect - there is a growing need to understand what specific contexts and elements a specific smartroom build encompasses to successfully implement curriculum designs that leverage the technologies available and give researchers and educators alike insight into the kinds of interactions available to them.

"Facilitating the reusability of components and established a common methodology for the development of services in a ubiquitous computing environment"
(From: An Otology-based Framework for Dynamic Resource Management - Pandis et al. 2005)

  • Framework works as a yellowpages registry/core vocabulary
    • In this case Ontologies are constructed for ease of integration and interoperability... in many ways this is very important for the ease of importing different services, technologies, and devices
    • Where it also comes in handy is in the co-design phase of the work - by having clearly defined ontologies (especially if they can be limited to falling within a series of defined dichotomies ) designers can understand the types of interactions available and methods for their successful deployment
    • This allows teachers, researchers, and curriculum developers to still be active in the design process without the need for high level technical and programming skills
      (Not sure if its from: An Otology-based Framework for Dynamic Resource Management - Pandis et al. 2005

III. Other Readings

(haven't read but seem important will add to Smart Classroom Papers)

  • Soldatos, J. (2006) 'Software agents in ubiquitous computing:
    benefits and the CHIL case study',
    Proc. of the Software Agents in Information Systems and
    Industrial Applications (SAISIA'06)
    Workshop, Karlruhe, Germany, 9-10 February.
  • T. Moran and P. Dourish, "Introduction to This Special Issue
    on Context- Aware Computing," Human-Computer Interaction,
    vol. 16, nos. 2-4, 2001, pp. 87-97.
  • A. Jameson, "Adaptive Interfaces and Agents," The
    Human-Computer Interac- tion Handbook, Lawrence Erlbaum
    Assoc., 2003, pp. 316-318.
  • No labels