It's about time: Purpose, methods and challenges of temporal analyses of multiple data streams
Workshop overview
Recent studies of learning have involved concurrent collection of multiple types of data such as computer activity logs and online discussion, or have applied multi-dimensional coding, resulting in related data streams, which highlight the dynamic nature of learning and require analyses from a temporal perspective. This workshop will explore issues emerging from integrating data streams by identifying a set of analytic difficulties researchers face and illustrating the application of specific methods that address these challenges.
Audience
This workshop is tailored towards those interested in exploring multi-dimensional quantitative analysis of learning, conceptualized as inter-related processes occurring over time. More specifically, two groups might find this workshop particularly relevant:
- Researchers who have completed some type of multi-dimensional coding of data such as interaction analysis, and
- CSCL researchers with time-related data, such as chat and log files of actions, who wish to integrate these data analytically.
Detailed Description
Situated and socio-cognitive learning perspectives emphasize the variety of influences upon learning over time. Recent studies of learning have involved concurrent collection of multiple types of data, such as computer activity logs and chats, or discourse and gesture, or have applied multi-dimensional coding, resulting in related data streams, which highlight the dynamic nature of learning and resist traditional analyses that lack a temporal perspective.
Integrated analyses of multiple data streams that can reveal dynamic relationships and support the development of theory and design principles. Such analyses reveal how phenomena such as utterances, gazes, gestures, co-occur, interact, and facilitate learning, and furthermore, show how they dynamically affect one another over time.
As these data often comprise thousands of data points, quantitative approaches are a promising avenue toward articulating integrated analytic techniques. This workshop proposes to explore issues emerging from integrating data streams by identifying a set of analytic difficulties researchers face and illustrating the application of specific methods that address these challenges.
This workshop will address five challenges to integrated analyses, given that data streams can:
- be discrete (e.g., conversation turns) or continuous (e.g., gaze);
- consist of group and individual data;
- have differing granularities (a conversation turn may last seconds whereas a keystroke occurs in a fraction of a second (Lemke, 2001) or yield overlapping data (McNeill, 1992);
- include anachronistic data (e.g., a child first notices an event minutes after it occurred;
- be poorly synchronized (e.g., computer software often uses internal clocks, but students writing essays on paper might only yield a partial ordering).
The workshop will engage participants in discussing and application of analytic approaches that address these challenges including (ordered by challenges listed above):
- Applying multi-category response statistical models that allow both continuous and discrete variables (Goldstein, 2003);
- Analyzing individual data by creating graph structures for each observation in both the individual and group data as potential explanatory variables, selecting the outcome data of the individual, and running vector auto-regression (Chiu & Khoo, 2005);
- Examining effects of granularity at multiple levels with multilevel analysis or by aggregating the smaller units of data (Goldstein, 2003);
- Linking event occurrence and event perception through graph structures (Voloshin, 2009);
- Interpolating unknown times using known information (e.g., number of words spoken).
Participants will explore ways to address these analytic challenges through hands-on activities, comparing multiple data streams of different research groups; discussion will be framed to explore related theoretical and empirical perspectives. Workshop outcomes include a collection of current challenges to these types of analyses, guidelines supporting multiple data stream analysis (comprising a manuscript for publication), a "vision" document exploring how these analyses can benefit the field of learning sciences (by the organizers) and new collaborations.
This workshop, second in a series of workshops considering learning as a process occurring over time, and considering this aspect to be consequential to understanding learning, addresses the conference theme by both fostering a community of researchers and a forum for comparing methodological, conceptual, and design perspectives related to learning processes.
Workshop outcomes
*A repository of participants' wiki contributions linked with social network software
*Guidelines/principles around a) solving problems related to combining multiple datasets and b) analysis of multiple data streams will comprise a manuscript (formal conference or journal submission) organized by workshop leaders and open to contributions by workshop participants
*Vision note (by workshop leaders): "How do multiple data-streams provide a richer more integrated understanding of learning the different disciplines?"
How to apply
See Workshop Application Instructions. The workshop is full.
References
Azevedo, R. (2008). The Role of Self-Regulated Learning about Science with Hypermedia. In D. Robinson & G. Schraw (Eds.), Recent innovations in educational technology that facilitate student learning (pp. 127-156). Charlotte, NC: Information Age Publishing.
Chiu, M. M., & Khoo, L. (2005). A New Method for Analyzing Sequential Processes: Dynamic Multilevel Analysis. Small Group Research, 36(5), 600-631.
Erkens, G. & Janssen, J. (2008). Automatic Coding of Dialogue Acts in Collaboration Protocols. International Journal of Computer-Supported Collaborative Learning, 3(4), 447-470.
Goldstein, H. (2003). Multilevel statistical models. Goldstein, H. (2003) Multilevel Statistical Models, 3rd edn. London: Arnold.
Lemke, J. L. (2001). The Long and the Short of It: Comments on Multiple Timescale Studies of Human Activity. Journal of the Learning Sciences, 10, 17-26.
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.
Voloshin, V. I. (2009). Introduction to graph and hypergraph theory. Huntington, NY: Nova Science.