Multimodality is an interdisciplinary approach, derived from socio-semiotics and aimed at analyzing communication and situated interaction from a perspective that encompasses the different resources that people use to construct meaning. At a methodological level, multimodal analysis provides “concepts, methods and a framework for the collection and analysis of visual, aural, embodied and spatial aspects of interaction and environments” (Jewitt, 2013).
For understanding embodied meaning-making in Full-Body Interactions Learning Environments, we employed this methodological approach to analyze users’ in situ interaction with the system. Specifically, we focused on analyzing both the affordances offered by the system (e.g. its physical configuration, the available physical/digital objects, etc.) as well as the embodied forms of interacting with it (e.g. the paths that users follow to explore the environment, the variations and repetitions in their sensorimotor enactments, their usage of the available physical/digital elements, their focus of attention and their reciprocal proxemics and social relations). This analysis, by fully acknowledging and taking into account embodiment, constituted an appropriate and consistent research method to better understand meaning construction and learning in this kind of environments. At the same time, it provided relevant insights to guide Design-Based research processes and defining design improvements.
In the pictures, two examples of different techniques for the graphical transcriptions for multimodal analysis.