Evaluation methods: Combining multimodal analysis and user-as-informants

This methodological approach combines the analysis of children’s in situ interaction with the system and their retrospective reflections on the experience. To evaluate these aspects, we employ the observation, analysis, integration and interpretation of a wide range of multimodal resources. At a procedural level, this approach is based on having children interacting with the system and subsequently involve them in posterior redesign activities. Methodologically, data proceeding from in situ interaction and from the redesign activities are analyzed employing multimodal analysis. This approach showed to be effective in encompassing the experiential learning cycle of getting engaged with an experience, experimenting with it and transforming it into an object of knowledge (Kolb et al., 2001).


Related Publications:

Malinverni, L., Mora-Guiard, J., & Pares, N. (2016). Towards methods for evaluating and communicating participatory design: A multimodal approach. International Journal of Human-Computer Studies.

Evaluation methods: Multimodal analysis

Multimodality is an interdisciplinary approach, derived from socio-semiotics and aimed at analyzing communication and situated interaction from a perspective that encompasses the different resources that people use to construct meaning. At a methodological level, multimodal analysis provides “concepts, methods and a framework for the collection and analysis of visual, aural, embodied and spatial aspects of interaction and environments” (Jewitt, 2013).

For understanding embodied meaning-making in Full-Body Interactions Learning Environments, we employed this methodological approach to analyze users’ in situ interaction with the system. Specifically, we focused on analyzing both the affordances offered by the system (e.g. its physical configuration, the available physical/digital objects, etc.) as well as the embodied forms of interacting with it (e.g. the paths that users follow to explore the environment, the variations and repetitions in their sensorimotor enactments, their usage of the available physical/digital elements, their focus of attention and their reciprocal proxemics and social relations). This analysis, by fully acknowledging and taking into account embodiment, constituted an appropriate and consistent research method to better understand meaning construction and learning in this kind of environments. At the same time, it provided relevant insights to guide Design-Based research processes and defining design improvements.

In the pictures, two examples of different techniques for the graphical transcriptions for multimodal analysis.



Evaluation methods: Redesign as an evaluation technique

Redesign is an evaluation techniques oriented toward grasp children’s retrospective understanding of an experience. After playing with an interactive experience, children are involved as critics to discuss on it and are required to propose design improvements and refinements. For this purpose, different PD techniques are employed. The use of critique and Redesign showed to be quite effective to grasp children’s understandings and to complement our observations of in situ interaction.

Participatory Design: Intuitive actions

The Intuitive Actions Elicitation technique is based on requiring participants to invent and perform physical actions related to specific activities during the interactive experience, using a mid-tech Wizard of Oz prototype of a Full-Body Interaction Learning Environment.

This technique allows the design team to analyze how children intuitively interacted without the influence of external mediation or requests. Furthermore, it facilitates observations related to the affordances of the spatial configuration of the environment, i.e. which sensorimotor experience the interface of virtual environment evoked. We conclude that this design technique is particularly useful to confirm and refine initial ideas for physical actions by comparing the designers’ proposals with children’s intuitive interaction in an existing prototype.


Related Publications:

Malinverni, L., Schaper, M.-M.,  and Pares, N. (2016). An evaluation-driven design approach to develop learning environments based on full-body interaction. Educational Technology Research and Development. DOI=http://dx.doi:10.1007/s11423-016-9468-z

Participatory Design: Transmodal translation

The transmodal translation, is an elicitation techniques that requires participants to translate the same idea across different modal resources (e.g. drawing, writing, enactment, etc.). Children are, therefore, provided with different tasks (e.g. make a drawing or a video report) to describe their experience with the interactive system.

Its application allows tapping into different shades of children understandings around the experience. Even if future research on this technique is still needed, we suggest that this approach can be particularly suitable to gather requirements when working with an early prototype of the system since it offers relevant contributions to delve into the different shades of children’s understanding of a phenomenon.

Participatory Design: the Pictionary activity

The Pictionary techniques is an elicitation method to evaluate which concepts children are familiar with and explore their embodied representations of those specific concepts. We suggest its usage in the initial stage of the design process, by involving children as co-designers to properly frame the experience.

The technique is based on using the mechanics of the board game Pictionary and using terms related to the addressed learning goals. Specifically, one child per time is asked to randomly pick-up one of the terms and to represent it through drawings on a whiteboard. The other children have one minute to guess it. In our studies, this technique showed to offer relevant affordances to grasp children’s representations across different modes. Furthermore, it showed to be particularly effective when associated with a careful analysis of the specific formal features of their drawings and their embodiment during the task (e.g. gestures, facial expression). The playful nature of the activity makes it engaging for children and its time-based structure avoids that they focusing too much on surface details. As a consequence, we suggest that this technique may represent a useful tool to grasp children’s conceptions around a specific topic in a playful and “quick-and-dirty” way.

Participatory Design Strategies with Children with Special Needs

During the last year I worked in different participatory design processes to design video games for and with children with Autistic Spectrum Disorder (ASD).

Pico’s Adventure

In 2013, I designed and facilitated a 10 sessions workshop for children with ASD to co-design the game Pico’s Adventure. The game “Pico’s Adventure” was funded by the European project M4ALL and aimed at scaffolding social interaction in children with Autistic Spectrum Disorder (ASD).

During the project I have been in charge of organizing and carry on the participatory design with ASD children. Four children worked with us as “co-designers” to transform defined goals into an enjoyable playful experience and evaluate which aspects elicit higher level of motivation and interest in children.


The workshop took place in the “Hospital Sant Joan de Deu“ on a weekly basis. The participants selected by the UETD professionals were four children how joined a total of five sessions, during which they were designing, discussing, drawing and experimenting with us in order to create an interesting and enjoyable game. The experience has been incredibly enriching both from the point of view of the research and the game design. We are very thankful to the “co-designer” children for their contributions!

For more informations visit: http://lab4.ccp.upf.edu/

Land’s of Fog

During 2015, I designed and facilitated a 4 sessions workshop for children with ASD to co-design the game Land’s of Fog. The game “Land’s of fog” was funded by the RecerCaixa 2013 grant (Feb/2014 to Jan/2016). The research project investigates how a full-body interactive environment can foster social initiation behaviours in children with Autism while interacting with typically developed children.

During the project I have been in charge of organizing and carry on the participatory design with 4 ASD children. In the different sessions of the workshop we worked together to design the environment of the game, its characters and their behaviors.

For more informations visit: http://inautistic.upf.edu/

Publications about the projects:

In case you want to know more information about the design process and methods you can read:

– Malinverni, L., Mora-Guiard, J., & Pares, N. (2016). Towards methods for evaluating and communicating participatory design: A multimodal approach.International Journal of Human-Computer Studies.

– Malinverni, L., Mora-Guiard, J., Padillo, V., Valero, L., Hervás, A., & Pares, N. (2016). An inclusive design approach for developing video games for children with Autism Spectrum Disorder. Computers in Human Behavior.

– Mora-Guiard, J., Malinverni, L., Pares, N., (2014) Narrative-Based Elicitation: Orchestrating Contributions from Experts and Children, in CHI ’14 Extended Abstracts Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems, Toronto, Canada

– Malinverni, L., Mora-Guiard, J., Padillo, V., Mairena, M.A., Hervás, A., Pares, N., (2014) Participatory Design Strategies to Enhance the Creative Contribution of Children with Special Needs, in Proceedings of the 2014 Conference on Interaction Design and Children, IDC2014, Aarhus, Denmark

Participatory Design: Sketching through the body

Or how to co-design for embodied interaction with children.

Involving users in the design of their own technologies is both an ethical standpoint as well as a fundamental research approach.However, what happen when we have to design for innovative interfaces such as embodied interaction?

Together with Marie-Monique Schaper we carried out several studies aimed at exploring techniques  to design specific gestures with children to improve the interaction design of a Full-Body Interaction Learning Environment.

For this purpose we explored techniques proceeding from theatre, performance and visual art. The results of our research indicate the potential of those Participatory Design methods which combine multi-modal resources as instruments to allow children to reflect upon their own knowledge and express it more precisely.sketch

If you want to know more on how research you can read:

  • Schaper, M. M., Malinverni, L., & Pares, N. (2015, June). Sketching through the body: child-generated gestures in full-body interaction design. InProceedings of the 14th International Conference on Interaction Design and Children (pp. 255-258). ACM.