Lands of fog: Participatory Design with Autistic Children

“Lands of Fog” is an open-ended Full-Body Interaction Environment designed for children with Autism Spectrum Disorder (ASD) to play with a typically developed peer. The experience has the goal of promoting exploration and of scaffolding social and collaborative behaviors between users. The project was founded by RecerCaixa grant.


My role

During my Phd  I have been collaborating in the design of  the experience. Specifically, my role focused on defining, carrying out and analyzing participatory design activities with children with Autism Spectrum Condition

The project

Goal: Develop a Full-Body Interaction experience for children with Autistic Spectrum Condition aimed toward promoting social initiation, understood as the promotion of behaviors such as approaching and looking for others, trying to start social communication and producing any verbal or gestural behavior for communicative goals.

Target users: 10-12 years old children with Autistic Spectrum Condition

The concept:  

  • The game as a mediator of social interaction
  • Support collaboration without enforcing it


  • Involve autistic children in the design of an interactive experience
  • Define methods to facilitate children expressing their own interests and understandings around the experience
  • Define analytical approaches capable of taking into account meanings and aspects that cannot be expressed by words

Children as partners in the design ideation stage


In the initial stage of development, we conducted a PD workshop, based on the “children-as-informants” model with four children with ASD (mean age: 11). During the workshop, children we employed a Wizard-of-Oz prototype of the experience and carried out a set of specific activities.

  • The explorer activity: children in pairs were invited to interact with the prototype and take pictures of anything that captured their interests
  • The detective activity: children who were not playing were invited to watch their playing peers and try to guess what are they doing
  • The designers activity: children were asked to make drawing to depict how they would like to improve the environment

The initial prototype

The system was based on a 6-meter diameter floor projected virtual environment, which is covered by virtual fog. Users explore and interact with the environment using a butterfly net that allows them to open peepholes in the fog and discover what is hidden underneath.  


A multimodal approach to analyse children’s contributions

To analyze these activities, we employed multimodal analysis and focused on:

  1. children’s exploration of the space (their position, paths, pauses and relative speed),
  2. their gaze and the pictures they took during the interaction with the system
  3. their verbal interactions
  4. their drawings and related explanations

Data were annotated on map for each child and analyzed according to a grounded approach

figure2 figure3


  • Children were particularly interested in detailed and liminal areas and in exploring the fog appearance
  • They tended to interpret the underlying environment as a geographical map
  • The act of moving around with the net was associated with the task of hunting or collecting something according to a strictly functional perspective (e.g. “You have to collect animals to make points”)
  • Strong influence of videogames culture

Informing design

The performed analysis allowed us to identify some relevant aspects to be addressed in future design iterations:

  • the use of the virtual fog showed to be an effective design choice to promote exploration. Future design iterations can eventually address the refinements of the fog’s behavior to enrich the possibilities for sensorimotor exploration
  • a realistic and videogame-like graphical style may be adequate to capture children’s attention and foster exploratory behaviors.
  • children’s tendency to interpret the environment as a large space (i.e. a continent) pointed out some possible issues related to defining an adequate scale of the experience
  • children’s lack of interest toward their peers requires a strong effort to promote more collaborative ways of interacting during the experience.  To tackle this latter issue, possible design solutions can use videogames mechanics as motivators of social interaction.


Lessons Learned

  • Multimodal analysis showed to be an effective approach to analyze users contributions across multiple sources and inform design
  • Relevance of analyzing users’ understandings across multiple modalities


Related publications

Malinverni, L., Mora-Guiard, J., & Pares, N. (2016). Towards methods for evaluating and communicating participatory design: A multimodal approach.International Journal of Human-Computer Studies.