Pico’s Adventure: A Kinect Game to Promote Social Initiation in Children with Autism Spectrum Condition

Pico’s Adventure: A Kinect Game to Promote Social Initiation in Children with Autism Spectrum Condition

 

picoadv_level1

Pico’s Adventure is a Kinect game for children with autistic spectrum disorder, aimed at promoting social initiation. The project was part of the European Project “M4all: motion- based adaptable playful learning experiences for children with motor and intellectual disabilities” and developed in the collaboration between the Universitat Pompeu Fabra and the Specialized Unit on Developmental Disorders of the Hospital Sant Joan de Déu.

My role

During my Phd  I have been working in the design and evaluation of the game “Pico’s Adventure” . Specifically, I focused in the requirements elicitation, participatory design, evaluation and reporting.

Defining the project

schermata-2016-10-17-alle-19-47-31Context: European Project “M4all: motion- based adaptable playful learning experiences for children with motor and intellectual disabilities”

Goal: Develop  a Kinect-based game for children with Autistic Spectrum Condition aimed toward promoting social initiation, understood as the promotion of behaviors such as approaching and looking for others, trying to start social communication and producing any verbal or gestural behavior for communicative goals.

Target users: 4-6 children with Autistic Spectrum Condition

The concept:  THE GAME AS A MEDIATOR OF SOCIAL COMMUNICATION = Design situations that:

  •  Are valuable for the children to communicate about them
  • Integrate therapeutic techniques used to facilitate social initiation
  • Require the child to look for external collaboration

Challenges

  • Make an experience that is relevant for children and effective in terms of therapeutic goals
  • Orchestrate requirements from therapists and children’s interests
  • Test the therapeutic effectiveness of the developed game

The Design Approach

  • Elicit and combine requirements from therapists and children

Approach: Narrative- based elicitation to combine narrative structure and therapeutic techniques

Method: Participatory Design with Children and Focus Groups with therapists

schermata-2016-10-17-alle-20-20-51

schermata-2016-10-17-alle-20-21-06

schermata-2016-10-17-alle-20-21-16

  • Iterative Design and Evaluation : field notes, video analysis, questionnaire for stakeholders feedbacks

The Game

During the game, the children will be introduced to a fantasy world inhabited by a friendly mascot (the virtual agent), named “Pico”, who will accompany them in the journey toward social initiation. Children – either alone or in collaboration with adults and peers – will have to help the mascot, an amicable alien, who landed on our planet Earth after having some problems with its spaceship. Children will have to help it in overcome different missions, each one designed to address a targeted behavior related with social initiation. Within that, major emphasis will be posed on designing game situations that require the child to seek for the collaboration either of an adult or a peer.

schermata-2016-10-17-alle-21-51-47

 

The evaluation

The game has been evaluated in two stages: a first exploratory stage aimed at evaluating its acceptance by the child, and a second experimental study oriented at assessing the effectiveness of the game in trigger target behaviors related with social initiation.

Experimental study

schermata-2016-10-17-alle-21-38-09Population: The study  involved 15 boys with ASD between 4 and 5 years old.

Goals: In the study we compare social interaction between free-plays activities and games activity

Methods: For the analysis we employed systematic video-coding by focusing specifically on: Social requests, Social initiation, Responses, Gestures

 

Results

schermata-2016-10-17-alle-20-41-27Results show a significant increase of social initiation behaviors in 1st and 4th game sessions compared to free play (p< 0.05)

 

 

 

 

Lessons Learned

  • The use of narrative resources showed to be an effective mediator between different disciplines, framework and generations.
  • We develop a set of effective participatory design methods for populations with special needs
  • Effectiveness of immersive, bodily-based interaction to support social initiation and collaborative behaviors

Related publications

Malinverni, L., Mora-Guiard, J., Padillo, V., Valero, L., Hervás, A., & Pares, N. (2016). An inclusive design approach for developing video games for children with Autism Spectrum Disorder. Computers in Human Behavior.

Malinverni, L., MoraGuiard, J., Padillo, V., Mairena, M., Hervás, A., & Pares, N. (2014, June). Participatory design strategies to enhance the creative contribution of children with special needs. In Proceedings of the 2014 conference on Interaction design and children (pp. 85-94). ACM.

Mora-Guiard, J., Malinverni, L., Pares, N., (2014) Narrative-Based Elicitation: Orchestrating Contributions from Experts and Children, in CHI ’14 Extended Abstracts Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems, Toronto, Canada

THE GAME “PICO’S ADVENTURE”  IS AVAILABLE FOR DOWNLOADING FOR WINDOWS 7 or WINDOWS 8 in the following webpage: http://m4all.widetesting.info/

Mapocci: a robotic companionship

Mapocci: a robotic companionship

img_4632

Mapocci is an interactive toy, designed to promote emotional learning and social communication in children. The project has been been awarded with the “Next Thing Grant 2012” and was produced with the support of Telefonica I+D and Laboral, Art and Industrial Creation .

My role

Together with Lilia Villafuerte we designed, developed and evaluated Mapocci, a robotic companionship for affective communication. In the project I worked in the design, the hardware development and users testing.

The project

Halfway between a toy, a media and a pet, Mapocci is proposed as a playable interface designed to allow children to experiment emotional communication through a physical and sonic experience. Thus, children can interact with Mapocci by touch, movement and physical activity.
The toy, thanks to a system of sensors and emotional intelligence software can recognize the care that receive and react accordingly to how it affects its own mood. These features allow Mapocci to function as a relational object, responsible for collecting and make visible emotional states expressed by the child in their physical interaction. This capability is expanded thanks to the possibility of Mapocci to communicate. The 3g Internet system allows two Mapocci to connect to each other, facilitating the remote exchange of emotional messages.
The communication system, inspired by the empathic functioning of the brain, allow physical interactions with one toy to be reflected in the behavior of the other and vice versa. The model generates a context where each toy serves as a metaphor of the other, emphasizing links between human beings and the active role of each one in determining the sensations and emotions of others.

The research 

map_research-01

Mapocci is an interdisciplinary project that crosses several research lines such as technology development, interaction design and cognitive science. Further researches will include the exploration of the use with children with special needs, such as autistic spectrum condition.

  1. Research background

    Mapocci arises from the researches related with the use of robotic companionships as relational artifacts.

    In this field we paid especial attention to three main aspects:

    – the possibilities related with augment play’s experience through internet connected toys by include social aspects in tangible interactive experience

    – the design of an affective interaction loop by merging narrative resources with bodily and tangible experiences

    – the opportunities of using object-mediated communication as a tool to address specific issues in special education contexts

  2. Mapocci: objetives and possible uses

    Mapocci works as a toy aimed at facilitate and enhance social communication and interaction between distant children. Moreover, through its emotional model, Mapocci can provide a direct affective feedback to child’s actions and in this way allow the development of awareness about his own behaviour.

    For these reason Mapocci can find possible uses both in playful activities or in learning and therapeutic contexts.

  3. Something new about robots..Something new about plush toys

The importance of materiality: design and development of innovative sensors for robotic companionships

One of the main innovotive features of Mapocci is that the toy is completely soft and plushy! This design decision arise from the hypothesis that materiality can have a great impact on user’s behaviour. For this reason we developed a soft circuitry and e-textile sensors that allow gesture recognition from tangible manipulation.

Being connected: embed M2M commmunication in play experience

What if you plush toy has a firend that lives in the order side of the world and can always speaks with him? What if you can send emotions to a fareaway relative using a plush toy? Mapocci explores the possibilities of augment play’s experience through internet connected toys
 and provides new pattern of interaction to allow differentiated forms of long distance communication.

Embed narrative resources into robotic design

In the Mapocci project we researched and explored how narrative resources can be embedded into robotic design and behaviour in order to enrich user experience and facilitate affective interaction.

The development

imag0314-1024x613

img_73181-1024x682

The development of Mapocci, conducted between March and November 2012, has completed with the construction of two functional prototypes. The prototypes are based on Arduino and a 3g shield developed by Telefónica I + D. The hardware make use of a novel system of sensors constructed with “smart” textile to ensure the softness and smoothness of the doll. The software is based on a emotional model that provides a particular personality to each Mapocci and determines the intensity of its reaction to the interaction of children.

The prototyping methodology was based on the extensive use of iterative design, aimed at optimizing the interaction and build a narrative universe around the toy. Multiple tests with children between three and eight years old were conducted. These users’ studies allowed us to improve the toy and explore the symbolic universe associated with its behavior. Through these tests we had been able to generate a satisfactory user experience and to produce concrete metaphors to explain abstract concepts to kids, such as remote communication.

Mapocci from lilia villafuerte on Vimeo.

If you want to know more about the project, visit: http://mapocci.net/

Embodiment & the Socio-Affective Aspects of Collaboration

Embodiment & the Socio-Affective Aspects of Collaboration

Analyzing the impact of different interfaces on collaborative attitudes.

collab

During my PhD I’ve been researching on how different interaction paradigms -one based on Full-Body Interaction and the other on Desktop computer interaction- may impact the socio-affective aspects of collaboration. The results of this research showed that Full-Body Interaction had a highly significant impact on how the participating children perceived collaboration in small groups and on how they felt about the other children in the group. This indicates that Full-Body Interaction may be beneficial for supporting the construction of a positive social space for collaborative learning, given its potential to enable the use of embodied resources, which are fundamental for social communication and social cognition.

Here you can find the paper that report the results:

 

Participatory Design Strategies with Children with Special Needs

Participatory Design Strategies with Children with Special Needs

During the last year I worked in different participatory design processes to design video games for and with children with Autistic Spectrum Disorder (ASD).

Pico’s Adventure

In 2013, I designed and facilitated a 10 sessions workshop for children with ASD to co-design the game Pico’s Adventure. The game “Pico’s Adventure” was funded by the European project M4ALL and aimed at scaffolding social interaction in children with Autistic Spectrum Disorder (ASD).

During the project I have been in charge of organizing and carry on the participatory design with ASD children. Four children worked with us as “co-designers” to transform defined goals into an enjoyable playful experience and evaluate which aspects elicit higher level of motivation and interest in children.

IMG_20131017_191017

The workshop took place in the “Hospital Sant Joan de Deu“ on a weekly basis. The participants selected by the UETD professionals were four children how joined a total of five sessions, during which they were designing, discussing, drawing and experimenting with us in order to create an interesting and enjoyable game. The experience has been incredibly enriching both from the point of view of the research and the game design. We are very thankful to the “co-designer” children for their contributions!

For more informations visit: http://lab4.ccp.upf.edu/

Land’s of Fog

During 2015, I designed and facilitated a 4 sessions workshop for children with ASD to co-design the game Land’s of Fog. The game “Land’s of fog” was funded by the RecerCaixa 2013 grant (Feb/2014 to Jan/2016). The research project investigates how a full-body interactive environment can foster social initiation behaviours in children with Autism while interacting with typically developed children.

During the project I have been in charge of organizing and carry on the participatory design with 4 ASD children. In the different sessions of the workshop we worked together to design the environment of the game, its characters and their behaviors.

For more informations visit: http://inautistic.upf.edu/

Publications about the projects:

In case you want to know more information about the design process and methods you can read:

– Malinverni, L., Mora-Guiard, J., & Pares, N. (2016). Towards methods for evaluating and communicating participatory design: A multimodal approach.International Journal of Human-Computer Studies.

– Malinverni, L., Mora-Guiard, J., Padillo, V., Valero, L., Hervás, A., & Pares, N. (2016). An inclusive design approach for developing video games for children with Autism Spectrum Disorder. Computers in Human Behavior.

– Mora-Guiard, J., Malinverni, L., Pares, N., (2014) Narrative-Based Elicitation: Orchestrating Contributions from Experts and Children, in CHI ’14 Extended Abstracts Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems, Toronto, Canada

– Malinverni, L., Mora-Guiard, J., Padillo, V., Mairena, M.A., Hervás, A., Pares, N., (2014) Participatory Design Strategies to Enhance the Creative Contribution of Children with Special Needs, in Proceedings of the 2014 Conference on Interaction Design and Children, IDC2014, Aarhus, Denmark

Learning through Full-Body Interaction

Learning through Full-Body Interaction

Or learning through the body in technologically enriched spaces.

logo_phygame

For my PhD I’ve been researching on how novel interfaces may shape learning experiences. In particular my research focused around Full-Body Interaction interfaces, understood as interactive system that allow users to interact with digital technology through the use of their bodies and the physical space.  This kind of interaction opens promising possibilities given its capacity to involve the users at different levels, such as sensorimotor experience, cognitive aspects and affective factors.

If you want to know more about Full-body Interaction and its application in learning contexts you can read my paper: Malinverni, L., & Pares, N. (2014). Learning of Abstract Concepts through Full-Body Interaction: A Systematic Review. Journal of Educational Technology & Society, 17(4), 100-116. Available at: http://www.ifets.info/journals/17_4/7.pdf

 

Participatory Design: Sketching through the body

Participatory Design: Sketching through the body

Or how to co-design for embodied interaction with children.

Involving users in the design of their own technologies is both an ethical standpoint as well as a fundamental research approach.However, what happen when we have to design for innovative interfaces such as embodied interaction?

Together with Marie-Monique Schaper we carried out several studies aimed at exploring techniques  to design specific gestures with children to improve the interaction design of a Full-Body Interaction Learning Environment.

For this purpose we explored techniques proceeding from theatre, performance and visual art. The results of our research indicate the potential of those Participatory Design methods which combine multi-modal resources as instruments to allow children to reflect upon their own knowledge and express it more precisely.sketch

If you want to know more on how research you can read:

  • Schaper, M. M., Malinverni, L., & Pares, N. (2015, June). Sketching through the body: child-generated gestures in full-body interaction design. InProceedings of the 14th International Conference on Interaction Design and Children (pp. 255-258). ACM.
Body, Technology and Learning

Body, Technology and Learning

Or whether our bodies may help us in learning.

archim

During my Master project I’ve been working in the design and evaluation of an educational application aimed at supporting the learning of  Archimedes’ principle through concrete and embodied experiences. Results proceeding from an experimental study carried out with 331 children show modest but noticeable improvements in test scores from children that played with the embodied interface, thus suggesting potential design directions for the development of educational games

If you want to know more about the project, read: Malinverni, L., Silva, B. L., & Parés, N. (2012, June). Impact of embodied interaction on learning processes: design and analysis of an educational application based on physical activity. In Proceedings of the 11th International Conference on Interaction Design and Children (pp. 60-69). ACM.

Available at: https://www.researchgate.net/profile/Laura_Malinverni/publication/235652686_Impact_of_embodied_interaction_on_learning_processes_design_and_analysis_of_an_educational_application_based_on_physical_activity/links/0fcfd5124bbc792e79000000.pdf