User Research & UX Projects – Laura Malinverni

The World-as-Support: exploring AR/MR technologies

Posted on 18 October, 201618 October, 2016 by admin

Defining the project

Context: the project was oriented toward exploring the affordances of novel technological solutions for Augmented/Mixed Reality system. The project form part of the methodological research “Evaluation-Driven Design” , funded by Ministerio de Economia y Competividad and was developed at Universitat Pompeu Fabra

My role

In this research project I’ve been in charge of designing and carrying out the study, analyzing data and writing related reports

The research framework

The most widespread technological configurations for Augmented/Mixed Reality systems are based on smartphones and tablet devices and on the interaction paradigm known as Window-on-the-World (WoW). This paradigm provides the user with augmented information on the surrounding physical space by using the device’s screen to merge digital contents with the user’s view of the physical surroundings.

In contrast to this approach, we identify a novel and emerging conceptual paradigm to design ARMR applications for mobile technologies: the World-as-Support (WaS) interaction paradigm. This paradigm is based on projective augmented reality configurations which allow the users to augment the physical surroundings by intelligently projecting the digital information on the physical world.

In order to understand the potential of these paradigms we compare their usage in an ARMR application aimed at supporting the collaborative improvisation of site-specific narratives by children.

Comparing the Window-on-the-World and the World-as-Support interaction paradigm

Research goals: Explore the affordances that the two paradigms offer in terms of usage of the space and social interaction

Apparatus:

For the study we employed the “EspaiStory” application and tested under two conditions:

Window-on-the-World (WoW): Based on an Android tablet, the system provided a view of the physical environment through the tablet screen as a live feed from the camera of the tablet. The contents created by the children were displayed on the tablet screen as an additional layer
World-as-Support (WaS): Based on the combination of an Android tablet and a Philips PicoPix PPX3414 pico-projector bundled in a box and case. The system allowed children to carry around the device and project their own contents directly on the physical environment.

Participants: 35 children (8-9 years old)

Procedure: Children created their own character and subsequently the explored the school telling the story of the character

Data analysis: : To analyze children interaction with the device we employed a multimodal analytical approach focusing on the following resources paths of exploration of the space, interaction with the device, gaze directionality, gestures, group disposition, verbal utterances. From these data we modeled different perfiles related to the ways of using the space and the ways of interacting with others

Results

Usage of the space

Modeling the ways of using the space

Using the space: Comparing the two conditions

Interacting with others

Modeling ways of interacting with others

Interacting with others: Comparing the two conditions

Lessons learned

The two interfaces strongly shaped social relationship
The two interfaces promoted different ways of inhabiting and making sense of the physical/digital environment

Espaistory: defining design requirements for site-specific story-authoring tool

Posted on 10 October, 201618 October, 2016 by admin

“Espaistory” is an application aimed at supporting the collaborative authoring of site-specific narratives by children. The application has two main goals. On the one hand, it aims at scaffolding children’s interest and awareness toward their social and physical environment. On the other hand, the application aims at facilitating conditions for collaboration and co-creation of content. The application can be used to create fictional narratives or to work on content knowledge related to specific places (e.g. historical sites, their neighborhood, etc.).

My role

This project is currently being developed at Universitat Pompeu Fabra. In the project I’ve been in charge of researching, defining design requirements, working with users and coordinating the design and research team

Defining the project

Context: the project form part of the methodological research “Evaluation-Driven Design” , funded by Ministerio de Economia y Competividad

Goals:

enable children’s collaborative site-specific content creation
support coordination and collaboration
support embodied improvisation in storytelling

Target users: 7-10 years old children

Challenges

Develop a novel Augmented/Mixed Reality solution
Allow children to explore and create meaning of their social and physical environement

The initial research: combining theory, ethnography and Enactment workshops

Theory

Review of traditional approaches to support storytelling
Review of related works

Ethnography

Ethnographical analysis of children activities related to creating audiovisual stories

Context: 8 days, 4 schools (8 hours per school), 60 children

Method: field notes

Findings

The importance of choosing	allow children to choose between different sources (e.g. images for the background / characters / music..)
The importance of creating	allow children to design their project (plan & create their narrative) offer prompts to stimulate creative process
Balance planning and improvising	offer instances for brainstorming and planning allow space for improvisation show to children the potential of the system so they can think about “suitable ideas” allow them to repeat
the human-machine choreography as a way of working on collaboration	assign children to different roles each role should be relevant for the final outcomes the different roles need to properly coordinate between themselves in order to have everything working do not centralize coordination
the shared perspective	the space should be shaped in a way that all children need to be able to see what the others are doing and what they are creating
the value of self-evaluation	record their creation to allow them reflect upon it

Enactment workshop

Using embodied empathy understand user experience

Method:

Researchers enact what children will do during the experience
After that researchers discuss around relevant design questions to postulate design hypothesis

Defining requirements

Content Creation	User Experience	Interface
Wide wall: Afford different possible paths to support storytelling and content creation and enable children to get involved at different level of content creation Modulating guidance : offer prompts to stimulate creative process Offer instance for brainstorming and planning Allow space for improvisation Show to children the potential of the system	Low floor / High Ceiling: easy to start with; allow for “expert” usage Afford space/time for reflection/observation Afford collaboration and social learning Assign children to different roles	The space should be shaped in a way that all children need to be able to see well what the others are doing; what they are creating Record their creation to allow them reflect upon it

The initial prototype

The design is currently underdevelopment. For the initial prototype we designed a hardware solution based combination of an Android tablet and a Philips PicoPix PPX3414 pico-projector bundled in a box and case. At the same time we developed an initial Android application to facilitate content creation.

In this initial prototype, children could create their own contents (drawings or pictures) and display them in the physical environment. The system allowed children to carry around the device and project their own contents directly on the physical environment.

Between Bodies: comparing embodied interfaces

Posted on 10 October, 201618 October, 2016 by admin

BetweenBodies is Full-Body Interaction environment aimed at supporting the collaborative creation of drawings and to serve as a starting point for guiding children’s reflection on group dynamics and discrimination.

My role

I led the design of the BetweenBodies project as part of my PhD in the Full-Body Interaction Lab of University Pompeu Fabra. In this project I’ve been in charge of conducting interviews with experts, designing and carry out Participatory Design workshops with children, defining design requirements, analyzing and interpreting quantitative and qualitative data. The project has been realized in collaboration with Gustavo Hitscherich and Marie Monique Schaper.

The challenges

Collaboration and group dynamics are important learning goals during childhood. In this project we aimed at developing a playful application that can serve as as a starting point for guiding children’s reflection on group dynamics and discrimination. Its development should be based on making a meaningful use of embodied resources through the use of Full-Body Interaction interfaces.

The design process

The design process was structured in three stages:

An ideation stage aimed at eliciting and integrating requirements from experts and children
A preliminary evaluation stage where we tested the first prototype and compared its usage with the two different interfaces (Vertical Screen and Floor Projection).
The definition of design refinements

The ideation stage

Elicitation of requirements from experts	Children as partners in the design ideation stage
Goal: Delineate learning goals	Goal: Analyze how children understand and live discrimination and groups’ dynamics.
Method: Semi-structured interviews with experts	Method: Participatory design workshop, employing methods based on the Pictionary Activity and on Game Design activities.
Outcomes: the need of addressing aspects related to in-group or out-group perceptions (e.g. social identity and categorization of the others) the need of not speaking directly about cultural difference	Outcomes: The workshop allowed spotting out some relevant insights to guide the design of the experience. Specifically: The role of emotional facial and bodily expressions as a mediator of meaning construction The role of physical proximity to express similarity and difference The gesture of Pointing as discriminating The role of touch and physical contact as an embodied metaphor for collaboration

The initial prototype

From participation to design concepts

We decided to work with the following design concepts:

identity / categories
proximity
approach movement
synchrony

Furthermore, we decided to evaluate the suitability of two different Full-Body Interaction interfaces (Vertical Screen and Floor Projection).

The prototype

We developed an initial application was based on a back-story that describes the world of “Pimpis”, the inhabitants of a faraway planet. The children had to help the characters to rebuild their planet by freely drawing a novel environment. Children are presented an almost empty scenario, where only four characters of different colors were present. Each child can control one character by using a small hand-held lantern.To be able to draw, two children needed to bring their characters close to one another. When the two characters were sufficiently close to each other, a drawing line appeared. If the children now jointly moved through the space and maintained the physical contact between their characters, the drawing line followed their paths and they could draw whatever they wanted.

The preliminary evaluation

Goals:

comparing the two interfaces (Vertical Screen and Floor Projection)
identifying design refinements

Methods:

Questionnaires for assessing social perception
The analysis of children’s in situ interaction with the two systems according to a multimodal approach
The analysis of the outcomes of a Redesign workshop after the play experience.

Outcomes

Questionnaires

Multimodal analysis

Results showed that children assigned to the Floor Projection condition (Mdn =10.9) rated collaboration significantly higher than children assigned to the Vertical Screen condition (Mdn =4), U = 76.50, p < 0.001.

The analysis of the multiple embodied resources showed that:

In the Floor Projection, the children tended to experiment more with their bodies than in the Vertical Screen.
In the Floor Projection, the children explored the overall projected space. Instead, in the VS, the children tended to explore mainly the area where they were standing and most of them did not change their initial position in front of the screen.
In the Floor Projection, the children displayed a higher variability in the selection of the play partner, while in the VS the children mainly interacted with the partner who was standing beside them
In the Floor Projection, the children performed more complex synchronized behaviors.

Design refinements

The Floor Projection may represent a more appropriate interface to promote collaboration, negotiation of the tasks and discussion on group behavior.
Need for a critical reflection on the consistency between the proposed sensorimotor experience and the task of drawing
Need of creating a stronger boundary between the visual output and the narrative of the game.

A final version of the system is currently under development

Lessons Learned

importance of moving evaluation of comparative studies beyond verbal reports
need for a proper understanding of spatial and material affordances

Pico’s Adventure: A Kinect Game to Promote Social Initiation in Children with Autism Spectrum Condition

Posted on 24 September, 201618 October, 2016 by admin

Pico’s Adventure is a Kinect game for children with autistic spectrum disorder, aimed at promoting social initiation. The project was part of the European Project “M4all: motion- based adaptable playful learning experiences for children with motor and intellectual disabilities” and developed in the collaboration between the Universitat Pompeu Fabra and the Specialized Unit on Developmental Disorders of the Hospital Sant Joan de Déu.

My role

During my Phd I have been working in the design and evaluation of the game “Pico’s Adventure” . Specifically, I focused in the requirements elicitation, participatory design, evaluation and reporting.

Defining the project

Context: European Project “M4all: motion- based adaptable playful learning experiences for children with motor and intellectual disabilities”

Goal: Develop a Kinect-based game for children with Autistic Spectrum Condition aimed toward promoting social initiation, understood as the promotion of behaviors such as approaching and looking for others, trying to start social communication and producing any verbal or gestural behavior for communicative goals.

Target users: 4-6 children with Autistic Spectrum Condition

The concept: THE GAME AS A MEDIATOR OF SOCIAL COMMUNICATION = Design situations that:

Are valuable for the children to communicate about them
Integrate therapeutic techniques used to facilitate social initiation
Require the child to look for external collaboration

Challenges

Make an experience that is relevant for children and effective in terms of therapeutic goals
Orchestrate requirements from therapists and children’s interests
Test the therapeutic effectiveness of the developed game

The Design Approach

Elicit and combine requirements from therapists and children

Approach: Narrative- based elicitation to combine narrative structure and therapeutic techniques

Method: Participatory Design with Children and Focus Groups with therapists

Iterative Design and Evaluation : field notes, video analysis, questionnaire for stakeholders feedbacks

The Game

During the game, the children will be introduced to a fantasy world inhabited by a friendly mascot (the virtual agent), named “Pico”, who will accompany them in the journey toward social initiation. Children – either alone or in collaboration with adults and peers – will have to help the mascot, an amicable alien, who landed on our planet Earth after having some problems with its spaceship. Children will have to help it in overcome different missions, each one designed to address a targeted behavior related with social initiation. Within that, major emphasis will be posed on designing game situations that require the child to seek for the collaboration either of an adult or a peer.

The evaluation

The game has been evaluated in two stages: a first exploratory stage aimed at evaluating its acceptance by the child, and a second experimental study oriented at assessing the effectiveness of the game in trigger target behaviors related with social initiation.

Experimental study

Population: The study involved 15 boys with ASD between 4 and 5 years old.

Goals: In the study we compare social interaction between free-plays activities and games activity

Methods: For the analysis we employed systematic video-coding by focusing specifically on: Social requests, Social initiation, Responses, Gestures

Results

Results show a significant increase of social initiation behaviors in 1st and 4th game sessions compared to free play (p< 0.05)

Lessons Learned

The use of narrative resources showed to be an effective mediator between different disciplines, framework and generations.
We develop a set of effective participatory design methods for populations with special needs
Effectiveness of immersive, bodily-based interaction to support social initiation and collaborative behaviors

Related publications

Malinverni, L., Mora-Guiard, J., Padillo, V., Valero, L., Hervás, A., & Pares, N. (2016). An inclusive design approach for developing video games for children with Autism Spectrum Disorder. Computers in Human Behavior.

Malinverni, L., MoraGuiard, J., Padillo, V., Mairena, M., Hervás, A., & Pares, N. (2014, June). Participatory design strategies to enhance the creative contribution of children with special needs. In Proceedings of the 2014 conference on Interaction design and children (pp. 85-94). ACM.

Mora-Guiard, J., Malinverni, L., Pares, N., (2014) Narrative-Based Elicitation: Orchestrating Contributions from Experts and Children, in CHI ’14 Extended Abstracts Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems, Toronto, Canada

THE GAME “PICO’S ADVENTURE” IS AVAILABLE FOR DOWNLOADING FOR WINDOWS 7 or WINDOWS 8 in the following webpage: http://m4all.widetesting.info/

Lands of fog: Participatory Design with Autistic Children

Posted on 16 September, 201618 October, 2016 by admin

“Lands of Fog” is an open-ended Full-Body Interaction Environment designed for children with Autism Spectrum Disorder (ASD) to play with a typically developed peer. The experience has the goal of promoting exploration and of scaffolding social and collaborative behaviors between users. The project was founded by RecerCaixa grant.

My role

During my Phd I have been collaborating in the design of the experience. Specifically, my role focused on defining, carrying out and analyzing participatory design activities with children with Autism Spectrum Condition

The project

Goal: Develop a Full-Body Interaction experience for children with Autistic Spectrum Condition aimed toward promoting social initiation, understood as the promotion of behaviors such as approaching and looking for others, trying to start social communication and producing any verbal or gestural behavior for communicative goals.

Target users: 10-12 years old children with Autistic Spectrum Condition

The concept:

The game as a mediator of social interaction
Support collaboration without enforcing it

Challenges

Involve autistic children in the design of an interactive experience
Define methods to facilitate children expressing their own interests and understandings around the experience
Define analytical approaches capable of taking into account meanings and aspects that cannot be expressed by words

Children as partners in the design ideation stage

In the initial stage of development, we conducted a PD workshop, based on the “children-as-informants” model with four children with ASD (mean age: 11). During the workshop, children we employed a Wizard-of-Oz prototype of the experience and carried out a set of specific activities.

The explorer activity: children in pairs were invited to interact with the prototype and take pictures of anything that captured their interests
The detective activity: children who were not playing were invited to watch their playing peers and try to guess what are they doing
The designers activity: children were asked to make drawing to depict how they would like to improve the environment

The initial prototype

The system was based on a 6-meter diameter floor projected virtual environment, which is covered by virtual fog. Users explore and interact with the environment using a butterfly net that allows them to open peepholes in the fog and discover what is hidden underneath.

A multimodal approach to analyse children’s contributions

To analyze these activities, we employed multimodal analysis and focused on:

children’s exploration of the space (their position, paths, pauses and relative speed),
their gaze and the pictures they took during the interaction with the system
their verbal interactions
their drawings and related explanations

Data were annotated on map for each child and analyzed according to a grounded approach

Findings

Children were particularly interested in detailed and liminal areas and in exploring the fog appearance
They tended to interpret the underlying environment as a geographical map
The act of moving around with the net was associated with the task of hunting or collecting something according to a strictly functional perspective (e.g. “You have to collect animals to make points”)
Strong influence of videogames culture

Informing design

The performed analysis allowed us to identify some relevant aspects to be addressed in future design iterations:

the use of the virtual fog showed to be an effective design choice to promote exploration. Future design iterations can eventually address the refinements of the fog’s behavior to enrich the possibilities for sensorimotor exploration
a realistic and videogame-like graphical style may be adequate to capture children’s attention and foster exploratory behaviors.
children’s tendency to interpret the environment as a large space (i.e. a continent) pointed out some possible issues related to defining an adequate scale of the experience
children’s lack of interest toward their peers requires a strong effort to promote more collaborative ways of interacting during the experience. To tackle this latter issue, possible design solutions can use videogames mechanics as motivators of social interaction.

Lessons Learned

Multimodal analysis showed to be an effective approach to analyze users contributions across multiple sources and inform design
Relevance of analyzing users’ understandings across multiple modalities

Related publications

Malinverni, L., Mora-Guiard, J., & Pares, N. (2016). Towards methods for evaluating and communicating participatory design: A multimodal approach.International Journal of Human-Computer Studies.

Ecosystem: design and evaluating a Full-Body Interaction Learning Environment

Posted on 10 August, 201618 October, 2016 by admin

EcoSystem is a Full-Body Interaction Learning Environment for children aimed at supporting learning about environmental relationships through embodied interaction

My role

I led the design of the EcoSystem project as part of my PhD in the Full-Body Interaction Lab of University Pompeu Fabra. In this project I’ve been in charge of conducting interviews with experts, designing and carry out Participatory Design workshops with children, defining design requirements, analyzing and interpreting quantitative and qualitative data. The project has been realized in collaboration with Marie Monique Schaper.

The challenges

EcoSystem is a Full-Body Interaction Learning Environment for children aimed at supporting learning about environmental relationships through embodied interaction. The main challenges of the projects were related to:

Defining appropriate learning goals
Designing an environment based on Full-Body Interaction that can serve as an effective complement to traditional educational methods
Evaluate the effectiveness of the learning environment

The design process

The design process was structured in three stages:

Stage 1: Children as co-designers

This first stage of the study had the goal of defining the educational needs through the use of Participatory Design methods.

Stage 2: Children as informants

The second stage of the study had the goal of analyzing children’s interaction and interpretation of the first prototype.

Stage 3: Children as testers

The third stage of the study aimed at evaluating the effectiveness of the designed FUBILE in supporting learning and comparing its use with traditional instructional methods.

Stage 1: Children as co-designers

Goal: The goal of the first stage was to define specific learning goals according to experts’ requirements and children’s understandings, previous knowledge and representations of concepts related to environmental education.

Methods:

Open-ended interviews with experts
Participatory Design (PD) workshop with children, using methods based on the Pictonary Activity and on Game Design activities.
Instruments: video-analysis

Results:

Understanding of children’s previous knowledge
Definition of bridging concepts (concepts that can bridge the gap between what children already know and novel knowledge)
Identification of misconceptions and knowledge gaps

Outcomes:

Definition of learning goals related to the system dynamics of the relation between the amount of pollution and the resources available to reduce or augment it.
Design of an initial prototype

First prototype: The EcoSystem Project

Children were presented with a large floor projection of the interactive environment. They have to teduce the amount of air-pollution in the environment. To do that they can assume different roles:

Wind power: producing wind energy to replace the energy produced by a steam power station and hence reduce air-pollution,
Plants: growing plants and vegetables to facilitate CO2 absorption and contribute to the consumption of local foods,
Recycling: recycling and producing compost to increase the growth of plants and reduce waste that would otherwise end up incinerated and hence increasing combustion.

Air-pollution was visualized as a cloud covering the game ground. Thus, as pollution increased, the amount of space available for playing decreased. In order to make the cloud decrease children needed to understand the relation between the elements present in the game and properly collaborate with other players.

Stage 2: Children as informants

Goal:

Analyze how children interact with the prototype and understand it
Define design refinements

Methods:

Video analysis of children’s interactions while playing with a Wizard of Oz prototype of the system
Retrospective analysis of children’s understanding of the system
Instruments: video-analysis, open-ended questionnaire, conceptual map, semi-structured group discussion and a short essay.

Results

Identification of misconceptions provoked by the design of the system
Identification of core meanings that guided children understandings
Identification of intuitive actions for embodied interaction

Outcomes

Definition of design refinements: changes in the spatial layout, change in visualization of interactive elements, definition of gestures for interactions

Stage 3: Children as testers

Goals: evaluating whether the designed FUBILE supports learning and testing whether it could be effective to complement Traditional Instructional Methods (TIM).

Methods: Between-subject experimental design:

1) Experimental condition: combined use of the system and traditional text-based learning materials

2) Control condition: use of text-based learning materials alone

Instruments: pre and post-test conceptual maps

Results

Significant learning gains for both conditions between pre and posttest for both conditions
Significant different between the two conditions in posttests results: children in the experimental condition reported significantly greater scores than children assigned to the control condition

Lessons Learned

Importance of carefully structure iterative design processes
Importance of methods to understand children previos knowledge, representations and core meanings
Relevance of experiential learning

Related publications

Malinverni, L., Schaper, M.-M., and Pares, N. (2016). An evaluation-driven design approach to develop learning environments based on full-body interaction. Educational Technology Research and Development. DOI=http://dx.doi:10.1007/s11423-016-9468-z

Schaper, M.-M., Malinverni, L. and Pares, N. (2015). Sketching through the body: child-generated gestures in Full-Body Interaction Design. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC ’15). ACM, New York, NY, USA, 255-258. DOI=http://dx.doi.org/10.1145/2771839.2771890

Schaper, M.-M., Malinverni, L. and Pares, N. (2014). Participatory design methods to define educational goals for full-body interaction. In Proceedings of the 11th Conference on Advances in Computer Entertainment Technology (ACE ’14). ACM, New York, NY, USA, Article 50. DOI=http://dx.doi.org/10.1145/2663806.2663867

Archimedes: Evaluating a Full-Body Interaction Learning Environment

Posted on 10 June, 201610 October, 2016 by admin

Archimedes is a Full-Body Interaction Learning Environment based on the Interactive Slide platform, a large inflatable slide augmented with digital technology. Archimedes was originally designed to support children’s hands-on learning on the notion of buoyancy and Archimedes principle. For its evaluation, we worked with 48 children from a local school (mean age: 11). Specifically, we carried out a qualitative, idiographic study aimed at exploring how children create bridges between embodied experience and meaning construction while interacting with a Full-Body Interaction Learning Environment. For this purpose, we analyzed children’s activity during the game and post-task interviews. Subsequently video were analyzed according to a multimodal approach. This analysis allowed to identify the possible paths through which children can transform embodied experience into an ‘object–to-think-with’ and delineate the different resources for meaning making that they employed. Furthermore, it pointed out the importance of reflection-in-action and how these reflective moments are embedded in the experiential flow and their embodied nature is displayed in users’ bodily actions, pace and use of the space (e.g. adopt an “observer position”, turn around to see the screen, etc.), and selective engagement. These outcomes contribute to expand the current understanding around embodied learning with interactive technologies, as well as suggest a set of qualities to think about interaction design and future research.

Related Publications:

Malinverni, L., Ackermann, E., & Pares, N. (2016, February). Experience as an Object to Think with: from Sensing-in-action to Making-Sense of action in Full-Body Interaction Learning Environments. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 332-339). ACM.