DISSEMINAZIONE

Persons with Autism Spectrum Disorder (ASD) experience, in general, cognitive rigidity and behavioral and communication problems. These issues dramatically affect their everyday life abilities, such as interacting with others or being autonomous in most daily tasks. 5A project aims at supporting ASD individuals by exploiting the combination of Virtual Reality (VR), Augmented Reality (AR), and Conversational Agents (CA). Using low-cost headsets and smartphones, 5A offers a unique intervention for ASD people. The VR training allows the user to learn how to perform daily tasks in a safe and controlled environment. The AR experience enables the generalization of the acquired skills in real-world settings. Finally, the customizable CA assists and prompts the user at every step in both experiences. As a first use case for 5A applications, we report its use in urban mobility with trains. A case study evaluation was conducted, measuring the effect of 5A apps on 5 ASD users. Our preliminary results showed that users had improved their autonomy level (perceived and measured by external caregivers). The first lessons learned by this study are also reported as a set of guidelines about technology usage, design, and experimentation for immersive technologies in support of persons with ASD.

Mattia Gianotti, Alberto Patti, Francesco Vona, Francesca Pentimalli, Jessica Barbieri, and Franca Garzotto. 2023. Multimodal Interaction for Persons with Autism: the 5A Case Study.  To be appear in Proceedings of the 25TH INTERNATIONAL CONFERENCE ON HUMAN-COMPUTER INTERACTION (HCI INTERNATIONAL 2023)

Cognitive rigidity and behavioral and communication problems are the most common symptoms that characterize Autism Spectrum Disorder (ASD). In fact, people with ASD face complications in everyday life, from interacting with other persons to being autonomous in the most ordinary daily tasks. To overcome these limitations and improve their quality of life, we developed 5A. 5A exploits cutting-edge technologies such as Virtual Reality (VR), Augmented Reality (AR), and Conversational Agents (CA), combining smartphones and wearable headsets to offer new forms of intervention to persons with ASD. Virtual reality experiences enable learning common tasks, such as using public transport, through simulation in a safe environment. When people are in similar real-life situations, AR can help them generalize the skills they have learned in VR by overlaying interactive media assets on top of the view of the physical world around them. A customizable conversational assistant provides individualized prompts and feedback in both VR and AR to provide maximum user support.

Francesco Vona, Mattia Gianotti, Alberto Patti, Jessica Barbieri, Lukasz Moskwa, Francesco Riccardo Di Gioia, and Franca Garzotto. 2022. Combining Virtual and Augmented Reality to improve daily autonomy for people with Autism Spectrum Disorder. In Proceedings of the 2022 International Conference on Advanced Visual Interfaces (AVI 2022). Association for Computing Machinery, New York, NY, USA, Article 74, 1–3. https://doi.org/10.1145/3531073.3534499

Autism Spectrum Disorder is a developmental disability characterised by cognitive rigidity and communication and behavioural deficits. These deficits negatively impact the lives of people with ASD by making them unable to complete daily activities and even communicate with others. In order to support persons with ASD in their everyday life, facilitate their social inclusion, and improve their quality of life, we developed 5A. 5A provides innovative tools based on smartphones and wearable headsets that integrate Virtual Reality (VR), Augmented Reality (AR), and Conversational Agents (CA). These applications help persons with ASD understand the environmental and socio-organizational characteristics of everyday life scenarios and correctly perform related activities. Experiences in VR allow simulating typical tasks of such scenarios, e.g., taking the subway. When people find themselves in similar situations in real life, AR helps them generalise the skills acquired in VR by superimposing interactive multimedia elements on the view of the surrounding physical world. In both VR and AR, a personalised Conversational Assistant provides prompts and personalised feedback.