There is currently no specific treatment for Autism, with educational interventions being the mainstay, supplemented by medication.
AI-Based Game (Serious Game) - Emogym
Role
Date
Team planner, Design decision, UI/UX, Code assistance
2022
Overview
What Is Emogym?
Emogym is a computer-based training tool which is developed as an intervention to help children with autism spectrum condition (ASC) improve their facial emotion recognition and expression ability, as well as social skills.
About Autism
Autism is a severe and widespread childhood neurological disorder, and the number of patients is increasing year by year. There is currently no specific treatment for autism, with educational interventions being the mainstay, supplemented by medication. Children with autism spectrum condition (ASC) have some delays in developing facial emotion recognition and expression. Although the optimal treatment age for autism is before the age of 6, thehigh cost of treatment or parental neglect has resulted in less than 33% of children with autism currently being seen. Although there are many apps on the market to help rehabilitate children with autism, research has shown that most of these train facial muscles and emotion recognition, with only few measures focusing on improving facial emotion recognition and expression.
Design Process
Discover & Define
Available Treatments for Autism
Recognition and expression of facial emotions are two abilities that can be trained simultaneously: people with autism need to recognise emotions expressed by others and then imitate them further. However, building trust and talking to children with ASC face-to-face for long periods is a daunting task, and many therapists take a long time at this step. At the same time, the high cost of treatment is prohibitive for many families. As a result, many families are unable to afford long-term treatment, which has led to a further decrease in the attendance of children with autism.
In recent years, with the advent of technology, electronic mobile devices have become an integral part of everyday life. Many apps are designed for children with autism. Research has shown that children with autism are more relaxed and feel less stressed when confronted with electronic screens or robots than people. Moreover, with technological interventions, there has been a significant improvement in children's communication and expression skills with ASC.
Users' Pain Points
Children with ASC have difficulties recognising and expressing facial emotions, significantly affecting their ability to communicate.
It takes a long time for children with ASC to build trust with others and be trained to talk.
The cost of treatment for autism is too high that many families have to abandon treatment for their children.
There is a lack of interventions available for facial emotion recognition and expression training.
Our target users are children with autism who have developmental impairments in emotional regulation, with the main age group being 5-10 years old
Target Users & Persona
“ When I'm feeling nervous, I laugh uncontrollably. This often leads many people to misunderstand me, but I just can't control my facial expressions. ”
Peter is a seven-year-old boy. He has an autism spectrum disorder, which has led to a delay in developing his innate emotional regulation. As a result, he cannot help but laugh as a mental defence when he has negative emotions, often making him misunderstood. Recently, Peter’s family has been relocated to another city, but Peter is still unable to accept this and cannot even tell the difference between moving and being homeless.
Perter, 7 years old, Child with ASD
Problem Space
How to design an affordable AI-based system to help train children with autism in daily facial expression recognition and expression.
Idea Generation
After conducting background research and identifying the problem space, our group undertook a round of brainstorming in which we generated a number of different design solutions from different perspectives in order to explore potentially better solutions. After 6-3-5 brainwriting, we developed 4 initial concepts, which we then evaluated and voted on to decide which one we would like to develop.
Usability Testing
Participants felt that the cool colours of the interface gave the user a sense of distance and did not attract the attention of children with autism.
Imitating emoji emojis for training purposes was not considered feasible as respondents felt that emoji emojis were too exaggerated and out of touch with reality.
Some participants felt that imitating static emoji pictures was too difficult for those children with autism, who needed to see the process of expression change to imitate it.
Respondents felt that asking people with autism to choose the right emoji based on words did not improve communication skills.
Live 2d & Paint tool Working Process
- animations of expressions
Coding Developing Process & System Architecture
Data Training Process
Final Concept
Emogym
Why Emogym
We have designed Emogym to functionally meet the needs of children with autism who need to be trained to recognise and express facial expressions simultaneously. The user can select and distinguish the expressions in the pictures through different scenarios and, at the same time, facilitate their thinking about how to express their emotions. On the subsequent training page, users can imitate these expressions, thus further training their facial muscles and proficiency with these expressions. Secondly, our product is an inexpensive training system that allows children with autism to do their daily training at home without paying a high fee and with a parent in attendance.
Our interface design uses highly saturated colours and cute cartoons to attract children to interact. The system also plays music while the user trains to help them focus and improve their training
Features
Emogym includes 4 emotions (Angry, Sad, Happy, Fear), each with 3 different expressions(12 expressions in total), 4 animated character gifs, descriptive texts and songs for music therapy for emotions. Facial expression detection also provided for the generation of user training status scores.
Design Concept - Front Page
Emogym's design style focuses on rich colours and cute cartoon images. Research has shown that children with autism are more attracted to rich colours. They also show more patience and interest when confronted with cartoons. Cartoons also help children with autism to focus quickly and without stress(Pathak, 2021). When the user clicks on start, soothing music is played on the web to help the child focus.
Design Concept - Question Page
In this interface, the user is given open-ended questions, assuming many different scenarios in life, and is asked to pick the emotion they want to express based on these questions. In this way, we hope that children with autism will be able to relate the different emotions to real-life scenarios and thus train their social skills.
The four options represent the four basic emotions (happiness, fear, sadness and anger) represented by cartoon pictures. Research has shown that pictures can help children with autism learn to communicate better than words(Pathak, 2021).
Design Concept - Training Page
Once the user has selected the emotion he wants to express, they will be taken to the training page, where a cartoon image on the left side of the screen will show how to express an emotion, with a text description at the top. Then, the user has to mimic the expression to exercise the facial muscles and express the emotion.
During the imitation process, the AI will evaluate the level of imitation in the background and comment. When the user successfully mimics the emoji will appear around their face with the corresponding emotion, and they will be reminded to click the following button on their phone to move on to the next question.
