Jojo is an AR application that facilitates alphabet learning for kids with dyslexia via gamification.The app uses a multisensory learning approach by incorporating visual, and auditory cues simultaneously to enhance memory and learning of written language. Links are consistently made between the visual (language we see) and auditory (language we hear) pathways in learning to read and spell.
Role and Responsibilities
AR / VR Designer & Developer
- Designed and Developed the gamified learnability features for Jojo.
- Performed a literature review and scholar research to understand the best available approaches to learning for kids with dyslexia.
Tools
Unity, AR Foundations, Aframe.JS
Duration
1 month (April'20 - May '20)
01. OVERVIEW
Problem
Dyslexia is a Specific Learning Difficulty of neurobiological origin that affects those linguistic skills associated with writing, reading, short-term memory, perception of order and sequencing. Dyslexia affects approximately 15% of children and is responsible for 40% of school failure leading to a series of emotional disorders. Dyslexic children learn differently but are forced to follow an educational system that does not enhance their strengths or motivate them.
Learning challenges in traditional teaching
Learning disability refers to group of disorders that causes difficulties in several academic abilities (e.g. speaking, reading, writing or working with numbers. Based on our initial research, we figured out that there are three main learning challenges when traditional teaching methods are used for kids with dyslexia.
Unintuitive learning
Lack of fun elements in books such as multimedia elements and only having static images and less interaction with students who have dyslexia
Lack of learning support
Teachers and administrators cannot provide one-on-one attention that would be a more nurturing and appropriate learning environment for students with dyslexia.
Lack of sensory interventions
Dyslexic kids require Interventions incorporating a multi sensory application techniques paired with a systematic approach to learn which is scarce in traditional teaching.
tl;dr
How might we imagine an alphabet learning app for kids with dyslexia that supports their needs for a multi-sensory learning without an instructor?
Process
02. RESEARCH
Literature Review
To understand the problem and needs of the children, we did a comprehensive literature review of the technologies and solutions that support development for dyslexic children. I’d highlight our major findings inspired by the paper from the works of Prof. Luz Relloin in her venture changedyslexia.com.
Insights
Augmented objects for a better textbook experience
AR can be used in order to enhance the interaction among real world and digital objects for dyslexic students, this helps them to understand the context and the usage of words by connected them with the objects they relate to,
Phonetics for a multi sensory approach
A multi sensory approach induces better learning by adding narration and phonetics to the traditional textbook experience, this would help the students build connections between spoken and written words.
Learning to write alphabets via storytelling
Storytelling reinforces the connection between the words, for instance an animated video of an alphabet, with phonetics helps to develop the understanding of the alphabet shapes.
Gamification for testing and feedback
Introducing a gamified approach where identifying the alphabets using the same objects used for learning is an intuitive way to test learning as opposed to the traditional tests.
Current teaching techniques
I did a comprehensive research on the current techniques used by professional trainers to teach dyslexic students. These videos helped the team understand what techniques and multi-sensory approaches are used by educators and how learning for dyslexia requires a more joyful and interactive approach.
Key Takeaways
- Multisensory learning exercises involving sounds, touch and visuals.
- Identifying the specific learning difficulty and devising strategies to help with them.
- Colourful, organised and inviting environment without distractions
- Exercises and games based on the students interests helps develop the learning
- Give information in small chunks, with positive reinforcement in each stage.
03. DESIGN
Ideating on an AR application
Based on our initial research, our team planned to ideate on an Augmented Reality application that will cater to the described needs of kids with dyslexia. Using AR as a learning tool would enable users to see and experience the real world mixed with virtual objects, without losing the sense of reality. This tool can benefit dyslexic students and educators since it can transform the learning procedures and make them more stimulating and entertaining.
The initial idea was to develop a multi sensory application that can be used by kids with minimal support from the educators. Using visual, audio and tactile interactions that were identified important in the learning from our research, we brainstormed on the feature list of the application.
Sketching
I sketched out some ideas keeping the learning techniques and technology feasibility in mind. The team decided to create a holistic
experience by including a three step learning process.
1. The first part of the application will focus towards learning by associating the alphabets with the real world objects.
2. The second part will focus on retention and re-enforce the learning. To make it fun and engaging, we decided to design simple identification games that will make use of audio, visual and tactile experiences.
3. The final step of the application will be to test the learning, we decided to keep a drawing application. To further make it an immersive experience, we decided to go with spacial drawing using a marker based technology.
User Flows
Based on the sketches and initial ideation, I developed a basic user flow to lay out the interactions and the different states of the application. The user flow helped to break down the application into several MVPs that were built separately.
.svg)
Technology decisions
We decided to develop the application on Unity. The prototypes were developed with 3d models. The touch interactions were developed using the Lean touch packages on Unity.
The application made the use of Vuforia engine as we decided to use a marker based technology. The application was deployed on IOS and Android and tested on both mediums to ensure accessibility.
Application sketches
I sketched basic applications sketches before jumping into the development. The sketches helped to ensure that the minimum requirements of the user flow is met. Due to time constraints the team decided to jump to development and carry out usability tests after the prototype was developed.
Final Prototype
Coming soon...
.png)