Jojo

An Augmented Reality based multisensory alphabet learning app for kids with dyslexia

About

Jojo is an AR application that facilitates alphabet learning for kids with dyslexia via gamification.The app uses a multisensory learning approach by incorporating visual, and auditory cues simultaneously to enhance memory and learning of written language. Links are consistently made between the visual (language we see) and auditory (language we hear) pathways in learning to read and spell.

Role and Responsibilities

AR / VR Designer & Developer
  • Designed and Developed the gamified learnability features for Jojo.
  • Performed a literature review and scholar research to understand the best available approaches to learning for kids with dyslexia.

Tools

Unity, AR Foundations, Aframe.JS

Duration

1 month (April'20 - May '20)

Team

Toshi Gupta

01. OVERVIEW

Problem

Dyslexia is a Specific Learning Difficulty of neurobiological origin that affects those linguistic skills associated with writing, reading, short-term memory, perception of order and sequencing. Dyslexia affects approximately 15% of children and is responsible for 40% of school failure leading to a series of emotional disorders. Dyslexic children learn differently but are forced to follow an educational system that does not enhance their strengths or motivate them.

Learning challenges in traditional teaching

Learning disability refers to group of disorders that causes difficulties in several academic abilities (e.g. speaking, reading, writing or working with numbers.  Based on our initial research, we figured out that there are three main learning challenges when traditional teaching methods are used for kids with dyslexia.
Unintuitive learning
Lack of fun elements in books such as multimedia elements and only having static images and less interaction with students who have dyslexia
Lack of learning support
Teachers and administrators cannot provide one-on-one attention that would be a more nurturing and appropriate learning environment for students with dyslexia.
Lack of sensory interventions
Dyslexic kids require Interventions incorporating a multi sensory application techniques paired with a systematic approach to learn which is scarce in traditional teaching.

tl;dr

How might we imagine a learning app for kids with dyslexia that supports their needs for a multi-sensory learning without an instructor?

Process

Solution

02. RESEARCH

Literature Review

To understand the problem and needs of the children, we did a comprehensive literature review of the technologies and solutions that support development for dyslexic children. I’d highlight our major findings inspired by the paper from the works of Prof. Luz Relloin in her venture changedyslexia.com.


Insights
Augmented objects for a better textbook experience
AR can be used in order to enhance the interaction among real world and digital objects for dyslexic students, this helps them to understand the context and the usage of words by connected them with the objects they relate to,
Phonetics for a multi sensory approach
A multi sensory approach induces better learning by adding narration and phonetics to the traditional textbook experience, this would help the students build connections between spoken and written words.
Learning to write alphabets via storytelling
Storytelling reinforces the connection between the words, for instance an animated video of an alphabet, with phonetics helps to develop the understanding of the alphabet shapes.
Gamification for testing and feedback
Introducing a gamified approach where the understanding of alphabets using the same objects is an intuitive way to test their learning as opposed to the traditional tests.

Current teaching techniques

I also did a research on the current techniques used by professional trainers to teach dyslexic students. These videos helped me and my teammate understand what techniques are making the use of multi-sensory approaches and what other things we needed to keep in mind for our ideation.



Basic Principals
  • Multisensory learning exercises involving sounds, touch and visuals.
  • Identifying the specific learning difficulty and devising strategies to help with them.
  • Colourful, organised and inviting environment without distractions
  • Exercises and games based on the students interests helps develop the learning
  • Give information in small chunks, with positive reinforcement in each stage.

03. DESIGN

Ideating on an AR application

Based on our initial research, we planned to ideate on an AR application that will cater to the described needs of kids with dyslexia. We planned to sketch a multi sensory application that can be used by kids to learn a language.

Sketching

User Flows