2015 Cover Page

iSTEP 2015: Cross-Cultural Technology Development Toward Language Access for the Deaf and Hard of Hearing
Maya Lassiter, Amal Nanavati, Erik Pintar, Minnar Xie, Ermine Teves, M. Bernardine Dias
CMU-RI-TR-16-32
The Robotics Institute Carnegie Mellon University Pittsburgh, Pennsylvania 15213
June 2016
Copyright © 2016 TechBridgeWorld at Carnegie Mellon University

iSTEP 2015: Cross-Cultural Technology
Development Toward Language Access for
the Deaf and Hard of Hearing
Maya Lassiter, Amal Nanavati, Erik Pintar, Minnar Xie, Ermine Teves, M. Bernardine Dias
ACKNOWLEDGEMENTS
Partner involvement has been essential for this work, and we were fortunate to collaborate with the Mathru Educational Trust for the Blind and their Center for the Deaf and DifferentlyAbled in needs assessment and user testing. The authors especially thank Ms. Gubbi R. Muktha and Mathru Center staff members for making our projects possible. Furthermore, many individuals participated in our needs assessment, generously volunteering their time. Because of research compliance procedures, we cannot name these participants individually, but we are grateful for their involvement and enthusiasm for the project. We also thank Bernardine Dias and Ermine Teves from the TechBridgeWorld research group for their guidance throughout the internship. We would also like to thank our sponsors who made the iSTEP 2015 internship possible, including individual donors and [email protected] donors.
ABSTRACT
This paper summarizes the work done over the course of nine weeks at the Mathru Center for the Deaf and Differently-Abled (Mathru Center) in Bangalore, India and aims to provide recommendations regarding technology solutions for deaf and hard of hearing students and their teachers in underserved communities. This work was carried out by a team of four undergraduate research students participating in an internship program called iSTEP – short for innovative Student Technology ExPerience (iSTEP) –organized by the TechBridgeWorld research group at Carnegie Mellon University. From the team’s work in designing and developing educational tools with student and teacher participants, this paper explores the practice of creating shared ownership over technological ideas through direct relationship with the target population and viewing them as equals in the process. The team’s partnership and work with the Mathru Center resulted in two newly created educational tools aimed at providing language access for Deaf primary school students and their teachers. The first, SignBook, provides greater sign language access through a sign language dictionary creation tool with abilities for custom video and picture capturing and categorization of entries and words into topics. The second, Speak Up!, motivates students to strive for greater verbal language access through a suite of voice-powered games aimed to familiarize pre- and partially-verbal users with the power of their voice.
1

TABLE OF CONTENTS
Acknowledgements.............................................................................................................................. 1 Abstract.................................................................................................................................................... 1 Table of Figures.....................................................................................................................................3 List of Tables .......................................................................................................................................... 3 Introduction ........................................................................................................................................... 4
TechBridgeWorld............................................................................................................................................ 4 iSTEP.................................................................................................................................................................... 4 The Mathru Educational Trust for the Blind.......................................................................................... 5 Information Communication Technology for Development in India ............................................ 5 Culture and Community ................................................................................................................................ 6 iSTEP 2013 Findings....................................................................................................................................... 8 Motivations........................................................................................................................................................ 9
Language Access Tools..................................................................................................................... 10
Study Setting ...................................................................................................................................................10 Study Participants.........................................................................................................................................11 Findings and Development ........................................................................................................................11
SignBook: A Sign Language Dictionary................................................................................................................ 12 Speak Up! Voice Powered Game Suite ................................................................................................................. 20
Discussion ............................................................................................................................................ 26 Additional Considerations for Future Work ............................................................................ 27 References ........................................................................................................................................... 30 Appendix .............................................................................................................................................. 33
Our Findings at the Mathru Center classroom for Students with Multi-Sensory Impairments (MSI) .......................................................................................................................................33
2

TABLE OF FIGURES
FIGURE 1. THE MATHRU CENTER FOR THE DEAF AND DIFFERENTLY ABLED...........................................................................................5 FIGURE 2. FLASHCARDS USED AT THE SCHOOL................................................................................................................................................8 FIGURE 3. FIRST GRADE SPEECH CLASS ......................................................................................................................................................... 11 FIGURE 4. A SAMPLING OF AMERICAN SIGN LANGUAGE WORDS WRITTEN WITH SIGNWRITING ........................................................ 14 FIGURE 5. “MY NATIVE TONGUE – 1” THE ONLY OFFICIAL SIGN LANGUAGE REFERENCE BOOK AT THE MATHRU CENTER ........... 16 FIGURE 6. AN INTRICATE DRAWING BY A TEACHER, USED TO EXPLAIN A CLASSROOM LESSON IN THE FIRST GRADE CLASSROOM. 17 FIGURE 7. THE SIGNBOOK INTERFACE........................................................................................................................................................... 18 FIGURE 8. THE SPEAK UP! MAIN MENU......................................................................................................................................................... 24 FIGURE 9. A MATHRU CENTER STUDENT USING THE SPEAK UP! VOICE POWERED GAME SUITE.......................................................... 24 FIGURE 10. A TEACHER USING SPEAK UP! COMMUNALLY WITH HER ENTIRE CLASS.............................................................................. 25 FIGURE 11. MATHRU CENTER TEACHERS CREATING A SIGN IN SIGNBOOK............................................................................................. 26 FIGURE 12. TECHNOLOGY IS OFTEN TREATED AS A NOVELTY, SINCE STUDENTS DO NOT ENCOUNTER TECHNOLOGY IN EVERYDAY
SETTINGS.................................................................................................................................................................................................. 27
LIST OF TABLES
TABLE 1. SIGNBOOK TABLE OF FEATURES................................................................................................................................................... 19
3

INTRODUCTION
TECHBRIDGEWORLD
TechBridgeWorld (TBW) is a research group based in the Robotics Institute at Carnegie Mellon University (CMU). The group forms and builds upon partnerships in underserved communities internationally to create culturally appropriate technology that adheres to each community’s vision of progress. Founded in 2004, TBW leads the innovation and implementation of solutions that help address challenges in communities not ordinarily served by technology development efforts [[1]]. This work often involves the invention of new tools and customization of existing technology as well as efforts to inspire and train future researchers and technologists in this kind of work. TechBridgeWorld employs the knowledge and imagination of faculty, staff, and students in all of its projects. TechBridgeWorld aims to create technology solutions accessible and relevant to all to improve people’s lives, enhance and share advances in technology across cultures, and promote sustainable development around the globe.
ISTEP
TBW launched the Innovative Student Technology ExPerience (iSTEP) internship program in the summer of 2009 to provide CMU students with real-world experience in applying their knowledge and skills toward serving marginalized or underserved communities. Through a unique 10-week summer research internship, students conduct technology research projects alongside partner communities across the world.
iSTEP offers selected students the opportunity to engineer with compassion through first-hand experience. Over the years iSTEP has brought together intern teams of undergraduate and graduate students from CMU’s Pittsburgh and Doha campuses with diverse experiences and disciplinary backgrounds. Together with TBW, each iSTEP team collaborates with local partners to better understand the needs of the community and design technological solutions that could help address some of the challenges that the community faces. As part of TBW’s philosophy, all solutions are built with the community and developed through trust.
Past iSTEP locations included Tanzania in 2009, Bangladesh in 2010, Uruguay in 2011, Ghana in 2012, and India in 2013 with projects in assistive technology, literacy tools, information exchange, and environmental sustainability [[2], [3], [4], [5], [6]].
The multidisciplinary iSTEP 2015 team was comprised of four undergraduate students from various schools at CMU including the Carnegie Institute of Technology, College of Fine Arts, Dietrich College of Humanities and Social Sciences, School of Computer Science, and the BXA Intercollege Program. iSTEP 2015 interns have built upon the iSTEP 2013 team’s work by developing new assistive technology projects in collaboration with TBW’s partner, the Mathru Educational Trust for the Blind.
4

THE MATHRU EDUCATIONAL TRUST FOR THE BLIND

While now working with

numerous global partners, TBW’s

longest standing partnership has

been with the Mathru Educational

Trust for the Blind (Mathru Trust).

The Mathru Trust houses both the

Mathru School for the Blind (Mathru

School) and The Mathru Center for

the Differently Abled (Mathru

Center). Ms. Gubbi R. Muktha

founded both schools after her own

debilitating injury. TBW has been

Figure 1. The Mathru Center for the Deaf and Differently Abled

partnering with the Mathru Trust since 2006 to build assistive technology based on observed

and communicated needs with continual community input and feedback. Additionally, TBW has

been partnering with the new Mathru Center since its inception in 2012. In 2013, an iSTEP team

completed a Needs Assessment report and since then, an ongoing partnership with the Mathru

Center has continued.

The Mathru Center was inspired after illness caused a blind student at the Mathru

School to suffer significant hearing loss, thus being unable to continue on at the school without

more specialized accommodations and support. Today, the Mathru Center is a residential

school offering free education for students with hearing impairments or multi-sensory

disabilities. The goal of the Mathru Center is to recognize the capability and power of all

individuals, regardless of the limitations imposed by society and environment regarding

physical ability. The Mathru Center currently has 47 students of which 31 are deaf and hard of

hearing students and 16 have multi-sensory impairments. While this year the Center serves students from nursery to 4th standard, additional standards will be added annually as students

grow into the next grade. The curriculum for the Deaf students at the Mathru Center follows

the state syllabus as well as teaching the additional curricular needs of speech and sign

language.

INFORMATION COMMUNICATION TECHNOLOGY FOR DEVELOPMENT IN INDIA
Though Information and Communication Technologies for Development (ICTD) is a young discipline, India has been an epicenter of research and field studies in this area [[7]]. The surge of Information and Communication Technologies (ICTs) in India has been growing for several decades, starting from India being the first country with a computer in the global South

5

in 1956 [[8]]. This has created a space for incredible empowerment as well as imposed solutions.
Technological development that caters to users in the Global North often makes assumptions that are not true of developing countries. To name a few: reliable power, Internet access, previous experience with technology, and literacy. In India, a country with only 12 Internet users per 100 [[9]] persons and unequal literacy rates across states and demographics, these common assumptions by developers deny large Indian populations from technological support [[10], [11], [12]].
For further marginalized groups, such as the disabled, access to and experience with technology is likely even more limited [[13]]. Thus, additional provisions were taken to ensure a solid understanding of the unique needs of our partner, the Mathru Center. Likewise, it was critical to employ participatory and empowering stances in both the creation and evaluation of our work.
CULTURE AND COMMUNITY
Currently, 360 million people worldwide have disabling hearing loss, which the World Health Organization defines as “hearing loss greater than 40 dB in the better hearing ear in adults (15 years or older) and greater than 30 dB in the better hearing ear in children” [[14]]. Individuals with disabling hearing loss have trouble hearing everyday conversation or perceiving sound in even quiet environments. As it stands, only 10% of the global need for hearing aids is met, 90% of the world’s deaf population has never been to school, and only 5% can read or write. The prevalence of hearing loss is unequally distributed across the world with 80 percent of the world’s deaf population living in developing countries [[15]]. Additionally, South Asia has the highest rates of deaf and hard of hearing individuals while the current production of hearing aids meets less than 3% of developing countries’ need [[1]]. In turn, South Asian Deaf communities face increased challenges and hardships.
Under India’s Persons with Disabilities Act of 1995, the integration of disabled communities into mainstream society was established as a government priority [[16]]. However, official documentation of sign language has been disregarded and unsupported, thus keeping India’s Deaf community from being widely understood and recognized.
Our work with the deaf and hard of hearing community at Mathru Center addressed two complexities. First, while the Mathru Center hopes to increase access to sign language, Indian society as a whole neglects sign language and thus prioritizes verbal language as a necessary part of deaf peoples’ societal inclusion. Therefore, work in this area must acknowledge the present expectation of verbal communication as a path to self-advocacy for the Deaf. Although India’s Deaf culture is growing, they are still decades away from having the same support and recognition seen by American Deaf culture - so much so that it would be
6

disadvantaging Mathru Center students to not enable verbal language access alongside sign language education [[17]].
A major difficulty that is somewhat unique to India’s growing Deaf community is the vast multitude of existing verbal languages. This situation can lead the Deaf to feel societal exclusion across all of India’s diverse language-scape in both the hearing and Deaf communities. Thus, deaf individuals in India experience the unique challenge of having to bridge a multitude of signed languages on top of a multitude of verbal languages. In India, deafness often assumes dumbness – the cognitive inability to communicate. This, along with India’s cultural denunciation of sign languages, bars India’s Deaf communities from support in both signed and spoken languages. Despite an increasingly global world, Deaf communities remain disconnected due to a lack of understanding, lack of interpreters, and lack of societal respect for sign languages [[18]].
The main “justification” for the linkage between deaf and dumb is the delayed speech and communicative development of deaf children. However this delay is in spoken language, an unnecessary metric if provided access to sign language [[19]]. Speech therapy remains prohibitively expensive and time intensive. Because the Mathru Center also lacks the resources to fully invest in individualized daily speech therapy, progress is slow and unfruitful in the short span of the school year. Access to spoken language remains difficult to incorporate and utilize in education for deaf Indians, yet they would be fully able to participate in society if given support and encouragement [[16]].
Efforts like the Azim Premji Foundation (APF) in Bangalore, India and the Digital Information Research Foundation (DIRF) in Chennai, India have set up thousands of centers across India to give access to shared computers with priority for the deaf and disabled. However, APF coursework uses largely audio-based local language instruction and DIRF aims to give opportunity for technical literacy for the deaf.
Very few of our related sources focus on the considerations of early Deaf education or if they do, assume a higher than present familial support network [[20]]. We aimed to create assistive tools that would standalone in function but not replace the power of the teacher. Existing projects considered accessibility in one sense, but none addressed the key constraints at the Mathru Center: low technical literacy and students with little means for communication.
Our work identified and addressed two main hurdles that exist for marginalized deaf Indians and their access to language and communication: firstly, formal language expression and formation through sign language, and secondly, awareness of voice and the effect it may have. The former requires more training and support by teachers, who need to teach and encourage sign language acquisition to students [[21]]. The latter is a necessary prerequisite for verbal language formation but is too often coupled exclusively with speech therapy. Our work follows the idea that awareness of voice and the power it has can be additionally supportive in formal language formation and learned without the need of a trained speech therapist.
7

The goal of the TechBridgeWorld-Mathru partnership and the work of this team has been to create assistive technologies that serve and empower the Mathru Center community. Based on a prior needs assessment at the Mathru Center (further detailed later in this report) the chosen focus of our work is to explore language access at the Mathru Center via both signed and verbal means. While creating agency through our custom sign language repository, we also acknowledge the need of the Mathru community to use speech therapy for instruction in immediate skills to assist students’ access to the broader Indian society. These projects begin to tackle the groundwork of technology to assist the communication of the deaf and hard of hearing in India: both in an increased recognition and access to local sign languages and in the training of verbal language through speech and one’s voice.

ISTEP 2013 FINDINGS

The 2013 iSTEP team performed a needs assessment at the Mathru Center to identify

key areas of difficulty or complexity in teaching to the differently abled students at the Center.

This included a two-fold consideration of teaching methodology and student needs. Through

observations, shadowing, affinity matching, and interviews, the 2013 team identified key

problem areas inhibiting teacher efficacy [[17]].

The findings through this needs

assessment recognized that the most

prevalent teaching tool, flashcards, had

both temporal and physical restraints.

Temporally, flashcards took a long time to

sort by subject area and difficulty, leaving

teachers with less time to teach. Physically,

flashcards are a limited medium requiring

teachers to present cards to an entire

Figure 2. Flashcards used at the school

group with varying ways of learning,

making it challenging for material to be

customized to each student’s needs. Teachers provide a lot of individual attention because

students are at varying skill levels, but that proved to be challenging with existing tools and

resources. Interestingly, the report noted that teachers felt that students learned faster from

each other.

Another significant limitation identified was teacher training. Because the Mathru

Center is in its infancy and India as a whole lacks teachers trained in education for deafness and

multiple sensory impairments, the Mathru Center at the time had only one trained teacher

during the summer of 2013. Thus, they rely in part on teachers without experience or training

in special education to lead classrooms, whose lack of background experience and high

turnover leads to many challenges relating to training and classroom efficacy.

8

The 2013 iSTEP team suggested a few problem spaces in which a new technology solution could be useful. The main suggestions for teachers included a teacher training reference for teachers to develop their skills and teaching style from published online resources and compiled notes from trained Mathru Center staff. This included a sign language training tool for untrained teachers, but this was noted as being difficult as the sign language used at the Mathru Center is a dialect of Indian Sign Language that has very little existing documentation.
Another recommendation for future work was a flashcard generator and lesson planning technology. These directly respond to teachers’ observed difficulties in the time spent preparing lesson plans. Such a tool’s database of words would need to consider a wide range of words and high quality images that are also culturally relevant and available. It would also need to recognize that the Mathru Center currently does not have Internet access, and thus would need to work locally on the computers at the Mathru Center.
The report also noted that teaching speech and word pronunciation was time-intensive and difficult for teachers, thus leading to a suggestion of speech practice software. This software could help assess and give feedback to students about the correctness of sounds they are making.
Other noted technology solution suggestions included Computer Training, Sign/Alphabet Translating, and a Multi-Sensory Classroom Assistant to support the wide range of students attending the Mathru Center.
MOTIVATIONS
The iSTEP 2015 research team aimed to address the foremost challenges affecting teachers and students at the Mathru Center with newly developed and customized technology solutions. According to the iSTEP 2013 team’s needs assessment at the Mathru Center, one of the biggest challenges at the Mathru Center was teacher preparation and training. In addition to validating this need through conversations with head staff and teachers, the 2015 team also assessed additional needs of the Mathru Center where technology may play a role. As a result, we have iterated designs and deployed early versions of educational technologies that address sign language access and speech therapy at the Mathru Center.
This report informs the work of future teams and the TBW research group on assistive educational technologies initiated during the 9-week field research internship at the Mathru Center. Secondarily, this report can serve to educate onlookers of the ongoing partnership between TBW and the Mathru Trust, as this report demonstrates the particular and unique considerations taken when developing for the deaf and hard of hearing community. Lastly, this report hopes to express the importance of ownership and empowerment when working with developing communities. In particular, this project contributes to the growing body of work
9