Becky Tyler has been an avid gamer since childhood. She started with web games featuring characters she recognized from TV, then moved on to games like Moshi Monsters and Club Penguin, where she could dress up her characters and move around virtual worlds. She also loves creating digital art, livestreaming and creating videos on her YouTube channel, and chatting and playing games with her friends online. Born with quadriplegic cerebral palsy, Tyler does all of this by controlling a computer using her eyes. By the time she was 12, she wanted what every gamer her age wanted: to play Minecraft.
Tyler got her start gaming with Special Effect, a charity dedicated to helping people with disabilities enjoy video games, but their assistive technology offered her just enough functionality to leave her wanting more.
For years, she made do with what she had, but her requests for something better never stopped, so Special Effect eventually turned to developer Kirsty McNaught to find a better way. McNaught started to build a fully custom solution, but quickly realized that she could save a lot of time and effort with open source. The OptiKey project offered the base eye gaze tracking functionality she needed, so she forked it to create EyeMine, an open source interface designed specifically to help people with disabilities play Minecraft with their eyes.
As with any project, there were bugs, and Tyler happily served as a beta tester. There were also accessibility issues that McNaught couldn’t predict, and Tyler helped her there as well. Soon enough, Tyler was bitten by a different sort of bug—she wanted to learn how to code so she could fix these issues herself. The two began working together, with Tyler helping McNaught understand her needs to create new and better features, while McNaught helped Tyler learn how to code.
Check out the video below—the first in GitHub’s Coding Accessibility video series—and keep reading to learn how Tyler and others innovate as developers with disabilities, from building custom hardware and software with open source to harnessing the power of generative AI.
A copy of this video with audio descriptions of visual content is also available on YouTube.
Tyler’s story is no different from any developer’s origin story except for one fact: the standard methods for using and interacting with a computer were inaccessible to her, as they can be to more than 1.3 billion people with disabilities worldwide. But Tyler is nothing if not tenacious—a necessary survival skill for those who navigate a world built for the fully-abled. Today she’s enrolled at the University of Dundee, where she studies applied computing under the guidance of Dr. Annalu Waller OBE, a professor of computer science who leads the Augmentative and Alternative Communication (AAC) Research Group at the university.
“I want to keep showing the world what is possible with technology, determination, and perseverance,” says Tyler. “I hope to open up many more opportunities for disabled people in the world of gaming and communication. Annalu has inspired me to study computing at university level. She shows that it is possible to overcome the barriers of disability and achieve the top levels of academia.”
Waller, who was also born with cerebral palsy, explains that AAC technologies like OptiKey enable people with disabilities to fundamentally participate in the world. “Communication is what makes us human,” she says. “If we support people with little or no function of speech to communicate, we are providing them with the opportunity to learn, be educated, be independent, and fundamentally to achieve their full potential.”
After more than 30 years in the field, however, there’s still one basic problem. “At the moment, most devices and systems are designed by fully-abled people,” Waller explains.
With open source projects like OptiKey and developers like Tyler, that’s finally beginning to change.
Where flexibility meets the diversity of disability
When Tyler codes, she uses Optikey Coder, another OptiKey fork built by McNaught that offers features specific to writing code. For example, OptiKeyCoder resets word prediction mid-word for snake_case or CamelCase variable names, allowing Tyler to change text prediction styles depending on whether she’s writing code or a comment. Context-aware word prediction is imperative, because each time she looks to see if the word she wants has been suggested or not, she’s pulled away from the main task: writing code.
Anna Kirkpatrick, a senior associate analytical consultant at SAS, has a neurological chronic pain disorder that over time prevented her from using a physical keyboard. She first started by trying speech-to-text software, but it always gave her a sore throat, so now she uses OptiKey with a custom keyboard layout and a series of customizable adaptive switches. While QWERTY was familiar, it hid common coding characters under several menu layers, so Kirkpatrick got creative and designed her own keyboard layout using frequency data from source code and her knowledge of coding practices and IDEs. She determined the most commonly used characters for the language she was writing, such as tab, enter, space, parentheses, and brackets, and then designed a keyboard with those keys placed closest to the center of the screen. Together with the vim keybindings, Kirkpatrick has altered and assembled a collection of open source components that fit her needs.
“I use Vim key bindings on my input line in bash, so that if I need to jump back three arguments, I don't have to hit the left arrow 20 times,” says Kirkpatrick. She points to open source’s flexibility as key to accessibility, comparing the ability to customize open source software to “the sort of settings and configurability that people in the hacker sphere have used for a long time in their .bashrc and .inputrc files.”
Other developers with disabilities, such as GitHub Head of Accessibility Ed Summers, use what now might be considered more mainstream and standard accessibility tools. When Summers first learned to code, he used a printed manual. But as time went on, Summers’ vision degenerated, and he turned to screen reader software like NVDA to use his computer. Over time, innovations like GUIs and smart phones would periodically present new barriers, and Summers would need to find creative ways to get things done. He says he went from being a very slow visual reader to a very fast reader with his hearing. He can now memorize an entire code base and work on solving code problems whenever and wherever he is. Kirkpatrick also says she’s learned to hold a lot in her head at once, so her first attempts at code are often less buggy than someone else’s.
Waller says that open source offers the perfect medium for creative solutions. “You can buy a tricycle and physically change it to meet the needs of the individual. Open source gives us that same adaptability, but for software.”
If you’re looking for commonalities instead of differences, you might find a preference for coding best practices like test-driven development, refactoring, design patterns, and concise lines of code. Even then, a preference for one person can be a disinclination for another. For example, Python uses whitespace to define code structure and hierarchy instead of the curly braces used in languages like C or Java. For Summers, this is added friction to work with using a screen reader. For Kirkpatrick, however, it simply means fewer keystrokes. She says that even some coding “best practices” can be problematic—but there is a potential way forward.
“There's a real tension in some of the style practices, where you want long, descriptive variables and function names, but you don’t want to type that much,” she says. ”That's where I think there's room for better completion prediction tools. Right now, a lot of what exists is based on natural language, which code is not.”
AI for accessibility
Recent advances in AI mean that technologies are starting to meet people with disabilities where they’re at, rather than the other way around. Open source language models like the GPT family of language models and the PyTorch-based Transformers library have formed the foundation for tools like ChatGPT and GitHub Copilot, which offer examples for the promise of generative AI and accessibility.
For example, McNaught explains that Tyler is dyslexic and sometimes her spelling is too far off to trigger text prediction for the correct word. So McNaught built a Python app to do phonetic lookup with awareness of dyslexic biases. And as Tyler studies new topics, McNaught processes her lecture materials to extract keywords and add them to the available vocabulary in her communication software. But lately, McNaught sees promise in generative AI, which can perform similar functions but doesn’t require her expertise. “If you just paste the course transcripts into ChatGPT and ask it for the keywords, it does a really good job,“ she says.
For Anton Mirhorodchenko, a Ukrainian refugee born with cerebral palsy, supported eye tracking hardware is too expensive, and speech-to-text isn’t an option—it doesn’t understand him. Instead, he uses a physical keyboard, despite his disability.
“Do you know how to type parentheses with one finger?” he asks. “Holding down shift with your nose.”
That hasn’t stopped him. He’s worked as a freelance developer since earning his Master’s degree in Computer Maintenance and Networks in 2012, and recently, he started exploring how to use ChatGPT and GitHub Copilot to better communicate and write code. Our emails provided a particular delight, as he likely realized a similar point—not long ago, our communication would be improbable, if not nearly impossible.
“Just imagine: I'm responding to you in Ukrainian and omitting about 60% of the letters from the words, but ChatGPT understands what I am saying and provides an answer in English,” he wrote in an email. He offered an example of how he composed a sentence.
Mirhorodchenko: h m n s Antn, m f ukr. im dev w cplsy.
ChatGPT: I think you are trying to say: "Hi, my name is Anton. I'm from Ukraine. I'm a developer with cerebral palsy.
Mirhorodchenko recently took his first steps into open source with a project he built to learn the basics of Laravel and Redux “while simultaneously exploring the capabilities of AI and its potential to help people with disabilities.” He outlines how he used roles (i.e. “Act like a software architect expert.”) to shape ChatGPT’s responses, and then prompted it to write high-level instructions for how to build the application (i.e. “Create a simple PHP application using the Laravel framework, a MySQL database, and the React JS framework on the front end.”). ChatGPT’s responses included everything from lists of necessary components to installation instructions, which he then fine-tuned using GitHub Copilot.
GitHub Copilot, he says, “is much more convenient for ‘granular’ use, such as implementing tests, small cycles/conditions, and so on.” And while he found that AI couldn’t replace expertise, he said that his experiment “demonstrated the potential for AI to revolutionize the way people with disabilities work and interact with the world around them.”
“This approach eliminates physical challenges for me and leaves only mental work (check results, incorporate them, figure out what to do next, etc.) that I find much easier,” he writes. “As a non-native English speaker with a physical disability (a severe form of cerebral palsy), it can be challenging for me to communicate effectively and efficiently.” With AI assistance from tools like ChatGPT and GitHub Copilot, however, “I am able to overcome these obstacles and bring my ideas to life,” he explains.
Collaboration and bridging the disability divide
Disability rights advocates frequently use the slogan "nothing about us without us" to emphasize that people with disabilities need to be included in decisions that impact them. Summers says they also need to be included in the design and development. “Including developers with disabilities as first-class members of development teams is the best way to ensure everyone can benefit from innovation and progress,” he says.
For developers with disabilities who contribute to assistive technologies and accessibility, Summers sees particular opportunity for innovation.
“I lost my vision over a decade at SAS while working with approximately 3,000 engineers across the research and development organization, and it's really hard to understate the impact that my lived experience had on the organization,” says Summers. “Part was because I grew up there, everybody knew me, and they saw me go blind. That was a catalyst for that company to do some pretty amazing accessibility work.”
Summers and his team created the SAS Graphics Accelerator, which enables users with visual impairments or blindness to create, explore, and share data visualizations. Now, he leads the charge to empower disabled developers to create, collaborate, and contribute at GitHub. Through open source and collaboration, these developers are not only able to participate, they are able to contribute, and their lived experiences are integral.
Tyler builds private EyeMine mods and contributes to OK, Game On!, an OptiKey fork for PC gaming that defines interfaces for particular games. Beyond code, Tyler speaks publicly at universities and conferences, and appears on national TV and radio to share her thoughts and experiences. Through assistive technologies, she has a voice.
Kirkpatrick sees open source contribution as a two-way street. She has published her custom keyboard layout for others and has made some small code contributions to OptiKey, but perhaps more importantly, she’s also commented on issue threads, helped new users, ran tests, and written documentation.
“That sort of collaboration doesn’t happen when you reach out to tech support for a commercial product,” says Kirkpatrick. “You build community by discussing and sharing how you use something, and in return, people are willing to consider your suggestions about new features, and maybe even implement them. There’s this resource exchange: You contribute what you’re able, and people tend to give back.”
Waller, meanwhile, whose contributions in AAC research span decades, is helping to foster the next generation of developers with disabilities.
“This is what excites me so much about helping people like Becky to be computing professionals,” says Waller. “They will lead the way in supporting the field to be totally inclusive and adaptive. I'm ready to retire tomorrow because a part of me thinks, ‘Wow, we are actually getting there!’”
While many companies see accessibility as something that only affects a small percentage of their current user base, Waller points out that this is a short-sighted perspective to take. We’re surrounded daily by seemingly mundane items originally built as assistive and adaptive technologies that now benefit us all—the typewriter, ballpoint pen, and electric toothbrush among them. Designing and building with accessibility in mind isn’t solely for the benefit of those currently dealing with a disability. Most people will face a disability in their lifetime, whether from injury, disease, or simply age.
“If you design for the extraordinary user, you’re designing systems that can be used by ordinary users in extraordinary circumstances. A system that can be accessed by someone with no sight or limited sight automatically helps someone who’s lying on the beach in the sunshine trying to read on a Kindle,” says Waller. “Disability is not abnormal, it's part of being human. We owe it to everyone to support each other, because one day it could be you.”
And the best way to do that, beyond following guidelines, can be quite simple: Include people with disabilities into the design and development process.
“Developers often go into situations with these great ideas, not understanding the environment or the milieu in which the tech needs to work. We don’t start with the user and their environment,” says Waller. “If we actually had participatory research and design, we would empower end users not only to be co-designers but to actually drive the research and innovation of new technology.”
Tyler agrees. “I hope gamers like me become more involved in the design, development, and testing of video games,” she says.
