Research & Publications
Vision Without Sight: Developing an Audio Game Engine for Blind and Visually Impaired Creators and Players
Vision Without Sight: Developing an Audio Game Engine for Blind and Visually Impaired Creators and Players
Freya Shaw
MA By Research
University of Huddersfield
Music Technology
June 2025
Contents
3.0 Literature Review and Context
3.1 Introduction
Abstract
This MA project, ‘Vision Without Sight: Developing an Audio Game Engine for Visually Impaired Creators and Players’, aims to demonstrate the critical need for technology, particularly development software for media such as game engines, to build screen reader access from its inception.
​
Without screen reader adaptions, many are denied access to digital tools, skills and occupations. A study of common screen reader frustrations outlined typical accessibility issues, including:
​
“(a) page layout causing confusing screen reader feedback; (b) conflict between screen reader and application; (c) poorly designed/unlabelled forms; (d) no alt text for pictures; and (e) 3-way tie between misleading links, inaccessible PDF, and a screen reader crash.” [34]
In spite of the study’s age, many of these issues persist in typically visual environments such as game engines.
​
“Regardless of advancements, game engines remain difficult to access for blind developers or those requiring non-visual interaction. This gap limits the potential for disabled developers to create games and experiences on par with sighted developers, as there are few tools that offer screen reader support or other accessibility-friendly design options.” [52]
​
Creative development software must ensure the inclusion of blind and visually impaired (BVI) users in navigating platforms typically accessed through complex graphical user interfaces (GUIs). To address these accessibility issues, I, a visually- impaired programmer who accesses devices with VoiceOver, Apple’s built-in screen reader, developed the Hodr Engine, an accessible game engine for BVI and sighted creators and players. I created this platform in Python, where users can make audio-based games using HodrScript, a basic programming language, developed with ANTLR, a tool for creating parsers to process text. Programming in HodrScript, running code, and packaging applications are all executed within an integrated development environment (IDE) powered by PyQT, a Python library for building user interfaces.
​
Research methodology involved gathering survey data from BVI users regarding their level of visual impairment, preferred game genres, suggested accessibility-based techniques and play styles, interest in games with visual elements, and previous experience with technology and game development. This helped determine the complexity needed for the user interface, common technological challenges for BVI users featured in software that must be identified and avoided, and the types of games that should be produced by the Hodr Engine. A participant, referred to as Herbert, was introduced to HodrScript through a tutorial, and used the engine to create an audio game and also provided feedback on his experience.
​
Initially, the engine was designed to run through the Terminal, which is a macOS command line interface (CLI), where users could code in a .txt file and bundle games using sound files and splash images. However, research revealed that the Terminal is challenging for some BVI users, especially those less familiar with programming or navigating CLIs. Contrary to my hypothesis that a text-based environment would be more accessible due to its historical use by BVI programmers, the complexity this virtual environment and terminal navigation proved to be a barrier. As a result, I developed a more accessible IDE with features such as a text editor, buttons for running and packaging games, shortcut keys, and tutorials, all in one place. The application can be downloaded from the Hodr Engine website.
​
This research concludes that a basic user interface is more accessible than Terminal for screen reader users who are not experienced coders, even though the latter is text-based. Including navigational shortcuts and accessible design features in the IDE significantly improved the engine’s usability for BVI users.
Thesis Contents
This commentary is accompanied by the following resources:
​
-
A Hodr Engine IDE folder and README.
-
Tutorial on utilising the Hodr Engine and programming in HodrScript.
-
A folder containing Herbert’s FinishedGameIE and The CeReNeM Audio Experience application.
​​
All applications function exclusively on macOS. Upon attempting to open an application, a message may appear stating that the app cannot be opened because the developer is unidentified. If this appears, navigate to System Settings, select Privacy and Security, and grant permission to open the application.
Acknowledgements
My research would not have been possible without the support of my family, friends, and colleagues. My gratitude goes out to my parents, brother, grandparents, and partner for their assistance throughout this year of research. From providing transport to and from the University of Huddersfield to helping me navigate online platforms, I cannot express how much you have supported me and how much I appreciate you.
​
I am also grateful to my supervisor, Dr Alex Harker, for his guidance and insight into audio programming, software development, and thesis composition, which I’ve accelerated under your tutorship. In addition, my programming mentor, Dr Adrian Jackson’s regular coaching over the last three years sparked the foundations of my programming knowledge and provided the skills to develop the Hodr Engine, which I cannot wait to expand.
​
I extend a thank you to the members of the CeReNeM Journal for allowing me to participate in your work and integrate the Hodr Engine into our journal project.
​
I would also like to acknowledge the team at Innovate UK, whose belief in the Hodr Engine and support in its establishment and promotion has made such an impact on the engine’s progress.
​
Last but not least, I thank the participants who contributed to my survey, particularly a participant who is referred to in this paper as Herbert, for their valuable input and perspectives.
​
I’m excited to continue building the Hodr Engine and will always value the help of everyone on this list. Thank you.
Collaboration with Dr Adrian Jackson
The Hodr Engine was developed with the input of Dr Adrian Jackson. Jackson’s involvement began during my undergraduate major project, in which I produced a prototype for an audio game called Immersion Sound Studio. This audio game is centred around a recording studio containing a portal door, which transports the user to various soundscapes where they can collect sounds and listen to them within a curated auditory environment at the end of the game.
​
At the time, I did not have the skills to code such a project independently, so Jackson scripted the game’s foundations, because common game engines, such as Unreal Engine and Unity, which include tools like Unreal Engine’s Blueprints and Unity’s Bolt for visual scripting, were inaccessible to me. His Python code included core files for the project, such as gameutils.py, which provided utilities for the game, like the menu system, a function to collect sounds during gameplay, and the ability to play sounds at the end of exploring soundscapes; soundscape.py, which handled stereo soundscaping functionality; and SoundsX.py, which contained hardcoded soundscapes.
​
We approached this game code as a rudimentary game engine, so I could build on Jackson’s groundwork. For instance, I added new soundscapes in SoundsX.py, modified gameutils.py to adjust player movement (like increasing stride length), and also introduced new functions within the soundscape.py file. The utilities from gameutils.py and the Soundscape module in soundscape.py were later incorporated into the Hodr Engine but were heavily modified by me. This transformation shifted the code from being game-specific to being a functional game engine.
​
During the Hodr Engine’s evolution, Jackson provided an introduction to ANTLR, a tool for parsing and processing structured text, used to create programming languages. This explanation of how ANTLR operates enabled me to construct the HodrScript language independently. Jackson also provided a structure for the HodrListener (a listener processes code during parsing, and a parser analyses text to extract structure). The rest of the game engine’s code, including its modification from a hard-coded game script to code suitable for an engine, was developed independently by me, as was the HodrScript and IDE’s creation, and new functions for combat and puzzles.
​
Overall, the development of the Hodr Engine was built from the original game code’s structure. These scripts were a framework from which I modified and produced new code to transform the specific game code into a more generalised game engine with Jackson’s mentorship.
Chapter 1 - Introduction
Origins of the Hodr Engine
The Hodr Engine, named after Hodr, the blind Norse god, was devised to make game development accessible to BVI creators. The idea originated when I developed an earlier spatial audio-based game where users could explore immersive soundscapes. This concept was inspired by the audio game Sounds of Eden [14], in which users navigate a binaural soundscape. However, while trying to build this game, I, a visually-impaired user, encountered the inaccessibility of existing game engines like Unreal Engine [19], which rely on complex GUIs. Such engines’ visually-oriented content is challenging for screen readers, which struggle to adapt to detailed graphical elements, and therefore is not accessible to screen reader users, who are typically BVI, alienating them from accessing those engines.
If you are a visual reader, you may notice a large gap between the text above and this block. However, the text above is written in white on a white background, making it invisible —much like the experience BVI users often face when struggling to access visually-focused user interfaces without proper support. Just as you may struggle to access the white text without assistance, BVI users encounter similar barriers. Below is the once invisible text, now in black on a white background:
​
This section is written in white text on a white background, making it invisible to visual readers: If you are reading this commentary with a screen reader or a refreshable braille display, you will likely have no difficulty accessing this block of text. However, since the text is white on a white background, it is inaccessible to visual readers, creating the impression of a large gap between this block and the one above.
Recognising the need for an accessible alternative to popular game engines, I collaborated with Adrian Jackson, a retired University of Huddersfield computer science lecturer, to build an audio-based game, Immersion Sound Studio, using Python, and to understand the foundations of code and audio game design.
​
This previous project was initially developed using Emacs [51], a code text editor known for its BVI-focused screen reader, Emacspeak [44], and later Visual Studio Code [40], another code editor that can be operated with accessible features like shortcut keys.
​
During Immersion Sound Studio’s production, it became clear how accessible a text-based programming environment is for BVI users compared to a non-BVI-adapted GUI. Screen readers and refreshable braille displays interact more effectively with text, making programming a natural fit for BVI users. I realised that the code developed for the Immersion Sound Studio could be restructured to create a BVI-accessible game engine. This idea led me to begin the development of the Hodr Engine.
Chapter 2 - Research Questions and Aims
Improving BVI Accessibility in the Context of Game Development
This project seeks to improve accessibility in the context of game development through the identification of inaccessible features in existing game engines and other software. A game engine was then created inspired by these findings that addresses these issues.
Chapter 2 - Research Questions and Aims
Building the Hodr Engine
Additionally, this commentary discusses the development of the audio game engine based on a literature review, the experiences and feedback from the participant survey, previous research, and a user’s experience developing a game with the Hodr Engine.
Chapter 3 - Literature Review and Context
Introduction
This literature review presents existing research on accessibility-related challenges and solutions for the BVI community, covering general technological challenges as well as issues specific to game development and audio. It also includes research on non-BVI-focused video games and audio which have inspired genres and techniques for developing games accessible to screen reader users in this project. The review aims to explore the current state of BVI-based accessibility, including the outlined issues that set back, and the innovations that might improve, the development experience for BVI creators.
Chapter 3 - Literature Review and Context
Challenges and Solutions for Accessibility in BVI Game Design
The Longitudinal Transitions Study [28] is an example of research into the general accessibility issues often faced by the BVI community, including technological challenges. The study provides insights into the experiences BVI youth face during their transition from education to employment. Hewett’s research demonstrates the lack and need for inclusive technologies and support systems. This can be expanded to the technology and education needed for the creation and use of an accessible game engine. The study reviews technological challenges faced by BVI pupils, struggling to access online educational platforms. As a participant explained, “The virtual learning environment was not screen reader friendly, making it hard to access lecture notes and submit assignments.” [28]. Pupils found similar struggles when collaborating with their sighted peers, “Group members used platforms that weren’t compatible with my screen reader, so I couldn’t contribute effectively.” [28]. These inaccessible platforms prompted users to turn to adaptive tools, which also failed to operate successfully: “The screen magnification software often crashed during exams, causing significant stress and time loss.” [28] The user would not have needed to rely on failing software if not for a non-functioning platform. The study, Digital Accessibility of Online Educational Platforms: Identifying Barriers for Blind Students’ Interaction [35], outlined some reasons for setbacks in educational platforms, which extend to most software, including game engines. “The lack of semantic structure in HTML documents and the improper use of ARIA (Accessible Rich Internet Applications) roles can lead to significant barriers for blind users relying on screen readers” [35]. Similar research, such as the chapter, Inclusive Online Learning: Digital Accessibility Practices, in the book, Diversity in Higher Education Remote Learning [25], discussed the following software deficiencies: “Common barriers include untagged PDFs, images without alternative text, and multimedia lacking captions or transcripts, all of which impede screen reader usability” [25].
​
There is a common trend between accessibility issues in educational software and creative software, such as gaming. The paper RNIB Accessible Gaming Research Report [46] presented the challenges faced by BVI gamers and the lack of accessibility knowledge amongst, and resources for, developers: “People with vision impairments reported substantial challenges in finding and being able to play video games. When asked why, they consistently selected ‘video games do not have enough accessibility features” [46]. Aligned with the research findings of Lisboa et al. and Frey et al., many BVI gamers encounter difficulties navigating buttons, menu screens, and complex visual environments. Similarly, as observed in Hewett’s study, participants depend on external accessibility tools, such as the participant’s “screen magnification software” [28], often used outside of their intended application. Consequently, BVI users often create bespoke accessibility solutions to adapt these tools: “Most gamers with sight loss report heavy reliance on their own, ad-hoc coping strategies, like playing with sighted guidance, memorising button sequences and menu layout or using Be My Eyes or Seeing AI to read what is on the screen” [46]. To mitagate these issues, Lisboa suggests “Developers should adhere to the Web Content Accessibility Guidelines (WCAG) and ensure that all interactive elements are properly labelled and navigable via keyboard” [35]. In conjunction, “Faculty can use accessibility checkers to identify​ common barriers with guidelines for remediation, such as adding alternative text to images and ensuring documents are properly structured” [25]. These experiences highlight the troubling prevalence of inaccessible platforms, spanning gaming and education, for screen reader users. To mitigate the issues raised by Hewett, the Hodr engine ensures all interactive elements, including buttons, links, and form fields, are labelled, the platform is navigable via shortcut keys, and images and visual layouts contain descriptive alternative text. The RNIB Accessible Gaming Research Report identifies potential solutions to gameplay for BVI users: “Audio-based solutions are most desired by blind and partially-sighted gamers. These include screen reader compatibility, audio description, audio triggers and adaptable audio settings and sound mixes (e.g., spatial audio)” [46]. Audio features, like spatial audio, could also be used outside of gaming, as they are not specific to this domain. The study also provides solutions for accessing visual content, which may align with more general accessibility requirements: “Customisable visual features are highly desired, including adaptable text size, colour contrast, and magnification options” [46]. The study concludes with a statement regarding gaming, also relevant to all forms of software accessibility: “Developers should be provided with training and resources to better understand the needs of gamers with sight loss and to implement accessibility features effectively” [46].
​
Resources on accessible gaming techniques for blind gamers are growing, such
as AudioGames.net [9], which offers forums and a list of audio games, connecting developers to BVI players. In addition, The Game Accessibility Guidelines website [26], contains information on making games accessible, along with studies like the RNIB Accessible Gaming Research Report [46], which highlight barriers and solutions for blind gamers, and offer guidance to studios wanting to design accessible games. For creators interested in partnering with a BVI-focused organisation to receive guidance on accessibility, AbleGamers [1], is a nonprofit organisation for improving accessibility in gaming for all disabilities. The charity also runs scholarships for those interested in game development with a disability, such as Alex James Ryan, who started his initiative, Inclusive Games [29] after winning the inaugural AbleGamers Fellowship in 2016. AbleGamers collaborated with Harman (JBL), and developed the JBL Quantum Guide Play software [31], allowing BVI users to play first-person shooter games through optimised directional audio, a feature that precisely aligns sound with its corresponding visual direction.
For studios interested in practical coding resources, Apple and Microsoft provide resources and tools for software accessibility. Apple’s Accessibility Inspector [3] is integrated into Xcode [5], Apple’s development environment, and identifies non-voice-over friendly software elements and suggests improvements. The Human Interface Guidelines [39] are a set of principles for developers to create inclusive and user-friendly applications. Meanwhile, Microsoft’s Inclusive Design Toolkit [38] offers practical tutorials and tools, as does the Xbox Accessibility Guidelines [41]. Educational institutions must inform their students about these resources, and there must be greater public knowledge of these initiatives and technologies if games are to offer greater accessibility to BVI users. Regarding documentation, the Hodr Engine is accompanied by detailed documentation in what it aims to be an accessible format for BVI users.
Chapter 3 - Literature Review and Context
Audio Games and Accessibility Techniques
To build an accessible audio game engine for BVI developers and players, popular audio games and their accessibility features have been explored to understand what makes them accessible.
​
The audio game Sounds of Eden [14] inspired my previous audio game, Immersion Sound Studio, a predecessor to the Hodr Engine, due to Sounds of Eden’s use of spatial audio to create an atmospheric soundscape. This idea was expanded upon in Immersion Sound Studio so that players could curate personalised soundscapes. Sounds of Eden is a meditative audio game in which the listener can explore a garden where plants are represented by slowly increasing arpeggiations. Sound of Eden’s soundscape organises audio spatially and is an ideal example of the type of game that can be developed with the Hodr Engine, especially as the Hodr Engine features an expandable map that allows audio files to be positioned using x- and y-coordinates. Sound of Eden’s stem files could be plotted in the Hodr Engine to recreate the game’s soundscape.
​
The paper, Accessible Games for Blind Children, Empowered by Binaural Sound
[17] documents the creation of a binaural game to improve the spatial awareness of BVI children within a game environment. The audio techniques presented in the paper provide a guide for methods that may be used within an audio game engine. The game implements popular audio game techniques in addition to the spatial soundscape explored in Sounds of Eden, such as auditory icons. The article,​ Auditory Icons [15] explains: “The rationale behind auditory icons is to support and supplement predominantly visual information with a corresponding sound. For example, the action of successfully emptying your trash on a Mac OS would result in the sound of paper being crumpled up and discarded. The most immediate benefit of these sounds is a tangible confirmation of the action you have performed. Much like in the real world, you press a button and it responds (in that sense even the click of a mouse could be considered an auditory icon)” [15]. Didlick suggests that there are four forms of auditory icon:
​
-
“Nomic: Sometimes referred to as literal or realistic, this type of auditory icon is a direct sonic representation of the event taking place” [15]. An example of this would be the iPhone’s camera sound effect when a picture is taken.​
-
“Symbolic: Symbolic auditory icons are perhaps the antithesis to nomic. They do not possess any specific connection to the event taking place other than by virtue of its association and repetition” [15]. For instance, Apple’s ‘woosh’ sound effect that plays when a message is received is an example of a symbolic auditory icon. Unlike the sound of opening an envelope on receiving a message, a chime, like a bell, is more abstract.
-
“Metaphorical: A metaphorical auditory icon is one that reflects a key aspect of its associated action with a dimension of the sound” [15]. For instance, a major chord might indicate a positive result, whereas a minor chord might symbolise s negative one.
-
“Verbal: As the name would suggest, verbal auditory icons are sound bites of spoken language. Whilst these are not always the quickest way of sharing insight, they are most effective when conveying very specific information” [15]. For instance, the announcement, “Mind the gap between the train and the platform edge” often repeated in train stations is a verbal auditory icon. In this case,“The transfer of information is performed by correct assignment of the communicated pieces of information to the actual semantic content of the auditory icon” [17].
​​
The Vale: Shadow of the Crown [21], commonly referred to as The Vale, is a combat-heavy audio-based action-adventure game, implementing spatial soundscapes and auditory icons throughout gameplay. In an interview, David Evans, the game’s director, discussed using auditory icons he refers to as “beacons.” These icons assist players in navigation and interaction within the game’s world. “Evans spent a lot of time wrangling some of The Vale’s more important cues – such as beacon sounds that direct players to objectives – into place. ‘One thing I wrestled with forever, and never really knew exactly why it worked, were the magic beacon sounds. I found those became competitive with other sounds as soon as you had two objectives, like find this magic object but avoid or fight these beasts’ ” [32]. The use of multiple auditory icons to guide the player through a soundscape in The Vale is dominant within its market scenes, where players explore activities, such as purchasing items, playing mini-games, and advancing the storyline, by approaching sounds and interacting with them to trigger an associated activity, such as speaking with a stallholder or buying an item. The auditory landscape is constructed through a combination of non-player characters (NPCs) positioned in different directions relative to the player, announcing their services simultaneously. These environments also contain ambience such as music and chatter from a tavern or church bells, indicating enterable buildings. Despite the complexity of multiple auditory icons, the scenarios remain comprehensible and immersive rather than overwhelming. Similarly, the Hodr Engine uses auditory icons to represent interactive items like puzzles or characters that can be engaged in combat, as well as to alert the user to other actions, such as indicating that the game has been saved.
Auditory icons are also incorporated within the Hodr Engine’s IDE, providing
auditory feedback for actions such as panning to indicate the cursor’s location within a text field or the use of auditory icons when buttons are selected. A similar approach is employed in the code editor, Emacs [51], which integrates shortcut keys in conjunction with auditory icons linked to specific actions, For instance, the shortcut Command+S saves progress, accompanied by a symbolic bell auditory icon.
Auditory icons assist BVI users in exploring the game environment within menu screens through key or controller combinations to navigate an environment, with verbal auditory icons announcing a menu option one at a time. This method is implemented in the majority of audio games’ menu screens. However, in the audio game series Blind Quest [6], a fantasy role-playing series centring around a mercenary’s adventures, the navigation mechanics are not designed with complete freedom of movement, and instead use a menu structure. A narrator announces the story and items within an environment in a menu-like structure, presenting one option or item in an environment at a time. When items are interacted with, they trigger auditory icons, and contain descriptions such as “Nathan opens the casket, he gets a healing potion” or “On the wall there are some coats of arms and old​ weapons.” This approach is limited in its flexibility when compared to the Vale or Sounds of Eden, but reduce cognitive overload, keeps the game at a controlled pace, and may be easier to comprehend when compared to busy environments.
​
A similar menu structure to that found in Blind Quest can be implemented within games made with the Hodr Engine for the purpose of controlling overall gameplay. These use arrow keys to highlight items such as “Start”, “Continue”, and “Cancel”, with selections made by pressing the spacebar.
Non BVI-exclusive publications, such as Studying Sound: A Theory and Practice of Sound Design [13] may also provide a wealth of information for developers to expand the realism of audio, such as in games design. This book provides an overview to better understand audio effects, like occlusion and reverb, in sound design. Collins’ work in sound design expands to audio games, such as Audio Defence: Zombie Arena [18], indicating her knowledge of audio-exclusive games. Studying Sound: A Theory and Practice of Sound Design presents sound design techniques relevant to audio and visual games so developers uninterested in audio-only games may still expand their knowledge of methods like spatial audio that might make a visual game BVI inclusive. The combination of spatial soundscapes, as seen in Sounds of Eden, auditory icons / beacons, as found in The Vale, and auditory menu screens, as found in Blind Quest, supplied a foundational framework for the Hodr Engine.
Chapter 3 - Literature Review and Context
Other BVI Accessible Creative Software
The paper, An Empirical Evaluation of a Graphics Creation Technique for Blind and Visually Impaired Individuals [22] introduced the SETUP09 system, a compass-based drawing tool for BVI users: “Little research has focused on the use of a screen layout to provide people who are blind and sight impaired users with the spatial orientation to create and reuse graphics. This article contributes an approach to navigating on the screen, manipulating computer graphics, and user-defined images. The technique described in this article enables features such as zooming, grouping, and drawing by calling primitive and user-defined shapes” [22]. Tools supporting creative skills, such as graphic design, are often integrated into game design for developing visual elements, such as models for environments, and user interfaces. An example of this is the integration between Blender [10], a software for 3D modelling, visual effects and animation, and Unreal Engine [19]. Given the prevalence of such tools it might also be important to explore how programs for creating visuals are made accessible.
As a screen reader-compatible game engine necessitates a non-visual approach for positioning audio within a map to construct an immersive soundscape, SETUP09’s use of compass directions to plot points might be useful for creating an environment using a screen reader and auditory icons: “The cells are not identified by numbers but by compass directions such as north, south, east, west, north-west, north-east, south-east, and south-west. Compass-based navigation also has nine unique points in a cell rather than one unique centre point as in the grid-based navigation system and a formal command language operates it” [22]. Fernando explains, “Compass-based navigation is derived from the grid-based navigation concept but with reformed locations, points reference system with cardinal directions, and implemented with a formal language specification to work with lexical inputs” [22]. SETUP09’s compass-based navigation system provides a structured framework for BVI users creating graphic designs, with limited risk of falling out of place. However, in an audio-focused platform, restricted spatial constraints may not be as essential as in a context where BVI users are creating visuals. BVI users, who are not deaf or hard of hearing, often rely on auditory feedback to evaluate sound placement, and can then adjust sounds’ positions effectively by modifying their coordinates in code. The more accessible medium of audio for BVI users, when compared to the visual, reduces the need for rigid compass-based frameworks. While compass-based navigation assists with the prevention of unintended movements, like drawing a line in a different direction than planned, it might limit freedom in more BVI-accessible contexts, such as plotting sounds in a soundscape. In contrast to a compass-based system, a grid-based system may allow for more flexibility to place sounds on a map. For this reason, the Hodr Engine adoptis a grid-based system, mapping sounds with X and Y coordinates.
Chapter 3 - Literature Review and Context
Accessibility Challenges and Solutions within Programming Languages
The publication, Improving the Accessibility of the EarSketch Web-Based Audio Application for Blind and Visually Impaired Learners [16] explores how the EarSketch platform, which teaches coding through music, was modified so BVI users can access it through auditory icons. The platform allows users to program in Python or JavaScript to create music. As discussed in the paper, this approach is effective for BVI students of coding because it allows them to experience programming through audio rather than visual cues. “EarSketch’s integration of music and programming languages significantly improves student engagement and persistence in computing” [16]. As discussed in the paper, the platform’s original design met some accessibility barriers. For instance, EarSketch relies on multiple panels in its user interface, including a content browser, digital audio workstation (DAW), code editor, and curriculum sections. This multi-panelled interface was, at first, not completely accessible via screen reader as the original version needed to be operated using a mouse. Mouse-driven interfaces do not integrate well with a screen reader because screen readers interact with a system’s accessibility framework, which tends to be keyboard-based. The original version of the software also included fewer shortcut keys than required to navigate the entire platform, rendering some areas of the platform non-accessible to BVI users. This previous visually-focused user interface “requires design changes to become more accessible to users who are blind or visually impaired” [16], for example, “prioritising keyboard-centric navigation” to facilitate easier access for blind and visually impaired users, particularly in educational settings” [16]. The paper also states that the inclusion of “descriptive audio cues” and “tactile feedback for learners who are BVI” would improve interaction. As a result of these identifications changes were made, so actions such as selecting a sound sample, moving between sections, or executing code are now accompanied by audio confirmations, and navigating the panelled interface and code itself can be completed through key combinations. A similar approach is perfect for the text-based Hodr Engine. For instance, when navigating a section of code, an auditory confirmation plays when moving to the start or end of a line with a shortcut key. This avoids the need to move through characters one by one or a screen reader user getting lost in sections of code.
Chapter 3 - Literature Review and Context
Blind Accessible Game Engines
The Accessible Gaming Research Report [46] found developers shared an interest in understanding the accessibility needs of BVI gamers. However, many felt they did not have the resources or knowledge needed to improve the accessibility of their games: “There is a knowledge gap in the games industry; whilst 75% of developers who participated in our research reported incorporating some accessibility features in their games, only 15% reported having sufficient understanding of the needs of gamers with sight loss” [46]. Developers acknowledged one of the primary barriers to achieving accessibility in their games is due to “a lack of game engine support for accessibility features” [46]: “Accessibility should be considered from the outset of game development, rather than as an afterthought, to ensure that games are inclusive by design” [46]. This necessitates designing game engines with inherent accessibility features. Even if a developer creates visually-oriented games, the project should incorporate built-in accessibility tools, such as magnification options, accurately labelled buttons, spatial audio, and auditory icons. As a result, the game would remain accessible regardless of the developer’s initial intent.
​
Developers included in the Accessible Gaming Research Report shared that due to a lack of knowledge and the lack of accessibility features in existing game engines, they tended to cater to an audience with some working vision, as their needs are easier to meet than those without any access to visuals. “Developers give more consideration to the needs of gamers with partial sight loss than those with severe sight loss, suggesting that the former are easier to address and that additional focus is required on the latter” [46]. This information gap is due to a concerning lack of distribution of accessible resources. Many studios included in the report discussed a need for wider resources on accessibility: “Over 70% of developers would like to see sharing of knowledge and technology within the games industry and better resources on accessibility good practice” [46]. As this commentary explored previously, the volume of BVI and general disability-focused resources is growing. This information must be shared more widely amongst game studios and educational institutions so developers can make use of it in their work. In addition, game engines must incorporate inclusive techniques into their core functionality so that developers might be able to create accessible games without even realising it.
This commentary illuminates existing research on BVI-accessible game engines, helping us understand how they have been designed for screen reader users. The paper, Blind Adventure: A Game Engine for Blind Game Designers [50] introduced a game engine design for BVI users to create audio games for mobile devices. “The main idea of Blind Adventure is to give a tool to blind people that lets them create adventure or other genre games, play them, share them with their friends, and generally be creative. The game engine and the games should run entirely on smartphones, which have a high penetration rate amongst visually impaired people” [50]. The application has two modes: one for playing and one for developing. In creation mode, the engine guides the user through a hierarchical menu system with audio feedback, navigated with swipes and taps, and accompanied by haptic feedback. Each level consists of nodes, such as collectable items or puzzles and combat mini-games, each tied to a specific recorded sound. “The app acts like an audio recorder, thus being the tool for creating the games, but also for playing them” [50]. In the playing mode, users may play their own games or previously created games made by fellow Blind Adventure users. Players navigate the game through audio prompts, completing levels and collecting items to progress.
​
Despite Blind Adventure’s innovative approach and potential, the project has not progressed publicly. Even so, the design outlined in the paper inspired ideas for the Hodr Engine. It is acknowledged that Blind Adventure appears to be built with creativity and fun at its forefront, rather than complex mechanics, so its abilities are purposefully limited for ease of use. With this in mind, the app uses audio recording directly on the device, which may limit sound quality based on the user’s phone or external microphone: “Audio recordings are done using the smartphone’s microphone and are stored as WAV files” [50]. The audio is also limited to a maximum length of time: “One of the biggest challenges is here given by the fact that such recordings must have a preset length. This was set to 30 seconds but requires that the unused time at the end is deleted afterwards” [50]. The audio length and quality are constrained by the device’s microphone or external microphones that can be connected to the device. The paper does not indicate support for importing external audio, even though many game developers often incorporate sound effects, voice-over, text-to-speech, or soundscaping from existing sound libraries, collaborators, and other software. The app design also limits the number of options per screen for ease of access: “Since user choices would have to be identified through touches in different areas of the screen by blind people, we decided to limit user options to only two per screen” [50]. This choice, while accessible, may restrict the complexity of interactions compared to more flexbile game engines. Blind Adventure appears suited for structured gameplay similar to the menu structure of Blind Quest. It is a great introduction to early forms of game design, meaning users could smoothly focus on the creation process with the knowledge their game will be accessible even for a novice. However, this restricted feature set would need more depth for more explorative designs. Games created in Blind Adventure demonstrate some of the popular audio game-based techniques that are simple to access and play, such as combat in which sounds of opponents are heard in certain directions, and the user must swipe in particular directions to defend themselves, similar to the combat of the Vale. A second game Blind Adventure included is steeple chase, in which users navigate through an obstacle course by swiping to avoid sounds in certain directions. The Blind Adventure design also proposed a quiz mini-game, in which the users hear a question followed by possible answers, with the user swiping through the answers to select the correct one. Similar conventional audio-focused mini-game templates have been implemented in the Hodr Engine within a larger set of soundscapes, so users may explore an open world and play these mini-games within the world to build a more complex story.
​
Mobile device users use of swipes and taps in response to auditory icons can be translated to desktop platforms by employing arrow keys to navigate in different directions rather than relying on finger taps and swipes on different parts of the screen. This allows the Hodr Engine to create similar gameplay mechanics to those in Blind Adventure in a desktop environment.
Chapter 3 - Literature Review and Context
Literature Review and Context Conclusion
In conclusion, this literature review presents existing research into broader accessibility experiences and requirements for BVI users, research into audio techniques and play styles in audio games, and an existing audio game engine. Information from each study provides context that has led to a design for a BVI-accessible audio game engine, the Hodr Engine. This was achieved by examining existing challenges in software, and outlining screen reader supportive solutions to make BVI-prioritised game development and gameplay widely accessible. Studies referenced in this literature review underscore the need for software to be screen reader adaptive, using solutions such as semantic HTML structures, alternative text for images, shortcut keys, and auditory feedback for screen reader compatibility within software development, especially for a BVI-inclusive IDE.
To address these accessibility issues, the Hodr Engine implemented clearly labelled buttons that reference their shortcut keys, as well as customisable features for visual layout like background colour, text colour and text size. Finally, the IDE includes auditory icons to provide feedback for actions, such as navigating different sections of code, saving progress, and confirming visual changes.
​
Audio games developed with the Hodr Engine include popular BVI-accessible techniques, such as interactive soundscapes explored with up, down, left and right keys, as demonstrated in Sounds of Eden, and directional audio for combat scenes as found in The Vale. In conjunction, structured menu navigation as seen in Blind Quest is applied in the menu screens used in Hodr Engine games. EarSketch’s approach to shortcut key-centric navigation has been a model for the Hodr Engine’s use of shortcut keys throughout the IDE and audio games it creates, with keyboard commands for navigating code, selecting every button within the IDE, and operating the audio games created by the engine.