Hi all,
I'm a university lecturer and Ph.D student working on I/O technologies and I love video games.
Some time ago I saw a video of a player who was completely blind competing against a fully sighted opponent in a fighting game and winning convincingly - and it blew me away! At first I couldn't understand how it was even possible, so I decided to look deeper into sonification and learn about how audio cues in games are designed and can be used. While some games are designed and implemented with accessibility in mind, many fall short in this regard - which got me thinking: would it be possible to provide additional audio cues to existing games that might help players with visual impairment?
Cut to the present day (after twenty months of development) - and I think the answer is yes!
I've written a software application called SoniFight which provides sonification cues based on changes in the game state. In essence, if the game keeps track of something then it must be in the process memory, and if it’s in the process memory then we can access it and respond to any changes. For example, this can mean providing audio feedback to help navigate menus, or providing audio updates based on in-game events such as player or opponent health being low, or the clock getting near to zero in a fighting game, or the player running low on ammo in a FPS.
SoniFight isn’t quite 100% yet, but I should be able to release it by the end of October 2017 – at which point it’ll be made freely available as a binary release with a full suite of user documentation for players, along with the source code for anyone who may want to modify or improve it.
I should also mention that the software is massively configurable and works with config files for any given game, so in theory you could create a configuration for your game of choice that provides sonification events for any manner of changes to the game state. In practice, it’s a little bit more complicated to enact because to find a value of interest in a game you need to find what’s called a pointer chain to that value, and that can be a bit of a bother. But there are ways and means of doing so which are documented and demonstrated in the user docs, and once you’ve found a working chain it’ll work forever for that version of the game.
Oh, and sonification events can be a file-based samples like “Health-Low.mp3” or you can opt to use your screen reader, in which case you might put something like “Health low” (which would say exactly that), or perhaps “Health {}” (which would say the word ‘health’ followed by your exact health value).
If I’ve piqued your interest or if you have any thoughts, comments or questions about SoniFight then please don’t hesitate to ask me about it – I’d be happy to discuss any and all aspects of the software.
Cheers!