2021-03-15 13:17:41 (edited by Tangle 2021-03-15 13:18:29)

I've been experimenting a little with first person spatial audio. Just a small hobby project, but I'm trying to make exploration and navigating and interacting with the world as obvious and approachable as possible to untrained ears of sighted people like me.

I've wondered if it would be helpful to constrain the player movement to grids or nodes, and the turning so you snap to facing one of 4 or 8 directions. I can imagine a lot of benefits to limitations, but I'm having some doubts. Waypoints can make it easier to form a mental map of the space as a network of nodes. A grid would be similarly helpful. But maybe it's more useful being able to walk around freely and probe the edges of the environment or changes in surface. Or repositioning slightly to "triangulate" the origin of a sound.

If you're only ever facing cardinal directions or something more granular, you can reliably turn exactly as much as you want and not over- or undershoot. But again, turning slightly might be useful for locating sounds and there's lots of ways to implement audio landmarks or a compass. Maybe all you need is to make it really clear when you're facing an object directly with cues.

Of course the environments and objects will have to be laid out to suit the approach. If movement is constrained to cardinal directions, paths and corridors will also have to match. If turning is limited, then objects and connections to other nodes must be arranged appropriately around each node.

I definitely need to try more audio games where you navigate with spatial audio, and prototype more as well. But I'd love to hear some insights or opinions on this to give me a head start.

2021-03-15 15:40:39

So, here's the thing you probably just missed.  We're always going to be better at this than you.  When you make an audio environment work for you and other sighted people, you make it simple by the standards of many of us.  In general, this is a mistake I see lots of sighted audiogame devs make.  Even the most complex games we have are simpler than what the real world is for us every day, and what we consider doable or easy in this arena is your equivalent of hard to impossible.

That's not to say there won't be blind players who will love it for the simplification--lots of blind people go blind later and don't have good spatial reasoning/audio processing/etc.  But you'll find that on the whole the audiogaming community can handle more complex stuff and is a pretty young contingent who didn't lose their sight at 40.  I'd look into Swamp (paid, unfortunately) or Shades of Doom for some idea what first person can be, and BK3 if you want to see what an audio sidescroller that's as complicated as a sighted one can be like.  Even just the readme for Swamp will probably be incredibly informative with respect to what typical approaches to the first person stuff are.

But perhaps better than that, at least for making the point I'm trying to make: take a look at audioquake, a mod that made Quake accessible.  audioquake.  Being able to play audioquake is...I guess the best way to put it is a talent.  But there's a small contingent of blind people who used to manage Quake with that even though it was 3d and absurdly complicated.

It is more common for people here to complain about the simplicity of games than to complain that they're too hard.  Most of us don't want simplified experiences.  Many of us are actually frustrated that the community doesn't have the skill to do complicated things.  We don't get many FPSs not because it's an unsolved problem, but because most audiogame devs don't know trig and vector math.  By the same token, we don't get complex games because most people here don't know software engineering.  If you know both and you want to produce something, I'd head in the other direction, or think about what kind of game you can make where blind people and sighted people can play on equal footing without having to compromise on the blind person experience for the sake of letting people such as yourself be able to manage navigating without graphics.

My Blog
Twitter: @ajhicks1992

2021-03-15 16:36:53

I think that limiting players to a grid can definitely make the understanding of movement and spatial positioning more obvious.
I have heard that Undead Assault is an FPS that restricts you to grid movement and turning, though I have not experienced it properly myself.
That said, I think it takes something away from the aesthetics of the game. Further, first person shooters and other action games can benefit from the mechanic of precise aiming with the mouse.
Swamp would be a good game for you to try, since it has a free single player mode, and is a first person shooter with some very low vision graphics you can turn off.
A Hero's Call (AHC) would be another interesting one for you to test, though it does not have a free demo.

In Swamp, aiming with the mouse gives the best experience for aiming at zombies and making precise shots.
But the hot keys for turning snap you to face cardinal/ordinal directions so you can quickly orient for navigating.
Moving into a new region speaks the name of that region.
The maps come with built in beacons for the interesting locations, and you can enable one to get a direction and distance announcement, as well as a beeping sound in the direction of the beacon as the crow flies.
There is a configurable radar which plays repeating beep sounds on points near your character, using different sounds to indicate walkable versus wall terrain.
My descriptions will not do any of this justice. You should definitely test the game for yourself.

There is no need to aim in A Hero's Call, so using the mouse is not really important.
That said, it still uses continuous rather than discrete movement and allows you to use the mouse to turn to face any angle.
The hot keys for turning only snap you to cardinal directions.
It uses the same region announcement as Swamp.
It also has a forward speech scan that announces changes in region and blocking terrain in front of you.
I think the automatic forward speech scan is too verbose and detracts significantly from the atmosphere of the game (and I am the one who conceptualized and built that feature), but I still think it is nice to be able to press a key and find out about things in front of you at a distance.
It is a worthwhile feature. I just think the automatic version should be disabled by default to help the game's atmosphere.
It also has a different style of radar than Swamp. Instead of constantly beeping around your character, it only plays beeps to indicate changes in walkable versus wall terrain around you as you move.
Imagine that you have a wall to your left and are moving forward.
Suddenly the space to your left becomes more open as you pass an alley way, so a sound plays to your left to indicate that the left is now more "open".
As you keep moving forward you completely pass the alley way and now have a building to your left, so a different sound plays to your left to indicate that the left is more "closed".
I conceptualized and built that feature because I would sometimes miss a doorway to my left in Swamp because the beeping sound takes time to cover all points and you can easily walk passed a doorway even when you were trying to listen for it.
I think it is a good feature, especially for navigation inside buildings, but may have inadvertantly led to AHC maps being built to be too maze like, as though everywhere is in doors.
AHC also has a beacon system, though it uses path finding to give players precise directions to their destination, rather than a beacon as the crow flies as Swamp does.
When enabled you will hear a voice in the direction you need to travel, saying the name of the cardinal/ordinal direction you need to travel.
Swamp's author intentionally chose the crow flies option because he wanted players to retain the challenge of finding their way around buildings and other obstacles, but to still know the general direction they needed to travel.
We intentionally added the path finding option to AHC to make things as easy as possible for players who have difficulty with first person navigation.
Both games use maps built on grids of terrain tiles, though they use continuous movement.

I agree with Camlorn that most blind gamers who have been blind for several years will be much better at navigating audio environments than sighted players.
We have just had much more practice at it.
That said, I have known gamers who were blind from birth who had very poor spatial reasoning, and as he mentioned, gamers who become blind later in life will have had much less practice.
You can see a similar difference in what practice can do when listening to text to speech (TTS) speeds.
Listen to some TTS on your computer at a speed that is comfortable for you, then listen to some recordings of blind people listening to TTS.
We usually have it at much higher speeds. Speeds that cause most sighted people's initial response to be "You can understand that?"
Though even within the blind community there is a lot of variation. I have heard a recording of Camlorn listening to TTS, and I could definitely not use his speed as my normal day to day speed, yet most sighted people cannot understand the speed I use.

I also agree that sighted developers usually create games that are too simple.
The most common action fighting game for a sighted developer to make is a bop it style game, where your character stands stationary and you hear an enemy in the forward, left, or right directions, and then have to press a key to block or hit them in that direction.
I believe Undead Assault was made by a blind developer, but my guess is that it is grid based because he didn't value aiming with the mouse.
Blind people don't use the mouse in day to day computer usage, and some don't even own a mouse.
Swamp helped introduce a lot of the blind gaming community to the mouse for first person shooters, and it was an uphill battle to get people to switch.

BK3/BD3/Bokurano Daibouken 3 is the best platformer we have, and the game is free, though there is a small fee to purchase the English translation.
ShadowRine/ShadowLine is the best top down old school Legend of Zelda like action RPG we have, but it has no English translation, so you have to use a screen reader addon to play it in machine translated English, and the setup for that is probably more than a sighted developer would be willing to do.
I think Swamp is the best FPS we have, though I did not play Shades of Doom very much.
The Blind Swordsman was made by a sighted developer. It is real time stationary combat, but with continuous turning, and very nice sound, music, and voice acting.
Also see Castaways and SoundRTS for good examples of real time civilization building and real time strategy.

Hope this helps.

~ Ian Reed
Visit BlindGamers.com to rate blind accessible games and see how others have rated them.
Try my free JGT addon, the easy way to play Japanese games in English.
Or try the free games I've created.

2021-03-15 16:52:39

@1
I would be willing to record some gameplay of games you don't want to buy just to try them, to give you an idea of how they work.

2021-03-15 17:32:54

Surprised no one has mentioned Code Dungeon yet.

https://stealcase.itch.io/codedungeon

Anyhow, agreed with most of what has been said here. Let your gameplay, not your views on audio navigation, dictate how traversing your world works. The next commercial game I'm working on will be top-down not because of simplicity, but because it's a roguelike-ish thing. But The Last Of Us 2 shows that even complex navigation is at least possible, though I do wish it exposed even more than it does, and think it's a great first effort rather than an ultimate end goal. I'm also mentally prototyping 3-D spaceflight in audio, though I haven't yet knocked together a sandbox to see how practical the ideas actually are.

2021-03-15 18:00:31

Let's talk about Wayfar 1444.  Wayfar 1444 has lots and lots of problems.  It's a text-based mud written in moo and with terrible lag and the admin doesn't seem to know what he's doing.

But it's basically factorio except the worlds are smaller and minus most of the fun mechanics.  People play it because there's no accessible Factorio.

People love Shadow Rine, which is a link to the past.  I had enough vision for a link to the past, so I know what I'm talking about.  They're of similar complexity, though Shadow Rine is slower because some of the accessibility navigation tools and audio don't quite let it be as fast, and there's a few things it doesn't do.  In order to participate in this experience, people install hacky screen reader add-ons that send the clipboard through Google translate because it's Japanese only and basically the only thing like it there is.

There's another mud called Conquest.   Yet another the admin doesn't seem to have a clue, so much so that the mechanics incentivize not logging in (yes, really).  People play it because it's the only online game we have with a reasonably complex combat system now that Godwars 2 is dead, even though 90% of the experience is you set up stuff and walk away for a week.

People put up with a lot just to get at a reasonably complex game around here.  I could probably list off more.  There's maybe 20 games of reasonable quality, from the perspective of putting stuff next to what sighted indie devs do.

I think it's reasonable to sacrifice/simplify navigation if you're focusing on something else.  If it's some really complicated RPG with cool combat or something, maybe.  Shifting the complexity around so that thing a is simple because thing b is really complicated makes a lot of sense.  But planning to remove complexity from the start is kind of defeating yourself before you've even begun, when it comes to the perspective of enticing us.

That's not to say that fixing the UI is unwelcome.  Every sighted game ever has that breadcrumb trail to get you to your quest objective nowadays.  There's a huge problem with UX.  Something that I'll bring up again here, since you probably won't otherwise see it/figure it out: all of us with the skill to do games justice don't because at that point we have the skill to have a programming dayjob.  This definitely applies to me.  Consequently most of the games are written by people who haven't really had any experience, and the UX follows.  But the answer isn't simplify, it's like: use a proper translation framework, so that people in any-language-but-this-one aren't using the clipboard to do shitty Google translations; provide the aforementioned quest guidance; if it's first person provide a third person map like what BK3 and Shadow Rine do.  And so on.  There's lots of room for innovation.  I'm just not sure how much of that is compatible with "I'm sighted and I also want to play without graphics" because that's equivalent to "I'm missing 20 years of practice dealing with this".

I live alone in downtown Seattle.  No help from anyone sighted for months because of the pandemic.  The same skills that make this not that big of a deal, in particular the ones that let me walk to Walgreens, are the same skills audiogames demand of me.  I know that this is objectively a difficult thing, but the real world version of this is mildly annoying at best and not something I really even think about, and the audiogame version is against an already-simplified world with bunches of navigation metadata added or whatever and which is also designed to be actively fun.

Hopefully this provides a bit more perspective.  I'll stop now, until there's more concrete questions or whatever else.

My Blog
Twitter: @ajhicks1992

2021-03-16 02:56:49

Thanks for all the input! This is really helpful.

I have played some, and watched quite a few videos of people playing audio games and it's been informative, but sometimes I find it hard to follow what's going on when I'm not aware of the player input. I kind of have to experience it first hand.

About blind players simply being better... Yes, I did miss that at first, but picked up on it when researching audio games. I appreciate that you bring it up because it's an important point. But like I said in my original post, this is a small project. I'm making it for myself first so I can play and appreciate it and learn from it. I don't have any illusions that I'm doing audio game players a service right now.

My possibly unrealistic ambition here is to make a little demo where you can walk around in an environment at your own pace, talk to NPC's and interact with objects. I'd like to be able to hand it someone who has never tried audiogames and while they will probably struggle at first they can play through the experience and focus on the narrative or puzzles and not just the challenge of navigation using spatial audio. I don't want the lack of graphics to feel like a gimmick.

I am definitely taking on board what you're saying about making games for both blind and sighted players rather than trying to make a unified experience that is worse for both. This project will be limited by that exact compromise, but I'm hoping that the experience will make me more equipped to make accessible games in the future, or potentially even collaborate on an audio game with someone blind and experienced with it. But assuming it's even feasible to make a mainstream-friendly audio game, this could just be a distracting rabbit hole and I'd be better served focusing on what I know.

That aside, I have to try out some of the suggested games and process all the great insights before I get a feel for what degrees of freedom and constraints I'd want to experiment with. I have time to consider whether it's a fools errand.

If anyone is curious, I use Unity and I've been learning Audiokinetic's Wwise audio middleware. Unfortunately I'm working a lot right now so I'm mostly just game jamming and tinkering on the side.

2021-03-16 03:18:37

It's definitely possible to make games that both blind people and sighted people can play on equal footing.  I'm just not sure that FPS is a genre where that works.  It would certainly be hard at best, because while we can handle stuff vision still wins just due to the reaction time.  I am too young to have been around for audioquake but I have friends who were there, and apparently how that ended is that blind people ended up making maps with no light sources so that sighted trolls couldn't drop in and say hi.

If you want to see blind-sighted game equality ending well, you might look into Sequence Storm (rhythm, accessibility is built-in) or Slay the Spire (via the mod say the spire).  I think there's some others that do it to a greater or lesser degree.  There's Last of Us II, but opinions on that are very mixed and "a good first effort" is probably apt.  I think a game called Skullgirls (mortal combat clone) implemented screen reader support.  It does happen, but it's the kind of does happen where I definitely won't run out of fingers counting them.

I don't want to hammer the point more than I have, as long as you know.  For what it's worth usually what happens is it's a grad student or something without a clue, who does the thing and never actually gets this no matter how much we point it out.  that doesn't just apply to audiogames either.  You're possibly literally the first person in my experience to say "I get it but I want to learn, and I'm using this simplified version to do so".  You get a lot of points from a lot of us for getting that far.

In so much as we care that you're using Unity, it's that we can't.  I'm actually the author of this library for 3D audio, which mostly exists because things like Wwyse etc. are inaccessible as hell and everything wants more money than most blind devs have (seriously: the only other good open source HRTF is OpenALSoft).  No one is going to judge you for it, but you won't be able to collaborate with blind people in any meaningful fashion if those are your tools.  Unfortunately for us, fact of the matter is that those tools are very impressive tools that let you do in days what takes us weeks.  Perhaps I shall eventually succeed at building the audiogame stack I want to build, but practically speaking I'm in the market for a time machine so who knows.  At least I've solved the audio problem.

My Blog
Twitter: @ajhicks1992