1

I'm still thinking about my project idea, and hope to have some sort of concrete design document or something soon--some sort of goal, and the like.  That or I might start smaller.
The first question today is this.  How much complexity would you be willing to put up with?  Do you read the manuals?  If I were to have 3 or four different navigational aids instead of just one, would you be willing to learn it?  How much would introducing them slowly help?  What about a tutorial level?
The other question is this.  I'm considering a camera.  In most audiogames that aren't side-scrollers, it's effectively first-person you see what your character sees.  But I'm considering a 3d platformer engine, and that means that one has to sacrifice smooth turning.  Read: you can face north, south, east, or west and that's it.  To me, first-person seems like a really bad way of doing this, so I'm considering possible camera controls (above, for example, or 3 units out) and controls that move the character, that is left arrow moves left based on the direction the camera is facing, etc.  I don't see this being a problem, but prototyping is the key, of course, and simply trying it.  Does anyone have thoughts on how horrible/awesome this could be?  My thought here is that if one is using an actual 3d audio library instead of the usual pitch/pan only, being able to control the camera separately might be helpful.
And I apologize if the bit about the camera came off as nonsense, as I'm having trouble thinking of a way to explain it to someone who has never actually seen what a game for the sighted looks like.  To use techno-programmer jargon, the listener's position would be separate from the character, with a varying up and forward vector.  But that probably only helps a few people get what I'm getting at here.

My Blog
Twitter: @camlorn38 (Try Chicken Nugget)

Thumbs up

2

Hi.
Sounds really awesome to me.
I'm whilling to read a huge manual and keep trying until I've got used to it. I have played lots of mainstream games, so I really like the idea of trying to get a picture of what a mainstream game looks like.

Best regards SLJ.
If you like the post, then please give it a thumps up.
Feel free to contact me privately if you have something in mind. If you do so, then please send me a mail instead of using the private message on the forum, since I don't check those very often.
Happy gaming... :D

Thumbs up

3

I say go for the camera, absolutely and definitely. Even 2D video games sometimes had camera controls (usually only for panning up/down, though they did other things from time to time).

I don't usually care for complicated tutorials/manuals (I'm use to the NES/Genesis games where everything you need to know to play fit in a blurb on the back of the box, and yes, I realize that DLE and Sengoku Jidai fail at said standard), but that is far from suggesting that the game shouldn't be as complicated as is playable.

Some of my games
Keep up to date by following @Jeqofire on twitter!
Ear Ninja?

Thumbs up

4

Personally, I'd rather a well designed tutorial level.  I've never had any sight, and never played a mainstream game, so I suspect text instructions could be difficult for me to understand without trying things out as they are explained.  Honestly, I'm having trouble understanding how having the listener apart from the character would even be helpful.  Is the game going to pause while I move the camera around, or something?  Why is this better than the radar systems that we already have in other games?  Why would I ever want to hear things from a different angle than my character during actual gameplay?  I guess I could use the camera to...look around corners before going there?  Isn't that cheating? 

Also, I find that personally, games like swamp are already pushing up against some pretty firm complexity limits for me.  Not in terms of interface or information conveyed, but in terms of the maps.  I don't know if this is a problem common to other completely blind from birth players, or if it's just me, but I find I have extremely strict limits to the size of maps I can keep straight in my head while playing an audio game.  Interestingly, this limit is *much* smaller for audio games than it is for games like muds.  I seem to find audible landmarks much, much harder to remember and position relative to the rest of the environment than I do text landmarks.  When I'm walking around in the real world, of course, I use a lot more than just sound (smell, feel, etc), and I don't find keeping track of where I am a problem at all.  But other people seem to have no problem with these types of games, so it could well just be a quirk of psychology unique to me.

Thumbs up

5

We're all different. Personally, i have this ability to mental map huge areas, whether it's a game or real life. As far as a camera goes, unless you have an actual viable use for the thing, i see no need for one.

Thumbs up

6

If a mainstream game has terrible camera controls, players will complain about it endlessly.
Some games allow you to switch between first and third person views (The ones I've played are the Harry Potter and the Sorceror's Stone game for PS1, and Duke Nukum 3D for PC). It's pretty much a requirement in mainstream 3D games to be able to adjust the camera.
Even the original Sonic had vertical camera panning (they took this out when it turned out to be relatively useless for Sonic, and they needed the down button for the charge dash and spin controls). The Disney games for Genesis and SNES were exceptionally designed, given that they were typically simple platformers or puzzle games; the Lion King game actually demonstrates the camera panning in the demo for the first level, and there are various easter eggs hidden throughout the game that you can find only by moving the camera at the right time (if anyone miraculously gets their sight back and wants to try this, might I recommend looking up at the end of the Elephant Graveyard?). Aladdin had some clever animations for just about everything (moving the camera involves showing Aladdin straining his neck to look in said direction, and these can be combined with sword attacks for stylistic effects, if nothing fancy gameplay-wise).

Realizing this actually felt like a huge breakthrough for me, resulting in the Swamp-based platformer. In that instance, the camera is an accessibility tool, but not in any form that wasn't available for sighted players 20 years ago.

I kinda think the camera should be a given in any project of significant complexity, hahaha.
What really concerns me is communicating the concepts nonvisually. Making audio-accessible analogs to mainstream games is nowhere near as hard as we've made it out to be (and I include myself in that "we"), but conveying things most easily understood visually without visual memory as a reference is hard. This is one reason I'm really into haptics; when it comes to third person or 2D, it's much easier to convert visual data to tactile than it is to represent it as audio.

(Representing things as audio in the Legend of Lu Chao actually presents me with a special problem: the sounds that objects make should imply some visible movement, meaning that I need some way to animate what would otherwise be static objects, so that they keep up with the sounds! Hopefully I can do this just by having them be drawn at a slight offset from their actual position on the frame when the sound plays. This could get ugly for certain objects/sounds, though...).

Some of my games
Keep up to date by following @Jeqofire on twitter!
Ear Ninja?

Thumbs up

7

While i certainly understand what the camera would be used for, as you stated, how could this be done in audio to be some what realistic without destroying a good project? Unless we are talking about Aprone's 3d headphones, how could you convey concepts like enemy on a higher level than player, versus enemy on a lower level than player. This is something no audio game in my experience has come close to representing.

Thumbs up

8 (edited by camlorn 2013-04-30 15:13:15)

Well, to try to explain:
The environment I'm considering is a voxel-based world, so you can only turn 90 degrees.  (you're not really turning, it's like JFMIA with more 3d and hopefully playable...if JFIMA had learn sounds I might be able to beat it) Standard tools like the swamp radar are basically useless, as they can't convey height.  I've considered implementing a simple version of the VoIC, but don't know how much that would help.
Anyhow, consider for a moment a 2d platformer.  You're looking "in" on the character, not out from the character's eyes.   All jumps are lined up from left to right, and everything works nicely.
But extend that to the third dimension.  I've established, in my mind, that first person really wouldn't work here, so you're now lining jumps, if you will, away from the screen.  This isn't so much a problem, save that instead of having the various scanners and what-have-you (I'm considering a safe jump indicator, and some other stuff) is going to indicate the distance of things based soully on volume.
But what happens when you want to jump towards the screen.  If I'm using the low-quality pitch and pan system, you might be able to do it, but you have to think of the navigational tools as coming toward you.
So, why not turn the camera?  You've got to jump a series of moving platforms that are metaphorically coming "out" of the screen, and the camera is facing north.  You are therefore jumping south.  Turn the camera to face east.  Now, you're playing that segment like it's a side-scroller, jumping to the right.  At the other end, there's a staircase headed "out" of the screen--that is west--so turn the camera back north and climb down it to the left.
And then you get to the big, flat, boss arena.  Which is flat.  Why not change the camera to look down while you fight the boss, playing it like an overhead 2d game, somewhat like, idk, the old Zelda (I can't think of an audiogame example, I know there is one).  It would sure be easier to dodge his projectiles that way, or so I'd think.
I'd have a tutorial level.  I'd have text descriptions of the locations.  I'd have text descriptions of the staircase along the north wall which begins on the west side of the room and the bridge stretching from north to south that can be reached by climbing it.  I would have a tutorial level, and I wouldn't just throw all the complexities of the 3d world at the player at once.  I'd wait at least 10 seconds before introducing the moving platform instead of just spawning you on one (only kidding, heh).  I probably wouldn't even put that much 3d in the first level.
And in truth, this project may be waaaaay too big for me to code successfully.  But that's what summer vacation is for.  And prototyping.

edit: Arqmeister posted while I was posting.  i'd be using OpenALSoft, which has the ability to do Binaural rendering to the quality pof papa sangre.  You can even reconfigure it, if you're a technical genius and know where to look (I'm not) to match your head more closely, by adjusting the hrtf data to your head width and hunting on the internet for a set that matches your ear shape.  See this:
http://www.youtube.com/watch?v=lqqiIR017SQ
Which isn't OpenALSoft because that video no longer exists but is instead Rapture3d, which is basically the same thing.  I have to do some profiling, this entire project could crash and burn.
Also, by using OpenAL, anyone with a Creative soundcard, or who goes and gets a Creative Soundcard can potentially have hardware accelerated 3d audio from this.  On account that Xaudio2 isn't real 3d and Direct3D no longer has the HAL, that seems to be the only way these days.

My Blog
Twitter: @camlorn38 (Try Chicken Nugget)

Thumbs up

9

I'm personally up for anything you put together. I'm programming my own game in bgt right now, which isn't the same but, i know that new project feeling, you want to get it off the ground. While i personally don't mind shelling out for some 3d hardware, not everyone is in my financial position, so you may be looking at a limited number of players. I think a ton of folks are scared by complexity lol.

Thumbs up

10

O, no.  You don't need the hardware.  you just might want the hardware.  I've not yet benchmarked OpenALSoft, but I don't think you "need" the hardware.  The more powerful the PC the more you can get out of OpenALSoft, as it does everything on the CPU, so low-end pcs might require turning off some of the features, but you don't need the hardware.  I don't have the hardware, even if I'd like it--it's not cheep, even though it is less expensive than i would have thought.
At this point I need to just prototype something.  I'm thinking I'll just try making an infinite spiral staircase and see how manageable that is.  I'm just trying to design something someone besides myself can play.  You know how it goes: the level designer can almost certainly beat the level, etc.

My Blog
Twitter: @camlorn38 (Try Chicken Nugget)

Thumbs up

11

(JFIMA was supposed to have a quasi-useful sound test, but I never got around to making it make sense. sad )

Conveying shape and size in audio is difficult, even if vertical positioning can be represented well with Open AL (someone write a wrapper for that on the level of BGT's sound pool class and I might love you forever). In addition to the examples Camlorn mentioned, a moveable camera would help give a rough estimation of shape/size, or so I'd hope. It kinda depends on how the soundscape is set up.

Some of my games
Keep up to date by following @Jeqofire on twitter!
Ear Ninja?

Thumbs up

12

Awe, simulpost!
I dunno, it took me a while to get to where I could beat some of the platforming sections in the JFIMA... which is actually a little sad, considering that they're not supposed to be that difficult (I guess it's the difficulty of keeping track of how far you've gone in the air, or something...).

If you want to demonstrate what you can do with bainaural-based 3D, a simple 3D space invaders-type thing might not be bad. You're in the center of a sphere, enemies can fly in from a random point on the surface, just aim and shoot. Unless I completely missed the point of what you were trying to demonstrate, lol.

Some of my games
Keep up to date by following @Jeqofire on twitter!
Ear Ninja?

Thumbs up

13

Camlorn, you wouldn't be limited to 90 degree turns in a voxel-based world.  I'm just mentioning this because of your earlier sentence "The environment I'm considering is a voxel-based world, so you can only turn 90 degrees.", but you might have meant the word "and" instead of "so".

I am extremely interested to hear how your binural 3D turns out!  big_smile  I don't believe many, if any, other audio games are employing binural sound so this will mark a huge leap forward.

I'm currently working on a voxel-based game as well, but mine will be from a first person view point.  It will be neat to see how the camera approach changes the experience between the two games.  smile  I think it might be a viable way to help players judge pits since they can rotate the camera to break each jump into a 2D plane.  Radars will be a bit tricky but I do wish you the best of luck figuring out what will work best.  My game only has 1 radar type planned actually, and it relies on the See Munkey headset, so it wouldn't help with your game if you are planning to use just a mouse and keyboard.

I'm generally excited to see that the current generation of games being developed are pushing us forward into 3D worlds!  big_smile

- Aprone
Please try out my games and programs:
Aprone's software

Thumbs up

14

Me too aprone, i have had a lot of experience with various friends in mainstream gaming, and what sighted players like. It's high time we blind folks get a piece of the pie too lol.

Thumbs up

15

Well, the problem with JFIMA is twofold.  First, the controls break all established convensions of controls, and the fact that you're actually controlling two characters was never really explained anywhere.  Second, the sounds for I can go up/down/etc aren't explained very well either.  i think it could be made playable, it just needs work.  Also, as a side-note: two-letter key combinations will fail epicly on some keyboards.
I know that you can have an arbitrary direction in a 3d voxel-based world.  Minecraft does this.  To make it simple enough to actually take advantage of the 3d part and have climbable structures and moving platforms and generally have it make sense without graphics, that's probably not a good approach.  If the world is mostly 2d with only simple use of the third dimension, that's fine, but the standard mechanisms for navigation break down as soon as you have a climbable wall or a staircase.  As I said previously, I think borrowing ideas from the VoIC would possibly work, but that's a degree of complexity that'd turn off many people, or so I think.  It'd also be processor intensive.  I've got ideas for scanning mechanisms, but they all rely on being able to scan a rectangular area instead of a cone.
Starting with 3d space invaders is probably not a bad idea, but I was more getting at the showing that a world with the third dimension is possible.  I should probably start small, though, and work up to this: a 3d space invaders thing, a simple tower with a staircase on the outer wall, and see where it goes after that.
I should also mention that I'm also thinking about trying to make something appealing to the sighted.  I believe that the particular constraints I would impose would allow me to make something, in the future, with graphics that would actually be appealing and gameplay that would actually be worth playing.

My Blog
Twitter: @camlorn38 (Try Chicken Nugget)

Thumbs up

16

(This is what I get for playing no PC games other than Mortal Kombat 4. I've been basing my console-style control schemes on the controls in MK4 since forever. However, you can customize the controls in the JFIMA, if they are too bizarre.)

I would expect the intermediates (ne/nw/sw/se) to be possible in a voxel-based world, at least; I wouldn't be all that keen on any more complicated angles than that for something in third person (Not without an analog stick, at least! tongue ).
Something I've always wanted to be able to do is mirror real-world navigation, in terms of object-sense, which I've been able to (completely by accident!) simulate with audio alone. This seems like something that warrants actual scientific-like research that , so far as I know, hasn't been done (unless you count me spinning around in my room with my ears covered, lol ). This was the basis of the cues I used in the JFIMA, though; I actually put a microphone near a wall and edited that sound, tried holding it over an airy gap and editing that. I think the ladder sound was me remixing the wall/drop sounds, somehow.
(*Adds "Seriously, do science-type research on object-sense" to list of things to do if winds up accepted to App Academy, since their locations are near either Silicon valley or Lighthouse International*)

Could you perhaps use a radar system more like Swamp's? Obviously, it would need to account for vertical size and gaps and such, but that doesn't seem too difficult. Actually, I remember coding exactly that; it was something that moved in a sort of cylinder around the player, and would make a sound if there was anything at its location on a given tick. It just went up and down before rotating to the right and starting back at the top.
I also experimented with a bizarre sort of look command where it could tell you whatever you were looking at, then had tones play at each corner of the object until you look away from it. You could turn the "head" in terms of both yaw and pitch, so in theory it should have been pretty versatile. (I got kinda annoyed with both of them and just tried to memorize which coordinates were the edges of buildings.)

Some of my games
Keep up to date by following @Jeqofire on twitter!
Ear Ninja?

Thumbs up

17

My current approach ideas, since there's interest:
1: For large open areas, a rectangular scanner.  It passes quickly from let to right, jumps back to the left and moves out a row, and repeats.  The width would probably be five blocks, and it would construct a tower of tones, for lack of a better description, for each sampled vertical column.  Instigated (as the rest are) with a keystroke to indicate whether to scan (relative) north, south, east, or west.  This comes back to the whole turn the camera thing, as it would be easiest to scan north which would incorporate a higher degree of panning.
2: For bridges and high places, a row scanner.  Plays sounds for the columns in a direction that are lined up with the player, useful for planning jumps and such.
3: A path follower.  For twisty paths and staircases.  Attempts to determine the next direction you are going to want to move based on the block you're on, the block you just left, and the places you can go, possibly notifying you of intersection.  For those twisty gantries and stairs that can't possibly stand under the laws of physics.
4: A number of passive hints, such as phantom footsteps (ghostly footsteps that indicate that you might want to walk this way), a variation on the fable breadcrumb trail, and a tone that indicates you can jump a gap in the direction you're facing.
And a whole bunch of other ideas that I haven't had yet.

My Blog
Twitter: @camlorn38 (Try Chicken Nugget)

Thumbs up

18

I'm not sure it would work very well.   From your description, you're thinking about a third-person overhead view, yes?

I snapped together a quick test (uses the web audio api, it should work in the latest version of chrome):  http://www.recompile.org/audio/test

You'll want to use a good pair of headphones for this.  The target and player are on the same plane, with the camera floating above, looking down.

The arrow keys pan the camera.  WASD moves your player character, represented by a blue ball and clicking sound.  A target location is represented by a red ball and a sawing sound.  Only the clicking sound starts automatically, you'll need to press "B" or "P" to start the sawing sound.

O and P start and stop both sounds.  V and B start and stop the saw, N and M start and stop the clicking. 

As I'm assuming you're sighted, close your eyes and try to align the two dots.   You'll find it's fairly simple to get both sounds aligned horizontally, but very difficult to align them vertically -- I couldn't manage it at all.

The problem I had was determining the position of each sound relative to the other.   I expect that this would make play very frustrating for a complex game with lots of sounds.

Thumbs up

19

Well, I can't get that working atm.  That makes me sad.  Does it use HRTF?  I don't have chrome.
There are other potential vertical solutions, if HRTF fails.  HRTF, from what I've found, is good, though.  That Youtube video had a lot of up/down going on, at least the way I heard it.
I'm partially sighted, that is all.  Not fully (don't I wish).
Assuming HRTF-based solutions only work reasonably in the horizontal plane, it would be possible to use pitch for height.  I'm going to go ahead and say, until such time as I am proven incorrect, that that audio API is not using HRTF.  Installing Chrome to look at that is on my todo list, and I'm going to go ahead and try to get that 3d space invaders idea working in the next week or so.

My Blog
Twitter: @camlorn38 (Try Chicken Nugget)

Thumbs up

20

I'm using HRTF, which Chrome has supported for a while now.  I'm sorry that you couldn't get it to run.  I'd tell you to try it in iOS6 but Apple doesn't yet support WebGL (which I use for the visual part) for anything but iAds at present.

Anyhow, it seems to work well if the sounds are far enough apart.  That is, it's easy enough to tell when one sound is well above the camera position and one is well below (er, and which sound is above and which below).  However, I find it difficult to determine their vertical position relative to one another when they get close.

Thumbs up

21

I'm not sure if that's a problem, or not.  I'm not going for first-person shooter, here.  To be honest, I'm not 100% sure where I'm going at this point, but I favor fast action over pinpoint accuracy, so it may or may not be a problem.
Also, for mine, the intent was that the player's position, relative to the camera, be fixed.  Since there's pretty much absolutely no prior experience to draw upon, it looks like I'll be breaking new ground whether I succeed (game is fun and playable) or fail (game sucks, even in my opinion).
HRTFs fail for distances less than one virtual meter, which might be the demo's problem, depends on the resolution of the soundscape, etc.  I need to install Chrome.

My Blog
Twitter: @camlorn38 (Try Chicken Nugget)

Thumbs up

22

camlorn wrote:

Also, for mine, the intent was that the player's position, relative to the camera, be fixed.

Ah, okay.  I was under the impression that you wanted the camera to move completely independent of the player character, under the control of the player.

camlorn wrote:

Since there's pretty much absolutely no prior experience to draw upon, it looks like I'll be breaking new ground whether I succeed (game is fun and playable) or fail (game sucks, even in my opinion).

I'm all for innovation!   I've though on and off for a while about what a game where audio only was the best method of presentation would be like -- while avoiding tropes like "music" and "it's dark".  Where adding a visual component couldn't advantage sighted players.  I've gotten nowhere so far, but if folks like you keep pushing boundaries, something like that is bound to turn up.

camlorn wrote:

HRTFs fail for distances less than one virtual meter, which might be the demo's problem, depends on the resolution of the soundscape, etc.  I need to install Chrome.

Playing around a bit, it seems that angular separation and distance from the center are the biggest limiting factors.  It's worse vertically than horizontally, which makes sense as there's less information for our poor brains to work with.  Just like when I move the two sounds far to one side, I need to increase their distance more and more before I can accurately determine if one of the sounds is to the left or right of the other -- far more that what you'd assume from the reduced angular distance alone.

My take away from this would be to keep any sort of overhead/2d perspective simple. I imagine that working from a very small grid (maybe 3x5), always fewer rows than columns, would be good.

... Or it could just be that I'm a terrible listener and you should ignore me. :)

Thumbs up

23

That makes me wonder if our model of HRTF could use further updating, since real sounds don't seem to get that blurry within a meter of actual heads (Or maybe that's just me?).

I think sighted players would typically have an advantage if there are useful graphics, just because of speed; visuals are a lot quicker to process than sounds. Maybe people blind from birth are much quicker when it comes to audio processing, though?
I'm comparing all this to the camera ideas I've been toying with lately, and it seems like the blind player's main disadvantage would be speed; they'd need to use the camera more than a sighted player would. This wouldn't be an issue in a memorized terrain, I'd imagine.

Some of my games
Keep up to date by following @Jeqofire on twitter!
Ear Ninja?

Thumbs up

24

Accept the act of having to figure out and use the camera could be time consuming, depending on the game's context. Of course, i'm still all for it, but if it's a fast action game, i would rather not have to deal with an overly complex camera system.

Thumbs up

25

From what I've seen, there aren't that many types of camera controls in mainstream games.
If it's a system with multiple directional controls (so any console made since 1995), the game might assign the d-pad or an unused stick to moving the camera.
Otherwise one or two buttons will usually handle rotation.
The 2D games with camera controls mostly just used holding up or down for roughly a second as a signal to pan. I want to think there's a game or two that will pan left/right if you keep holding the button and something is blocking you from moving, but I'm not sure about that.
And something unique to audio sidescrollers is keeping the player centered. Visually, there would need to be _something_ filling up the space beyond the edge of the level. Most games don't try that, and just fill the screen with the level itself; they keep the character centered if there is enough space to do so. (I don't think I explained that well. The point is, you can't see outside the level, because it does not exist. So the edges of the level are the edges of the possible viewing area, and if the character is too close to an edge to be centered, then the character isn't centered.)

I still think this would be easier to explain with a braille display. lol

Some of my games
Keep up to date by following @Jeqofire on twitter!
Ear Ninja?

Thumbs up