I'm pretty sure there was something called "The sound", "the voice", or something to those regards, that was talked about on here a few years ago.  If more has been done in that area, I'd be interested to learn about it.

It would be awesome if we could here images with sound. That would be epic!

Just to be clear, if anyone isn't quite understanding what Magurp is saying:
The vOICe is much, much better for real-world first person.
I've played with it a lot in the hopes of making 2D games more accessible, with very little success.
But the one time I tried walking around my house and outside with it (using my laptop's webcam), it was much more helpful.
Generally speaking, I expect my primary use of the vOICe anymore to be if I make videos with my computer, so I can hear if I'm in frame or not.

I might try to throw together an example of what the Swamp Radar might sound like if it were handled by the vOICe. I think I still have sonified screenshots from Sonic3 in my dropbox somewhere.

That depends, but I think I should go into abit more detail on how it works.

The vOICe software can only convert images in to Black and White, even if the original image is in color. The time it takes a sound to play represents the X axis of the image, so a pixel on the far left of the picture would be at the very beginning of the sound, and the pitch of the sound represents the Y axis of the image, so a very high pitch means the pixel would be at the top of the image, low pitch at the bottom. The brightness of each pixel is represented by the volume, the louder the sound, the brighter the pixel, the quieter the darker the pixel.

Now this means that moving in a real life 3D space with shadows and lighting creates Contrast and provides a sense of Depth, allowing you to more easily make out shapes and determine distance between them by how loud/quiet parts of the sound is, when it plays and what its pitch is. In sighted 3D First Person games its less effective because alot of them have crappy lighting, it often being just as bright down a hallway as it is right next to you among other details, so when converting the image from black and white in to Sound you get little contrast and can lose your sense of depth and positioning.

Its often worse in 2D games, which often have no shadows and use a lot of bright colors, making it very difficult to make out whats going on.

This doesn't necessarily mean its impossible to make 2D and 3D video games using The vOICe, but I do think it takes a very different approach compared to sighted games.

@sik
This is the exact approach i've taken with my own 2D engine, I've rendered the background black and the terrain as dark grey, and the player as a white block to help give contrast to make it easier for the player to know where they are in the environment, combined with positional feedback like sounds for walking, bumping into obsticals, and jumping. I've also tried embedding the Audio Renderer into the game itself and wanted to try incorporating different musical instruments to represent colors when Sonifying each screen to help identify different kinds of objects and opponents, as the vOICe can only do black and white.

@CAE_Jones
Hmm, I think you could try adapting your program to make a Match 3 game like Candy Crush, since you can tell what color your on and what adjacent colors are next to it, you then press a button to swap the colors and try to get 3 squares of the same color in a row. You also don't really need to know whats happening on the rest of the screen until you pan over it, and it would work well on a tablet. Or you could potentially adapt it into a Maze game, where you have to trace your way along lines of a particular color to get from start to finish, maybe even something like packman perhaps. These may depend on whether you can tell what position the adjacent squares are relative to your current position though, if thats the case, there could also be other potential applications for it as well.

Color in Audio Games I think is best used as information, which is why I want to incorporate it. To give an example, lets say that the color red is represented by the sound of a trumpet, the color blue by a piano, and the color green by a drum. Now lets say your playing a side scrolling game and you hear a trumpet off to the right, that represents a powerup. Hear a drum further behind it? That's an enemy. Hear a piano off to your left? That's a door. Instead of associating each sound with a particular color, you associate the sound with a particular effect or object, the color in this case is purely arbitary and used by the game itself for data and tracking purposes, the players will never know what color the sound actually represents, and they don't need to. Audio games already do these sorts of things to a degree with 3D positional audio and representational sound effects, the difference is that this method maps those effects precisely to a position on the screen with greater accuracy.

This has a lot of practical applications, for example in theory someone could make a top down Real Time Strategy game like Dune 2 or Command And Conquer with different factions represented by different solid colors, by using the above method a non-sighted player could then tell where all the units are on the screen and who they belong too in a sweep because of the specific sound of each factions color.

The real difficulty seems to be two things: information overload (sound is nowhere near as parallelalizable as sight), and distinguishing signal from noise. In a room filled with people, it's easy to pick out an individual face, if one has perfect vision, but even with perfect hearing, picking out an individual voice if everyone is speaking is extremely difficult.
The vOICe's solution is to organize data temporally--instead of getting everything at once, you get a column at once. It can still be messy, but once everything is playing on a predictable interval, it becomes easier to search for the desired information.
I'm still not sure how well this would work on an RTS scale, though.
The vOICe comes with a Tic-tac-toe game, which is very easy to pick up. Would something similar work for, say, Chess? Or even just Checkers?
I have no idea; it seems like trying to search columns of 8 (well, Checkers is more like staggered columns of 4) woul dbe much harder.

Actually, that's probably worth an actual experiment. Try to determine how much information can fit into a single column and still be usable--bonus points for speed and a decreased learning curve. It's probably possible to design a game based around that.
(I would consider doing it myself if I felt like setting up an online scoreboard to track the results. )

I think this is relevant to this discussion.

Yes, I just went ahead and gave it a try, after a tad of tweaks to the rendering engine to remove as much superfluous information as possible (during the early tests it resembled CrazyBus). It's preliminar work and needs tweaking (as well as then adapting the rest of the interface) but it's something.

I have the full blown audio renderer already implemented.

First the game renders things in a completely different way: the level gets turned into a solid white silhouette, while objects get turned into colored blocks (with colors for player, goodie (items) and danger (enemies, hazards)). Then this gets filtered so only lines at the top and bottom of each color area is stored (otherwise it becomes a mess), and this gets processed as follows:

Horizontal position: panning
Vertical position: frequency (pitch)
Color: waveform (how it sounds)

Note how unlike vOICe I'm not using time as a factor, so I can update this at 60FPS without problem. So far I only got a person to test but the little testing so far seems to suggest that yes, the idea works (and now is in the process of tweaking).

If you want to test just follow the contact instructions and I'll send you want you need over e-mail. (I'd have uploaded a recording of how it sounds, except for the part where the recording code still isn't setup to record this output)

@Sik: could you upload the recording and provide a link? I'd be really interesting in listening to it.

@Sik
Ahh, yes that makes sense since were working strictly with square blocks instead of abstract or 3D shapes.

I like your stereo implementation but there may be a few things to consider. I think the vOICe does have stereo panning support, and I know Meijer has HiFi stereo code on his site that i've only just started to reverse engineer. His code sample has source for both Mono and Stereo output, and I think he may have implemented it this way because some systems may not have stereo support, but even if they do some non-sighted people may not be able to fully perceive stereo because of partial or total hearing loss in one or both ears, have cochlear implants, hearing aids, etc. So, for example someone with partial hearing loss in a stereo setup could end up with perceptual gaps in the rendered scene, or in a mono rendering case it may be difficult to orient without a point of reference.

It's also difficult to say how using different waveform's may effect accessability with the above issues either since I don't think the vOICe has ever been tested with these parameters. Either way, getting a wider test group among different ranges and hardware will be important to weed out and determine whether any these may cause a problem or not.

For reference there is another program based on the vOICe that use color encoding called EyeMusic developed by Amir Amedi (http://brain.huji.ac.il/site/em.html), which creates soundscapes with musical notes instead of tones. Theres not alot to go on other than a video on his youtube channel and some research papers, though it can be bought on itunes.

Wow, that kind of sounds like a Lucas Arts adventure, hehe. But yeah, the audio seems abit harsh at around the 2 and 4 seconds mark. 

Its hard to tell whats happening without abit more context, though I can tell your jumping twice. Look forward to your play through.

I've noticed that problem in some of my builds too, seems to be the signal to noise ratio CAE_Jones mentioned. It gets too hard to distinguish between competing sounds in close proximity, which could prove problematic when using different waveforms/layers in a single soundscape.

Hmmm.. I think I have an idea that might help, i've recently been working on proximity audio with conventional audio cues to work along side of, instead of in the soundscape. So for example the soundscape sonifies the terrain, but when the player gets close to, say, a body of water, the closer the player is the louder the sound of the water, further the quieter.

You could try making objects volume proximity based to the players position, or make them proximity based to each other, so a cluster could reduce their collective volume based on adjacent objects to a more readable level. So when processing your image data for audio rendering you can adjust the brightness levels of the pixel data.