2024-02-20 11:42:45

hello there,
what may or may not have happened otherwise if steam didn't enter the linux market, it now happened. Steam audio, the whole thing, is now open source. This means, essentially, that we have one more audio library to play with, and this one is pretty advanced too, because you can get a sense of modeling objects and such with audio, though it's unclear to me whether we can access that aspect of it while blind. In any case, here's the link I got from mastodon on the matter
https://steamcommunity.com/app/596420/e … 4663731579
hopefully this will spark some discussions as to if this has the potential to change the audiogaming atmosphere at all, how could it best be made use of, etc.
with that, discussions open, I'm out!

2024-02-20 16:38:54

is steamaudio fully accessible on windows?

discord: hellblade09

2024-02-20 17:00:33

what is steam audio?

2024-02-20 17:27:30

it's not an application, at least not from what I know, it's a programming library that helps audio playback in 3d space

2024-02-20 18:17:38

Yes. It's accessible but someone will have to wrap it in something more convenient before most of you can use it.  The more advanced things such as automatic reverb and the like require integration with engines that most of you don't have the math for and they also only work for first person stuff, so even with a simplified binding you'll have that problem.  It's always been possible to wrap it--STW does--and being open source doesn't change that either way.  You'll also have to be using a fast language to use the really advanced features, which require raycasting in realtime (Python is too slow for this).  If you want to make space invaders or something like BK3, Shadow Line, or Mist World, this isn't the library for you.  It would, however, go well with Swamp or something along those lines.

My Blog
Twitter: @ajhicks1992

2024-02-20 18:22:04

@5
C#? Also, mw has hrtf? That's a surprise, most told me it doesn't, I don't know myself because it's sorta hahrd for me to tell the difference at all.
So steamaudio is something like fmod huh, core thing that makes it special hard to use for us, go figure.

2024-02-20 19:37:19

No, MW doesn't have HRTF.  I'm saying that this library does not work for MW because it is designed for first-person viewpoints pretty much exclusively.  Pathing reverbs and the like don't make sense for non-first-person viewpoints and (for example) unrealistic environments without ceilings.

C# is fast enough.  C# is within a factor of 2 or so of C and also doesn't have the global interpreter lock and half the modern videogames (anything in Unity) are written in it.

It's not like FMod.  Someone could integrate it.  Sam Tupy did.  It just requires being the perfect kind of game and knowing the math needed to convert your geometries into what it can consume.  I don't think STW even uses half of it based on what STW sounds like when one gets close to objects.  I'm pretty sure he's mostly just stopped at using it for HRTF.  I could make this work if I wanted.  Unlike FMod it's not do complicated things and then you don't have studio and you did all those complicated things for little gain.  You do actually get benefit out.  Indeed SteamAudio doesn't have anything even close to studio, I think.  But there are two complications:

First is that HRTF and the like only make sense if you're "looking out of the character's eyes".  Even with OpenAL or Synthizer, doing a sidescroller or top down or something and trying to use HRTF is a way to just get frustrated.  People have, but it's usually carefully hacking the listener settings until it sounds just right and even then it's not really so great.

Second is the input data.  If you think about Swamp for example it's really a 2D map.  When you go into the shopping center or whatever it's really just the walls.  There's no ceiling in the map data.  You might as well think of them as giant vertical cliffs that go up to space, and the entire thing is specified as tiles.  With the raytracing-based libraries like this, that's not what they need.  They need realistic geometry specified typically as triangles.  like, you can't go no ceiling and the church is an infinitely high pillar made of stone, you have to have the materials and model it properly shaped as a church with windows or whatever.  And then if you want to simulate all the things based on moving objects or closing doors, you have to actually have modeled doors that close and modeled objects that move realistically.  You can do all this in an audiogame, where by you I mean me, because there's enough ways to fake it off tiles if we wanted.  But it's a lot of subtlety.

As I said--someone would have to wrap it.  You could possibly even wrap it in a higher level C API that takes tile grids and does something smart, I dunno.  But it ain't trivial.

It's not a Synthizer basically.  It does one highly specific thing well.  It's intended to be integrated with other things that do the rest, e.g. background music through the rest of FMod when using the FMod integration.  It does first-person viewpoint HRTF stuff super, super well.  But...that's it.  That's what it's for.  I don't think it even has file reading in it.  I'm 99% sure it can't even do audio output for you.

New Synthizer will probably be able to let someone write custom nodes, and so for example someone could work all the hard bits out and then mount it into Synthizer.  Which is what steamAudio is for, that's how it's intended to be used.  But see above about all the hard stuff.

My Blog
Twitter: @ajhicks1992

2024-02-20 21:41:06

@7 is correct, it does not produce audio output or decode files, it operates on raw and already decoded data of 32 floating points, so to use it, you first need to read the file, decode it, render using steamaudio, then use the result to feed into some sound mixer like Bass, for instance. Right now, I'm trying to use it with FMOD through the plugin, but even so, it's proving to be a quite challenging battle.

2024-02-21 01:13:51

Using it via the FMOD plugin is annoying because they say that you need to call a function in the library to initialize it, but of course they don't provide a header that you can include, so you have to manually load the library, call the symbol, and hope for the best. I don't know how well that would even work, given that FMOD and your code would have it loaded, which may or may not work.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2024-02-21 01:25:35

I know Windows at least only loads assemblies once even if you load it a second time. I think Linux and Mac are the same.  But now, it's open source so that shouldn't be a problem, just build it yourself however you need it.

Surprised there's no headers though, are you 100% sure you didn't miss something somehow?  that's a huge oversight, Valve and anyone who's capable of building this in general are way more competent than to not have headers.

But I mean good luck I guess, this is why I don't fuck with FMod.

My Blog
Twitter: @ajhicks1992

2024-02-21 01:31:32 (edited by Ethin 2024-02-21 01:33:22)

@10, it's not FMOD that's the problem, which is why I'll keep using it. It's that Valve doesn't have headers for the FMOD integration part. The headers they do provide are just for the SteamAudio core, and doesn't include the FMOD-specific functions you need to call to initialize the plugin after loading it into FMOD (i.e. to initialize a context for FMOD usage).
Edit: to be clear, these functions are internal to SteamAudio. FMOD doesn't expose these because they're things you need to call within the SteamAudio FMOD plugin library (something like IPLFmodInit, IPLFmodTerminate, etc.).

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2024-02-21 01:53:30

So why not just use the SteamAudio FMod plugin?  I mean FMod surely has the ability to initialize standard plugins?

Shrug, you do you I guess.

My Blog
Twitter: @ajhicks1992

2024-02-21 02:31:26

@12, because if you look at the docs, it says you have to call a specific function. FMOD does but they make it so you have to call some function in the lib.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2024-02-21 07:17:19

@7
YEAH, i SAW THAT, i SAW AND UNDERSTOOD HOW YOU NEED TO HAVE RAW PCM THEN THROW THAT PCM ON THE FACE OF STEAMAUDIO, It DOES SOME VOILA MAGIC THEN GIVES It TO YOU BACK, AFTER YOU WRITE SOMETHING LIKE 30 LINES OR DUNNO, THEN YOU SOMEHOW PUT It BACK WITH the audio library you want to use, then somehow syncronize the changes of 3d space with it. I saw all of that, and I went hell no. Especially after I heard how you said that literal hrtf is never goign to work with how audiogames are done, with player being a point on the map and all that.
I'm curious though. I've heard many saying openal soft is very very good with it's hrtf, as I said I absolutely can't tell the difference, so how? IS it really better than synthizer's?  I do remember camlorn saying that he added something like openal's hrtf so not sure.

2024-02-21 08:20:50 (edited by Still_Standing 2024-02-21 08:29:08)

it's possible to use it along side miniaudio, miniaudio provides the audio decoding and playback while steam audio processer it, i've only used it for basic hrtf effects, however i think it's possible to do more with both
@camlorn, what kind of math you're reffering to?

And as anyone who's gone mountain climbing knows ,The serene snow-covered peaks that look so tranquil from a bdistance, Are the deadliest
sound is my vision
i rarely check my private messages on the forum, so if you want to contact me please use my email, or dm me  at oussama40121 on tw

2024-02-21 16:29:13

@9
Have you been able to use it through the plugin? I haven't managed to yet, and it would be good to have some tips. I can share my initialization code if necessary. I compiled the plugin myself, got the .lib and .h files, and forced inclusion because previously, just depending on fmod, the dll wasn't even loaded.
And the documentation is extremely confusing. Both the plugin's and the official steamaudio documentation. I hope it improves now with open source.
The plugin was already open source before, and I did these tests a few weeks ago, so I had to set them aside to solve things at my internship, but I want to try to make it work again soon.
Things with SteamAudio are so strange that not even the official usage example worked properly. I don't know if they fixed it or not; basically, it just produced silence, and I even had to open an issue on the official Github for them to find out that they forgot to add a hrtfSettings.volume = 1.0f; in the example code.
So basically, you need to read the documentation, do your tests, and keep your fingers crossed for everything to go smoothly...

2024-02-21 17:39:29

@14
You seem really confused.  HRTF is good for first-person games like shooters.  That's definitely something we have.  The player being a point isn't the problem.  The geometry of the world being unrealistic is the problem.  Walls aren't perfectly smooth.  Doors don't vanish when opened.  Things aren't perfect squares.  So on.  Think about Shades of Doom for example.  Corridors only have a roof in our imagination, in the code it's a 2d map, there's no height involved.  Let alone textured surfaces, curves, windows, etc.  All of that is our imagination, in every audio fps I know of.

And then of course for non-first-person games yes it falls down.

Default OpenALSoft HRTF is as good as SteamAudio.  Tuned OpenALSoft HRTF if you take the time is arguably better because if you process the dataset with more demanding settings you can really bring out the spatial part at the cost of some realism.  But OpenAL as a whole is a really hostile API with limited flexibility.  I never got Synthizer as far as OpenALSoft's for a variety of reasons, not least of which was the lack of engagement from the community.  I intend to revisit that in the Rust version.

@15
The more advanced reverb stuff involves baking map geometry or doing realtime raytracing which requires having good geometry to bake whether you bake it up front or do it in realtime, and you must be able to do raycasts against it efficiently.  I believe that the library itself does that but is opinionated on what you feed it.  All of the stuff that's not basic HRTF that they advertise relies on this working.  Audiogames maps are almost always tile grids with no concept of height, let alone realistic world features.  So, you have to somehow get from something convenient for blind people to work with to something convenient for the library, and that probably means (for example) figuring out how to add in all these missing features.  Then at runtime you can't cheat with moving objects, they have to actually be shapes and can't just be points like most games do.  This all almost certainly means working with triangle meshes and 3D models, all be it ones generated from simpler descriptions.

Don't get me wrong, some of it is going to work without, but in terms of using the library beyond what for example OpenALSoft gets you, you need that kind of thing.  Otherwise you might as well just hand tune reverb parameters.  Maybe their API is much better for that than OpenAL, but if we're going to talk about it moving the needle and changing the future and all then it needs to do more and how you get more is the above.

My Blog
Twitter: @ajhicks1992

2024-02-21 17:57:26

@16, I haven't, no, because though I loaded it into my apps I didn't know for a while about the manual init thing and assumed it would auto-initialize. I could probably figure it out now; the last time I tried it it was a while ago, but still, the fact that they don't provide headers for that in the (public) API, and you have to go fishing around in the sources, is annoying.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github