This is the Synthizer thread, where I update from time to time with progress and try to answer questions.
For those who don't know, Synthizer is my third attempt at solving the 3D audio problem. I've got several more years experience under my belt from Libaudioverse and I know what I did wrong there, but unfortunately those mistakes make finishing Libaudioverse harder than doing this. So I'm doing a game-specific library that drops a lot of the Libaudioverse "you can build any sort of synthesizer" features in exchange for speed and maintainability around my dayjob.
For those who do know, this isn't dead. It's maybe 5000 lines of C++ and Python for HRTF and wave file reading. I've been plucking away every weekend, but I didn't have progress to show, until now. As I said this is certainly an undertaking.
To cut to the chase, have a demo. This is Jewel Thieves from Sound Image going in a clockwise circle around your head. It's not perfect, there's still a little bit of buggyness with the panning and some artifacts, and going through virtual audio cable certainly didn't help. But it's better than (what used to be and I think still is) OpenALSoft's default settings and it's certainly better than Libaudioverse.
So: to answer the obvious questions. Elaboration below. It will probably be usable from BGT. Not by design, just because the design of this overlaps well with what BGT can call. You can't use it yet. I estimate 3 weeks until you can. It's very, very, very fast and I haven't even optimized it yet. Reverb isn't here but will be coming. You can't customize the HRTF but you will be able to customize some parameters, most notably the speed of sound and the radius of the listener's head. Currently supported file formats are wav, but MP3, Ogg, and Flac are coming.
License is the Unlicense and I'm taking care to only use permissively licensed components, so you can do whatever with it without even crediting me. I ask that you do credit me, but you don't have to per the license.
Now the long form:
The current state of the project is such that there's a file wav_test.cpp which grabs a bunch of internal components, mashes them together, and produces HRTF. I tackled that first since if I can't do all the things Libaudioverse didn't do to make this sound good and be fast, it's not worth continuing. The HRTF probably isn't done, but it's to the point of proven: there's some automated Python that runs the MIT Kemar dataset and produces some C files, and a bunch of C to run it.
Wav is implemented because it's the easiest, but code is in the repository to implement Flac, MP3, and Ogg. It's just not wired up as of yet. MP3 is now patent free, so we get that, and a bunch of people have written helper libraries that aren't libsndfile so we get all of these formats while maintaining the ability to statically link the library and producing only one DLL if you use it dynamically. The system to manage file I/O is extensible, and will be able to read from anything. HTTP streaming, custom DRM, and possibly even streaming realtime encoding from someone else's machine are all on the table, all be it not implemented.
I know how to do reverb, and when we get to that point it shouldn't take me long. A bunch of DSP components toward that exist. The initial version of that won't support reverb zones or anything, but there's a good algorithm in Libaudioverse that I can port over.
Unfortunately we don't currently have surround sound, and likely we also won't get it either. I'm no longer living in a big house with a home theater I can borrow. Once this is further along, if someone with the gear to test and the knowledge to write the algorithms properly offers to contribute I'm certainly open to making it happen, but I live in an apartment where my living room is also my kitchen and my office so even if I wanted to spend that kind of money I couldn't put it anywhere (however: if you buy me a house with space for this, I also promise to do surround sound).
The next task is taking what I have and turning into a C API and a Python package. This will take a couple weeks because i only have weekends and I need to abstract things just so. How that will look is something like this very undocumented and wildly incorrect pseudocode:
syz_createTable(&file_handle, "file", "bla.mp3", ""); syz_createSource(&source_handle, file_handle); syz_linkGenerator(ssource_handle, file_handle); syz_generatorStart(file_handle); // then in a loop or somewhere, you call the function to set a vector of f3 given a property... syz_setf3(source_handle, SYZ_SOURCE_POSITION, x, y, z);
Unlike Libaudioverse and OpenALSoft it will be possible to batch commands like a database transaction so that you don't get halfway done changing sources and then it decides to update. One of the places that a lot of libraries for this fall down (including OpenALSoft) is that all the commands grab a mutex or something and it's fast but only until you start calling into it all the time. That opens up a few things like streaming Synthizer commands over the network if you want but primarily it's for performance.
And that brings me to the last point that I think is worth bringing up: how fast is it? I don't know exactly yet because it's kind of unfair to benchmark it without having a proper file reader and stuff like that, but the above demo is live streaming a file from disk and applying HRTF in a debug build of C++, and (according to task manager and very unscientific measurements) is taking about a second to synthesize a minute of audio for 4 sources (you always synthesize at least 4 sources; if you only do 3 there's a silent fourth there). So in theory a debug build gets you into the hundreds. That won't be the case in practice because debug builds are horrible once abstracted, but I fully expect to see upwards of 1000 sources on a single core when this is done, possibly much more. I've run an OpenALSoft benchmark on my machine, it gets 1600 sources per core. So preliminary indications are that this competes with OpenALSoft even as-is, but we'll find out in the longer run if that lasts. For the curious, you can thank Clang's vector extensions for easy SIMD stuff.
For people who want to see the code, it's here.