2020-12-21 00:47:31

synthizer 0.8 is finally live.  Undoes the rollback, makes property writes 5 times faster.  Object creation, generator manipulation, routing, and property reads are several orders of magnitude faster as well.  You can just read buffer positions now without penalty in other words.  Linux and MSVC support should both be back.  I'll be getting to the various miscellaneous issues people have outstanding next.

That said, two things.  First you really want to read the release notes.  There are some compatibility breakages.  The biggest is that you need to explicitly enable hrtf per source now, as it's off by default.  Second, this was a major internal refactor, and while it works I do expect issues.  I'm going to be off for like 10 days starting Christmas Eve, so I'm pushing it now in the hope that people will be able to file all those issues right away so I can deal with them promptly while I have the time.

@chrisnorman7
Would be useful if you could tell me whether your latency problem is gone.  Also, whatever your reverb crash was.

My Blog
Twitter: @ajhicks1992

2020-12-21 05:35:42

Even without HRTF Panner strategy, sound remains mono. Will it remain like that, or it will be eventually changed that stereo panning doesn't downmix the sound to mono?

2020-12-21 06:32:36 (edited by Ethin 2020-12-21 06:33:00)

@327, there's a reason for that. Trust me, stereo panning sounds horrible. You can do 3D and stereo panning with stereo sounds, but not downmixing them to stereo completely destroys the auditory illusion. Your not supposed to pan/use 3D/etc. with stereo audio or multi-channel audio in general.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2020-12-21 06:33:19

If you're trying to just do stereo music, grab DirectSource.  We've had that forever.

If you're trying to pan a stereo sound I might tackle that eventually, but I only know of a couple things that do it and the results tend to be surprising.

My Blog
Twitter: @ajhicks1992

2020-12-21 10:54:10 (edited by SkyLord 2020-12-21 10:54:39)

@328, i really don't understand why it destroys the auditory illusion. When sounds are hrtf and mono they sound nice, however when they are mono and not hrtf they sound meh. For me, having a sound without hrtf and in mono is just lowering the sound quality.
@329, i know about the direct source, but that's not what i need. Thanks for the answer anyway.

2020-12-21 16:55:15

@330
You will find that no one offers what you really want here.  Turn HRTF on and call it a day.  Synthizer can't because synthizer needs to be in a "safe" configuration if you will, something that works on all audio devices.  But nothing stops you saying "but I'm writing an audiogame, therefore it's always HRTF".

If Synthizer did offer what you want, you'd have to design sounds perfectly for it because if the stereo channels in the .wav or whatever are of different volumes, the panning algorithm can't know that and you end up with the sound panned weirdly.  You could design for it, but designing for it means making sure your stereo sound has no panning of its own, which I'm pretty sure is also not what you want.  I'm not adverse to playing with this eventually, but no one's going to use it for a long time if ever and it has the aforementioned weird constraints, so it's certainly not priority.

My Blog
Twitter: @ajhicks1992

2020-12-21 17:18:24

@331, i understand.

2020-12-22 01:39:54

@326
Good work mate.

I'll play with it properly tomorrow, but upon running the Earwax test suite:

earwax\mapping\box.py:601: in play_sound
    sound.generator.buffer.get_length_in_seconds() * 2
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   NotImplementedError: Can't read object properties

synthizer\synthizer.pyx:122: NotImplementedError
======================================================================= short test summary info =======================================================================
FAILED tests/box_test.py::test_play_sound - NotImplementedError: Can't read object properties

Is that expected behaviour?

Cheers,

-----
I have code on GitHub

2020-12-22 01:50:11

Yeah, it's in the release notes.  I had to yank the ability to read object properties because insert essay about how shared_ptr isn't a trivial layout type and something something something spinlocks that results in reading them possibly causing audio glitching.

Seriously, there are things man was not meant to know, and the subtleties of lockfree and waitfree programming for hard realtime requirements are worse than Lovecraft.

My Blog
Twitter: @ajhicks1992

2020-12-22 14:20:03

@334
Haha, that's fine mate.

Honestly, my use was suspect at best, and will be replaced with something better after today.

Thank you again for your ongoing hard work. It's really appreciated.

-----
I have code on GitHub

2021-01-04 16:01:19

Hi all, does synthizer support ogg formats in the latest version?

best regards
never give up on what ever you are doing.

2021-01-04 16:22:02

No, it doesn't, and it won't for a while because it turns out that stb_vorbis is broken at the moment and potentially very lame.  But it does support MP3, so just go use that instead.

Not sure why people are so attached to ogg to be honest.  MP3 is patent free these days.

My Blog
Twitter: @ajhicks1992

2021-01-04 16:40:37

No, i am not in to ogg, i was just asking, because i need to know if i need to convert the sounds or not, but thanx for the information

best regards
never give up on what ever you are doing.

2021-01-04 22:20:14

You can check the OGG support status on this issue ( https://github.com/synthizer/synthizer/issues/37 ), where I'm tracking it.
In summary, the OGG decoder is implemented, but at the moment though stb_vorbis has a bug that makes it unable to load some files. There have some pull requests on the upstream stb_vorbis repository but they haven't been merged yet so the pull request for the OGG support in Synthizer needs to wait until this is resolved.

2021-01-04 22:35:36

@339
Yeah, but I have reservations about that still frankly, being as there are outstanding fixes for it from months ago.  It is hard for me to be excited about taking a dependency on stb_vorbis but also we forked it when it's apparently easy to break and not getting quick maintenance.  The only reason it's still in the running at the moment is that I know BGT used it (strange as that is to say, but BGT's audio worked well enough for lots of people).

I didn't realize you were still hoping for this to be done in the near future, but I probably won't put it in 1.0.  There's enough instability in the library that I am really not thrilled by the idea of adding more until after Synthizer gets used in a larger project or something like that.  Again, we have MP3.  Maybe dr_mp3 will prove to be unstable or something but it hasn't yet, and I've gotten quick fixes from mackron before.  I may consider sponsoring him to finish dr_vorbis further along, we'll see.

My Blog
Twitter: @ajhicks1992

2021-01-05 18:12:34 (edited by chrisnorman7 2021-01-05 18:18:16)

So I was considering loading buffers with Python's ThreadPoolExecutor class.

Is that OK? Or are buffers not thread safe? I'm loading everything else in the main thread.

I've ran some tests, and I can load a large directory in 31 seconds or so, rather than 86. It slows my computer to a crawl, but it would be optional behaviour.

Thanks in advance.

-----
I have code on GitHub

2021-01-05 18:40:31

Synthizer's thread safety guarantee is that all calls into synthizer are threadsafe unless otherwise documented, so it's fine, yes.  The Python bindings don't have cases where a python call is more than one C function, so you should be good to have all sorts of fun.

You won't need that thread pool trick yourself long term as Synthizer is going to add it.  But that's a ways off, so you'll want to implement this right.  To do so, you need something called an LRU cache, and to compute the size of the cache as you add to it so that you can keep the decoded assets under an estimated target ram usage.  I would also suggest limiting the thread pool to max(cores/4, 1) which may seem surprising, except that most things for reporting cores will report double the cores the machine actually has due to hyperthreading.  Idk how far you want to go with this yourself, but the Synthizer plan was to allow hinting: you say "I think I'm going to need this asset in the near future", it starts decoding in the background, if you need it before it's done then it blocks on the decoding finishing.

My Blog
Twitter: @ajhicks1992

2021-01-05 19:29:49

@342
That's great, thank you. If Synthizer is going to cache for you, would it be possible to tell it how much RAM you'd like it to take up, either as a percentage, or in bytes?

That way you could let users with less RAM tell Synthizer to cache only say 100 MB worth of data, and users with 64 GB cache up to 16 GB. If the limit was hit, Synthizer could then remove the least-accessed (or oldest) asset.

When you say about this RLU cache, do you mean I should calculate file sizes? Or is there a way to get the size of a buffer in terms of Synthizer RAM usage?

-----
I have code on GitHub

2021-01-05 20:49:47

Synthizer will eventually let you tell it approximately how much memory to use.  For your reference it's channels*length_in_samples*2 for the approximate amount used by a buffer.

AN LRU cache is a least recently used cache, which can tell you which item was least recently used.  If you're going to cache buffers, you need an eviction strategy that can release buffers if/when the cache gets too full.  So it's something like track the bytes the cache uses and start dropping the least recently used entries off the back.  LRU caches are a common need and you should be able to just find a library for it.

Don't expect any of this in Synthizer before 1.0.  At the moment, unless it's a bug or very small, it's not happening unless it's in the GitHub milestone already.  But I will put it there as getting all of this right is subtle, and Synthizer internally has some more flexibility than I will ever expose in the public API. Plus performance, of course.

But out of curiosity, how big is your directory?  I am very surprised that you're taking 86 seconds single-threaded and 31 seconds multithreaded.  This doesn't exactly sound like a bug given that it works, but I would have expected that to be much faster so maybe I need to work on it.  Eventually there'll be benchmarks and stuff but that hasn't happened yet.

My Blog
Twitter: @ajhicks1992

2021-01-05 21:49:02

I encountered rather high decoding time as well, but I was under the impression that this is one of the downsides of MP3, AFAIK the decoders and encoders are really slow when comparing it to e.g. OGG. For example, when feeding a 6 minute long MP3 to Synthizer, I can already notice the lag until the sound is finally ready to play. I can go ahead and time it if you like, although i'm pretty sure that its easy to reproduce with just a longer MP3 file as well. Didn't try plain wav/flac though yet.

2021-01-05 22:14:01

yeah, that's probably not right.  I'll look into this.  I don't know specifically how MP3 compares to ogg, but it decoded fine on iPod nanos from 10 years ago, so it should probably be faster than that.  My bet is that I need to buffer I/O or something.

My Blog
Twitter: @ajhicks1992

2021-01-05 22:27:57

@344
Great info, thank you. I'll start work on it now.

Please don't worry about my directory. I was stress-testing, with the idea that conceivably, loading all the sounds for a given level might be time-costly.

The directory I was testing with is 1.3G of wave files, so intentionally stupidly large. Fasted I decoded it in was 78 seconds.

-----
I have code on GitHub

2021-01-05 23:17:48

@347
Yeah this is an almost-bug.  Wave files should be almost as fast as your disk can read data.  It should be like 10 seconds at most, single-threaded. I will find/make a test corpus and look into this once I bring other projects to rest.  I suspect there's going to be some easy win somewhere that makes this way better.

My Blog
Twitter: @ajhicks1992

2021-01-06 00:02:36

buffering is the key here. MP3 decodes fine if you're buffering in parallel to playing. When decoding the entire file before even returning to the main loop however, MP3 tends to be very slow compared to OGG or OPUS, not to speak about the lossless formats. You're currently decoding the entire file when loading it, thats why it really takes some time to load things up. Games like Manamon with about 250 to 300 MB raw sound files encoded in MP3 might actually take quite a while (i'd guess 15 to 20 seconds) to decode, especially if we're talking about hundreds of files instead of just 10 here. Thats what makes OGG IMHO better than MP3 for games, although the decoder speed might be kinda comparable, the encoder speed is just messed up in MP3, that thing takes ages, due to not being properly multithreaded at all.

2021-01-06 00:35:36

Buffered I/O is something else to do with making sure that the disk isn't asked to perform a bunch of small reads.  I will not be getting into partial decoding of buffers and then the rest magically appears because no one else does this and you really, truly can't do it reliably. I can identify at least 2 sure-fire wins and potentially 2 more wins that I can probably code in a Saturday, and 1 more potential win after that that will take a bit longer.  If it's still somehow too slow after all of that, I will revisit the situation.

My Blog
Twitter: @ajhicks1992