2020-06-22 22:38:40

75 is right. However, I don't think that BGT was (necessarily) an apt lesson given that the encryption was poorly implemented/the way the executable image was generated made breaking the cryptographic primitives trivial. Such a method would not work under a PIC/PIE binary using ASLR (though the overall point is still valid).

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2020-06-22 23:48:03

@76
I could break it in a weekend even with ASLR on, but a quick Google for how to disable ASLR gives this.

My Blog
Twitter: @ajhicks1992

2020-06-23 00:06:53

@77, the fact that any OS in modern times would allow you to disable ASLR is troublesome. I'm of the opinion that OSes should force security features like ASLR to be enabled, without a way of turning it off.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2020-06-23 00:14:56

@78
Not to take my own thread majorly off topic or anything, but there's lots of good reasons to disable security.   Your hard stance on such issues is a little naive, if I'm being blunt.  "You can't disable security" only works in an academic setting, what you really want is "You can't disable security by accident, and without being authenticated", and that's not a problem here because of UAC.  As far as I know there's no control panel for it, and this is the sort of thing lots of developers need from time to time--for one thing, it's useful to security researchers, I'm sure.  If you want something much worse than ASLLR with respect to being able to toggle things you shouldn't, install the Windows 10 sdk and poke around in gflags.exe (and note that you can reimplement gflags yourself, because it's just writing the registry; and before you say that's Microsoft-unique, it's not, Linux has this kind of stuff too).

Also, even if it's not a registry key, you could just edit the PE header anyway.

My Blog
Twitter: @ajhicks1992

2020-06-23 02:12:35

@79, point taken.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2020-06-23 22:59:54

@63
I know the things that you are talking about, but only superficial; I don't take these courses in the university because I preffered the tradicional software way; You are going deeper into the most specific and math  area of audio.
And I imagine the efort that take you to know all these things.

smile

Againg, thanks for doing this lib :3

2020-06-25 18:58:14

I know the developer of the new environment suite is looking for a good sound API, I'm not sure what they're using right now but it's like BGT's sound system but a bit worse, it claims to load flac and then has weird truncation problems. Thing's written in pure basic. This is my first time opening the forum in months and my first time seeing this thread so forgive me for being out of date on where things are, don't wanna read all the posts smile. I noticed from the end it doesn't support in-memory streaming yet. That would be needed. HRTF would have to be able to be disabled, I don't think HRTF should be applied on a side scrolling map! And do we have reverb yet? The reverb in LAV, least the one I saw SBYW using, was actually not all that good, though maybe that was the BGT calls not initializing it to sound right. It just ended up sounding like a delay, wasn't smooth enough to sound like a reverb. Though I'm guessing the algorithms present in something like freeverb3 aren't compatible with your license?

----------
An anomaly in the matrix. An error in existence. A being who cannot get inside the goddamn box! A.K.A. Me.

2020-06-25 20:16:15

@82
Why is in-memory streaming relevant?  is it downloading over the internet or something?  You can stream from disk fine, and if you're downloading over the internet all the time and not caching on disk, you have much bigger problems than your audio library (also, a good way to truncate flac through any audio library is to mess up your in-memory buffers that you think are working).

Libaudioverse reverb is fine if it's configured right.  It's not amazing, but it's not necessarily choppy unless you set one of the properties low.  It's possible that I gave that bad defaults and never noticed, or that someone found a bug in it.  The bigger problem with it is that for a variety of reasons it's much slower than it ought to be.

Stereo panning is easy and will be here shortly but the bigger problem is that properties are literally 10 times slower than they need to be.  I plan to fix that this weekend, at the cost of making reading weird: you won't get the last value you set, just some value in the recent past.  I actually planned to do that particular change eventually anyway, but the benchmark is proving that it unfortunately needs to be the default.  Since reading is always going to be slow enough that you probably don't want to do like source.x += 3 all the time anyway (since no matter how you slice it locks have to be acquired and handles converted, etc) that's not a huge loss, and suffice it to say that there's good reason and avoid the essay-length explanation that I would have to give as to what that is.  I'll probably have to write the essay-length explanation anyway, I fully expect a bunch of pushback on this particular design decision, but eh.

Once I've done that and put stereo panning in this is usable, if usable means no reverb and you have wave files, and obviously there will be enhancements to fix those problems in short order behind that.

I have never been impressed by freeverb, but it might be compatible with the license.  It always sounded very metallic to me in everything that uses it.  I can do reverb though one way or another; Libaudioverse's reverb was 4-5 years ago, I understand much more now than I did then.  And if I don't there's sources to pull from.

My Blog
Twitter: @ajhicks1992

2020-06-25 21:11:38 (edited by Ethin 2020-06-25 21:13:28)

@83, I'm not saying this is feasible, but is there a reason you can't use atomics instead of locking algorithms? Or would that be over-optimization? I've always liked the idea of lockless programming, though I've rarely ever done it (I did it when initializing SMP recently in UEFI). I've read up on it though -- MS has a (supposedly) good one for Xbox360 and Windows, but I don't have many resources on it. If used properly though it sounds like it would make life easier (though I'm not really sure, heh). I'm too used to mutexes and reader-writer locks. (I actually heard of something interesting called a Seqlock, which is used in Linux, though I don't think anyone has truly mastered its implementation.)

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2020-06-25 21:16:46

Streaming in-memory gets relevant as soon as you want to implement some sort of encryption, because you'll probably have the encrypted file on disk and want to decrypt it in-memory first before feeding it into Synthizer, or you'd have to decrypt, write it back to disk and stream it from there, which sounds rather inefficient to me, or do you have any other idea how this would be to be realized with Synthizer if in-memory streaming isn't possible?
Best Regards.
Hijacker

2020-06-25 21:24:07

@84
atomics work only to some extent.  The problem is that you only get one atomic operation at a time.  So as soon as it involves any more than one variable, or if the operation might have side effects, or anything like that--well, too bad for you.  You lose.  There is more than a little bit of lockfree stuff in synthizer already but proving that it works is almost impossibly difficult.  ideally (and in the future this might be the case) you'd put some Synthizer properties in atomics and others in a queue of deferred writes that get applied at the next tick, but things like setting the buffer on a buffer generator also need to change position etc. so for example:

You use atomics for position and loop points and etc. You don't use atomics for the buffer.  The code sets the buffer, then position etc. but because buffer was in the queue it clears everything after the fact because total order is gone.

And for things like defaults for sources that are properties on context, this approach also means they can possibly be set out of order there too, depending on the strategy.

There's only so many strategies. OpenALSoft does some complex agglomeration of locks and atomics.  Synthizer is borrowing the WebAudio idea of using message passing.  The current approach is to have a lightweight concept of an Invokable and to use a lockfree queue and a semaphore to submit them, but that's too slow.  So I'm replacing it with a lockfree ringbuffer, separating validation from setting, and then they just apply at the next audio tick, or when manually flushed if the queue gets full.

ideally we'd then have some sort of per-thread cache value thing but the thing is, reading an int property or something like that is always going to be slow enough that you don't want to do it, and anything I do to make it more right just makes it slower.  Writing this, I'm realizing I could just flush pending writes before every read, so maybe it doesn't need to be eventually consistent, but in either case the choice I made has the consequence of making something less than ideal about the read path.

But the advantage is a pretty big one, in that there isn't really any risk of deadlocks, and in the long run it's possible to get the audio threads down to a finite, constant number of syscalls.  They already are, but memory allocation/deallocation happens on them still in a few cases, for the time being.  That's fine, it's good enough, but these are the kinds of choices that take good enough to the sorts of latencies you get out of Reaper when configured to its most extreme settings sometime in the future.

My Blog
Twitter: @ajhicks1992

2020-06-25 21:26:43

@85
This thread has already covered encryption. Scroll up some. I don't have the exact post numbers.  But in-memory streaming will happen.

But more to the point--to cover this yet again--it doesn't matter.  Someone can always break it.  It's valuable enough that I'm going to support the use case, but being as I can't offer unbreakable encryption (and you can't, either), it's not high priority and it'll happen when it happens.

My Blog
Twitter: @ajhicks1992

2020-06-25 21:38:24 (edited by Ethin 2020-06-25 21:40:08)

@86, aha, yeah, that makes sense. I'm trying to figure out where I can use atomics in my kernel right now lol (I use RW locks quite a bit, especially in the memory manager... thank statics for that).

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2020-06-25 21:58:48

@88
A good start is this, this, or this for unbounded queues.  And though I don't have a reference handy, hello world of atomics is a single-producer single-consumer ringbuffer where the producer's write index is an atomic.

You might also look into ConcurrencyKit.  This is what we use at work.  It's got some good stuff in it for SPMC sets and hashmaps, and a concurrent version of queue.h.  Also an implementation of SMR.  Unfortunately it's BSD and doesn't work on Windows I don't think.

But: you're a kernel, you don't have to switch into kernel mode to make a syscall, and atomics don't solve the signalling problem.  Mostly the benefit here will show up in memory allocators.  But overall it's important to note that lockfree algorithms can often be slower than just using a lock.  You're probably more interested in waitfree algorithms, but waitfree algorithms are very rare and very hard.  Synthizer would just be using locks, the reason I'm not isn't performance, it's because I want to avoid being in the kernel as much as possible.

My Blog
Twitter: @ajhicks1992

2020-06-25 22:30:21

@89, the problem with this is I don't need any of this yet (though I should probably either find a crate that implements it or implement the algorithms myself anyway) because my kernel has yet to support SMP. I'm definitely trying to get there though; currently trying to figure out ACPI (which is an absolute utter pain, even with my acpi helper crate that the rust-osdev community wrote). I think submitting this to some place like HN is probably a good idea about now... I still need to get file systems in. I can tolerate single-core computing with cooperative multitasking for as long as I need, but I'd eventually like to at least run basic/primitive programs. But to do that, I need file systems, syscalls, and a few other things first. But I've dragged this topic off the tracks; I'll probably open an issue on my repo for what still needs to get done. smile

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2020-07-05 04:07:01

Synthizer 0.2 is on Pypi.  We're now releasing through an automated release process, and you can now get non-Python artifacts through GitHub releases for released versions rather than digging through CI.

The repository has moved to synthizer/synthizer so that I could get GitHub pages how I wanted.

We have a manual.  The manual has a Python tutorial with a worked example, specifically a small media player.

As discussed already in this thread, buffers are in as well, so you don't have to stream audio.

Release notes for the release after this one will be in the manual, as opposed to random posts on this thread.

It's probably fragile. I'm aware of at least one bug that'll happen if you hammer on it from a bunch of threads at once.  We still only support wav. HRTF is still a bit un-HRTF.  The list of features that it needs is long.

But it's fair to say it's arrived, in the sense that we actually have a pretty sizable manual, and generally all the other things actual projects that actually do stuff are supposed to have.  You could build a game with it.  You won't get music (because we're missing DirectSource) and you won't get stereo panning (because we're missing other panner strategies).  but the info that you need is out there, I expect the C API to be mostly stable moving forward as long as you obey the warnings about property getters, and if it's broken then it's a bug and not because there was something I'm only halfway done working on.

I have a 4 day weekend.  There's still 2 days left.  We'll see if we can get MP3 and/or Flac, and maybe stereo panning.  But if not I expect 1-2 weeks from now to be able to say with a straight face that it's at the point where it's got everything that you absolutely have to have to do Swamp (which notably doesn't use reverb).

My Blog
Twitter: @ajhicks1992

2020-07-05 05:12:04

I like the manual thus far! Good work!

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2020-07-06 02:11:01

0.3.0 with Flac and MP3 is up.

My Blog
Twitter: @ajhicks1992

2020-07-08 12:39:56

I checked it out, and i must say, it sounds great so far. I can't wait for hrtf to be added.

best regards
never give up on what ever you are doing.

2020-07-08 15:16:57

@94
We have HRTF.  It's just not great HRTF because reasons.  We only have HRTF, you can't even turn it off at the moment.

I have to tweak the filter normalization to get more of the frequency-dependent effects to show up is all.  That's like a whole Saturday all, mind you, which is why it hasn't happened.  But it's there and it's got really good left/right (as in better than stereo), just not so good in front/behind and up/down.

My Blog
Twitter: @ajhicks1992

2020-07-08 16:25:43

yeah i heard that it is verry good at left and right. but back and forwards, not to good, but i do fully believe it can improove. keep up the good work please.

best regards
never give up on what ever you are doing.

2020-07-08 20:51:00

Here are some small bugs I found while playing around.
You probably already know about some of these.
The Buffer property in the manuals BufferGenerator section is accidentally labeled as SYZ_P_POSITION.
Buffer has no property length.
Python: Accessing buffer property of BufferGenerator before it is set causes a crash.
File test example seems to sometimes skip first second of a sound.

2020-07-08 20:56:30

@97
If you mean the C++ test, that's not really an example, more a random thing I use for debugging at this point, and the first thing it does is seek forward by a second. I will look into the others this weekend. They all sound like easy fixes.

I didn't give buffers length yet.  As explained in the manual reading properties isn't recommended save for debugging. It'll probably be a function for that reason.

My Blog
Twitter: @ajhicks1992

2020-07-08 21:04:37

Will you be working on making Synthizer compilable on GCC? Is that required in order for it to comile on Linux?

2020-07-08 21:15:00

@99
It compiles under Clang.  This means it probably compiles under recent GCC as well.  There will be Linux CI at some point, though getting it working and keeping it working will probably require someone interested who's on Linux to volunteer.  But getting Appveyor Linux CI going is on my to-do list and fortunately it builds under WSL, so that can be done pending me having the time.  Frankly it probably also works on Android with a little work and someone writing JNI bindings, and iOS as well.

My Blog
Twitter: @ajhicks1992