2021-04-05 22:33:44

@425
Go ahead.  It'll be a long time until I get to dart.  Matching versions is nice, but I suspect the bindings will drift, so if that doesn't happen just document what version it's on.  I suspect that for anything but Python we're going to find out that bindings need incompatible changes apart from Synthizer itself.  Python won't because Python gets special status for weird reasons.

That said, two things.  First, Synthizer 0.9 is going to break the API.  No way around it.  I don't know the timeline, but call it sometime in the next 6 weeks tops.  Second, I would appreciate it if you didn't upload to package managers unless you're sure you're going to be around to maintain it.  I'd rather not end up with a bunch of official-looking packages that are n versions out of date.  That applies to everyone, not just you. It doesn't exactly do harm, but it is still kind of meh when you find something that looks like it does what you want but it hasn't been updated in a year and is missing all the features.  Don't get me wrong, not asking for signatures in blood, but do think about whether uploading now is going to provide a good developer experience and so on.

My Blog
Twitter: @camlorn38

2021-04-05 23:46:39

@426
I don't mind the breaking changes. That's partly why I love Dart so much: I'll know from the linter how bad everything is, so it's no worries.

I absolutely get you not wanting stuff on package managers. I don't imagine I'll stop using Dart or Synthizer any time soon, but I'll bear it in mind.

Also, when I use ffigen on Synthizer, I've just noticed it's complaining about syz_event:

Precompiling executable...
Precompiled ffigen:ffigen.
Running in Directory: 'C:\Users\chris\src\dart_synthizer'
Input Headers: [.\synthizer\include\synthizer.h, .\synthizer\include\synthizer_constants.h]
[WARNING]: Removed All Struct Members from syz_Event(syz_Event), struct member has an unsupported type.
Finished, Bindings generated in C:\Users\chris\src\dart_synthizer\lib\synthizer_bindings.dart

I've tested, and it's because of the union:

    union {
     struct syz_EventLooped looped;
     struct syz_EventFinished finished;
    } payload;

Is this something I need to take up with the ffigen package maintainer? Or is this a known FFI issue?

Thanks as usual.

Take care,
Chris Norman
Selling my soul to andertons.co.uk since 2012.

2021-04-06 00:15:51 (edited by Ethin 2021-04-06 00:19:47)

@427, its probably because unions have special semantics in C, and, I imagine, are kinda hard to represent across C boundaries. A union is only as large as its largest field, so you have to know the type of the data your accessing before you access the union, because to do otherwise is undefined behavior. If Dart doesn't support unions, there's your problem. (I can't seem to find anything about this via a quick search.)
To be honest, I'm not surprised dart doesn't support unions; unions are a fairly (old) way of dynamic typing, of sorts, and programming languages have evolved to hold much safer, and better, methodologies. Sadly, those methodologies are not yet transferable across the FFI boundary, so unions stay. (Rust, for example, only supports unions for C interop; all that unions provide is achievable with much better methods, like enumeration discriminants with fields.)

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

Thumbs up +1

2021-04-06 01:16:44

Yeah, I'm not sure what we can do about the union.  For now, you're actually okay because nothing yet uses that union, but you'll need to go upstream and talk to whoever it is.  I looked over the dart FFI out of curiosity, and it appears to be missing a lot of stuff I would have expected it to have.

Also on the note of calloc/malloc, one of the things I would have expected it to have is a way to put things on the stack.  But since it doesn't, note that you can and should pull the 64-bit integer handles out of the memory you allocated and store them inline.  Specifically, I suggest allocating the memory exactly once at startup for a syz_Handle, then reusing the location for every create call.  Malloc on really tiny objects like that can be incredibly inefficient, and it's just there to give Synthizer a place to write to so that the function can also return an error code (notice that none of the functions that use them take a pointer).

My Blog
Twitter: @camlorn38

2021-04-06 01:42:02

@chrisnorman7
Also, you're aware that there is a 99.99% chance that dart is going to support letting you list a package in a git repository as a dependency, instead of uploading to the package manager, right?  You shouldn't need to do a package manager upload to depend on it.  Every language with a package manager that I know of offers this feature.

My Blog
Twitter: @camlorn38

2021-04-06 07:42:11

@430
Yes, you can absolutely depend on a git repo, it's just minimally more effort. If and when you're ready to go live, I'd rather it were on pub.dev, whether that's under your name or mine (yours makes more sense).

@429
So you're suggesting holding a single "out" pointer as a global somewhere? Isn't that dangerous because you can no longer rely on what data that pointer is pointing to?

I've no idea about the Dart FFI's completeness honestly; it's not long come out of beta, and I wouldn't know a decent FFI implementation if it started selling fortune cookies.

@428
OK, thanks for the info mate.

Excitingly, I'm *nearly* at the stage where I can play sounds. Just gotta make the generators, and we're laughing.

Take care,
Chris Norman
Selling my soul to andertons.co.uk since 2012.

2021-04-06 08:12:59

@431, No, you can hold global pointers, but I'd (strongly) encourage you to use locking of some kind if your going to do that. Or hold a lock along with a vector of pointers if you want to keep track of that kind of thing. Point being that you'll need locking of some kind, especially if you believe that this will be working in an environment where multiprocessing or multithreading will be used. (Generally if yoru going to be doing anything global, I always think you should use atomics or locks or something like that, even if your only running on a single thread, because it means less work in the future when your ready to transition.)

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

Thumbs up +1

2021-04-06 09:09:23

@432
Thanks for the info mate, unfortunately I have no idea what half of your post even meant. Things are working as are, and I don't know enough to make them better.

Think I'm going to just have to learn on the job, until somebody who knows what they're actually doing fancies having a bash at the bindings. smile

Right now I'm just creating handles as and when needed (although obviously I'm sotring object handles on the objects themselves). If this isn't the best way, I'll have to revaluate later on if and when things start to break.

Thanks again.

Take care,
Chris Norman
Selling my soul to andertons.co.uk since 2012.

2021-04-06 15:51:55

What you actually want is to put a malloced pointer behind a thread local that's never freed for the lifetime of your program, have it hold handles when syz_createWhatever writes them, and then pull them out immediately after.  This is, of course, predicated on dart having thread-local storage.  It's not a big deal considering that object creation is expensive, but smashing malloc is nonetheless not so ideal especially for something that shouldn't even require it in the first place.

There are workarounds for unions.  They seem quite involved.  But it looks like Dart hasn't done anything for them since this issue in 2019.  I will have to see if other languages have this problem.  It's kind of hard to swallow "you can never use unions" in a library that needs to be as efficient with space as it can be, but if it's going to prevent more than just Dart then something will probably have to be done.  Suffice it to say that in general Dart's FFI implementation seems quite immature.

My Blog
Twitter: @camlorn38

2021-04-06 16:30:08

@434
OK, you keep talking about malloc. What's the difference between that and the calloc I've been using? I've read this page, but I'm still unclear on the details.

I guess what I'm asking is which should I use all of the time so I can stop worrying about the details? If that is even answerable in this context.

I don't know about thread locals, Dart talks about isolates. I'm not planning to use them myself, so I'll not be worrying about it unless it becomes a problem.

Take care,
Chris Norman
Selling my soul to andertons.co.uk since 2012.

2021-04-06 16:54:48

Malloc and calloc are the same, but calloc zeros the memory and malloc leaves it to be whatever random garbage happened to be there or whatever.  There's no real difference besides that; for Synthizer, either is fine because Synthizer is writing the location on success.  The reason to use calloc is that calloc doesn't actually use a for loop or something to get zeros there, but instead relies on OS and CPU magic.

The problem is that you can't actually malloc 8 bytes.  You can malloc 8 bytes sometimes on 32-bit Windows, but everywhere else is 16 minimum (passing 8 will silently round up) and usually you actually are going to get a few kilobytes.  Where this lands basically from your perspective depends on the phase of the moon, and weirdly on Liux it ends up not mattering.  It's also anywhere from instantaneous to incredibly slow, again depending on the OS and the phase of the moon from your perspective.  I'd have to start by reading Dart VM source code to find out what it does, so it's also a bit phase of the moon from my perspective too, honestly.  But it can be really bad.

It's fine for now, as I said.  But it's less than ideal and if you put that sort of implementation in front of someone trying for a graphical 60 fps on a phone or something small like that, they're probably going to be displeased with you.

My Blog
Twitter: @camlorn38

2021-04-06 17:21:20

calloc has the other difference that it can allocate arrays a bit easier. I.e.: calloc(size, array_dimensions); if I'm not mistaken, will allocate an array as though you'd done array_type array[len][len]...

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

Thumbs up

2021-04-06 17:37:58

I don't think that difference exists in Dart, only in C.

My Blog
Twitter: @camlorn38

2021-04-09 09:23:08 (edited by bhanuponguru 2021-04-09 09:31:52)

hey all. so i upgraded to python3.8.5
now i am building synthizer to make sure that there is no issue in building in python 3.8 using cython. but when i say python setup.py install, it says cython3 does not support cdef variables. it's kind of warning. and after that i get weerd errors. here is the entire prompt
d:\my projects\synthizer\bindings\python>set BUILDING_SYNTHIZER=1

d:\my projects\synthizer\bindings\python>python setup.py install
Building Synthizer from repository. Adding additional directories.
Repository root is d:\my projects\synthizer
Using build dir d:\my projects\synthizer\build and library synthizer
Compiling synthizer/synthizer.pyx because it changed.
[1/1] Cythonizing synthizer/synthizer.pyx
warning: synthizer\synthizer.pyx:260:10: cpdef variables will not be supported in Cython 3; currently they are no different from cdef variables
warning: synthizer\synthizer.pyx:261:10: cpdef variables will not be supported in Cython 3; currently they are no different from cdef variables
running install
running bdist_egg
running egg_info
creating synthizer.egg-info
writing synthizer.egg-info\PKG-INFO
writing dependency_links to synthizer.egg-info\dependency_links.txt
writing top-level names to synthizer.egg-info\top_level.txt
writing manifest file 'synthizer.egg-info\SOURCES.txt'
reading manifest file 'synthizer.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'synthizer.egg-info\SOURCES.txt'
installing library code to build\bdist.win32\egg
running install_lib
running build_ext
building 'synthizer' extension
creating build
creating build\temp.win32-3.8
creating build\temp.win32-3.8\Release
creating build\temp.win32-3.8\Release\synthizer
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x86\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD "-Id:\my projects\synthizer\include" -IC:\Users\bhanu\AppData\Local\Programs\Python\Python38\include -IC:\Users\bhanu\AppData\Local\Programs\Python\Python38\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt" /EHsc /Tpsynthizer/synthizer.cpp /Fobuild\temp.win32-3.8\Release\synthizer/synthizer.obj
synthizer.cpp
creating d:\my projects\synthizer\bindings\python\build\lib.win32-3.8
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX86\x86\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO "/LIBPATH:d:\my projects\synthizer\build" /LIBPATH:C:\Users\bhanu\AppData\Local\Programs\Python\Python38\libs /LIBPATH:C:\Users\bhanu\AppData\Local\Programs\Python\Python38\PCbuild\win32 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\lib\x86" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\lib\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x86" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\ATLMFC\lib\x86" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\lib\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x86" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x86" synthizer.lib /EXPORT:PyInit_synthizer build\temp.win32-3.8\Release\synthizer/synthizer.obj /OUT:build\lib.win32-3.8\synthizer.cp38-win_amd64.pyd /IMPLIB:build\temp.win32-3.8\Release\synthizer\synthizer.cp38-win_amd64.lib
LINK : fatal error LNK1181: cannot open input file 'synthizer.lib'
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.28.29910\\bin\\HostX86\\x86\\link.exe' failed with exit status 1181

d:\my projects\synthizer\bindings\python>

so, is cython strage for every one? or just me?

if you like this post, please thum it up!
if you like to see my projects, feel free to check out my github
if you want to contact me, you can do here by skype. or you can follow me on twitter: @bhanuponguru
discord: bhanu#7882

Thumbs up

2021-04-09 13:47:21

This issue is obviously not related to cython, but to you not having the synthizer build results (dll and lib) in the appropriate places for the setup script to find it when linking.

Thumbs up

2021-04-09 16:44:57

Yes.  Also, please just use the wheel?  A large part of me wants to start responding to any Windows build question with "why aren't you using the wheel".  We have a CI pipeline.   I promise that if it breaks I won't put out the release, because it is literally impossible for me to do so unless it builds on all supported platforms.

My Blog
Twitter: @camlorn38

2021-04-09 16:49:18

i am just testing my environment of python3.8. since i have to install every thing after ersion change, i just tryed to build it just for testing my python environment. weather is it building or not. but nvm, i think i don't follow building gude properly. because my other c++ modules are building just fine

if you like this post, please thum it up!
if you like to see my projects, feel free to check out my github
if you want to contact me, you can do here by skype. or you can follow me on twitter: @bhanuponguru
discord: bhanu#7882

Thumbs up

2021-04-09 17:24:35

If you install the version on pip, it should install fine even from source.  If you go around grabbing versions of stuff from git repositories on GitHub you will have these problems, and you shouldn't do that to test your environment.

My Blog
Twitter: @camlorn38

2021-04-09 17:59:45

oh got it thanks camlorn

if you like this post, please thum it up!
if you like to see my projects, feel free to check out my github
if you want to contact me, you can do here by skype. or you can follow me on twitter: @bhanuponguru
discord: bhanu#7882

Thumbs up

2021-04-12 06:01:22

On the topic of occlusion, is it recommended to adjust the sources for individual sounds? Would it be more efficient to create a separate context with the settings already applied?

Trying to free my mind before the end of the world.

Thumbs up

2021-04-12 09:06:06

@445
From what Camlorn has told us before, you should probably be sticking to 1 context, unless you've got really good reason not to.

You can absolutely adjust the settings per source as often as you like.

Take care,
Chris Norman
Selling my soul to andertons.co.uk since 2012.

2021-04-12 15:46:37

Yeah, 2 contexts is expensive.  Eventually you'll get control groups or something so that you can "group" objects but that's a post-1.0 thing since writing that is super easy without my help.

The filters are basically free.  I think the only thing that's not basically free is reverb but you should still be able to run like 10 of them without an issue.  Keep in mind that whether or not a source is occluded will change at runtime and you can't migrate them between contexts, in any case.  If something here turns out to be surprisingly expensive, open an issue.

My Blog
Twitter: @camlorn38

2021-04-21 16:32:21

Hi there. Does synthizer work on linux and Mac as of yet?

best regards
never give up on what ever you are doing.

2021-04-21 16:42:17

@448
I believe it works on Linux, but not mac.

That said, I couldn't get it to build on a Raspberry Pi, so maybe Camlorn had more problems with linux.

Take care,
Chris Norman
Selling my soul to andertons.co.uk since 2012.

2021-04-21 16:43:22

Linux, yes.  Mac, no, but I've got a Mac Mini showing up sometime around the 30th and that should land in 0.9.  No promises when 0.9 is happening, but it'll be in 0.9 when it does.

My Blog
Twitter: @camlorn38