2017-10-13 04:01:28 (edited by magurp244 2017-10-24 10:57:06)

I recently got around to finishing some sonifying examples available [here] demonstrating various filtering techniques i've been playing with, the first method allowing you to isolate and sonify specifically selected colors within an image.

The Second method applies to filtering objects in 3D environments, it captures images from both the frame and depth buffers, then isolates an object via its solid color from the frame image and uses it as a stencil to cut out a region of the depth buffer for sonifying. This was one of the idea's I'd had with Audiocraft for isolating specific resources or objects. For example, in minecraft there are over 200 different block types, but using this method you could isolate something like coal so when you sonify the area in front of you, only coal would appear, or you could set it to only sonify enemies, switches, or highlight other important objects within a 3D space.

The third method is a localized sonified mouse cursor, where it sonifies a small area around the mouse, representing its position in the soundscape. This particular method might be quite useful for 2D point and click adventure games using a depthmap as a backdrop in place of a conventional scene.

-BrushTone v1.3.3: Accessible Paint Tool
-AudiMesh3D v1.0.0: Accessible 3D Model Viewer

2017-10-24 10:40:14

Just thought i'd mention I found a solution for preventing unwanted pitch shifting when adjusting playback speed so it stays the same. As it turn's out I'd overlooked the min/max frequencies when generating the wave form as a way to compensate, as opposed to going through the trouble of 3rd party libraries, building a phaze vocoder, or using time shifting... I've re-uploaded the I2S filter examples with the changes. Some relevant code for the curious:

class sonifier(object):
    def __init__(self):
        ...
    #adjust frequencies to maintain pitch when adjusting speed
        self.max = int(5000*self.speed)
        self.min = int(500*self.speed)

        ...

    #Generate sine wave to multiply image data against
        self.w = np.asarray(range(self.y))
        self.w = self.TwoPi * self.min * np.power(1.0*self.max/self.min, 1.0*self.w/(self.x-1))
        self.w = np.tile(self.w,(self.ns,1))
    ...
-BrushTone v1.3.3: Accessible Paint Tool
-AudiMesh3D v1.0.0: Accessible 3D Model Viewer