Hello. I am trying to add realtime voicechat feature to my game. My code works when only two people voicechat at the same time, but once the third comes in, the audio gets choppy (not laggy, so there is no latency, but the audio is choppy). Its not do to network latency or something, since I tried it in localhost, i opened two clients at the same time and turned on voicechat in both, and same happened. Also, Its an error in the client, not server. Because lets say 3 people voicechatting, but they are in different maps so they can't hear each other, the choppines doesn't occur. So it happens when one client receives audio data from multiple players at the same time. When a client receives multiple audio data from different players, only the audio in that client gets choppy. How can I fix this issue?
Here's a bit more info so you can learn more about my setup.
I use opus as the encoder
The frame size is 960 samples, so 20 ms, and I use 48000 sample rate.
None of my operations are blocking, E.G the issue is not because the playback of one audio stream blocks the other, they play at the same time. My network operations are also non-blocking, I use enet, which uses nonblocking system builtin. Only blocking operation is the opus decoding and encoding phase, but this is required, because the audio has to be first decoded and then played, so the opus blocks the thread when decoding, but when it starts playing, its not blocked.
I also buffer audio for each player, so I don't play the data right when it comes in, each player has an list which stores audio temporarily, when its length reaches a certain amount, I play the audio, and clear the list.
And the way I make them play at the same time is that each player in the game has an openal buffer and source, when the server broadcasts the data to clients, it includes the players name and the audio data in the packet, so clients can know which player the audio is coming from. Clients then put the data in to the players audio buffer (not openal buffer), when the audio buffers size reaches a certain limit, I clear the audio buffer, copy audio buffer to openal buffer, then rewind the players openal source and play it, so it plays the new data without blocking, and it can play more than 1 player's data, because each logged in player has an seperate openal buffer and source in the client.
I was using pyaudio Before for playback also, but switched to openal because pyaudio's stream read and write functions were blocking, and callbacks were the only way to make them not blocking, but since they were overly complex, I switched to openal. Since openal is low-level library, it gives me more control on how I will play the audio. But I still use pyaudio for recording, only I use openal for playback.
I was using opuslib as opus encoder before , and since it was old, I thought maybe that's the issue, and switched to pyogg library, which is another opus wrapper for python, but that also didn't fix it.
Any help would be appreciated. Thanks!
Regards...
Bilal
