New audio plugin (libao) ?

[TD-Linux]

I've b
een messing with the SDL audio plugin in order to make it less fail. Unfortunately, it's harder than it might seem. Here's why:

• The Nintendo 64 code builds chunks of audio, then sends the chunk out out to the hardware when finished (which then calls a function in the audio plugin)
• SDL maintains two buffers of audio (let's call each buffer a chunk). It plays from one chunk at a time. When the chunk empties, it switches to the other chunk and calls a function in the program (plugin) to fill the empty chunk.

Do you see the problem? The problem is, both the emulator and SDL are calling the plugin when they have or want audio! This doesn't happen at the same time, so we can't just take audio from the emulator and drop it right into SDL.

The plugin's solution is a big buffer. Each time the Nintendo 64 sends out audio, it appends it to the buffer. Each time SDL wants audio, some audio is pulled out of the buffer.

The problem with this approach is both SDL and the emulator want fairly large chunks - that means our buffer should be able to hold at least a couple of these to prevent dropouts. Unfortunately, buffer sizes add up fast, and when you are buffering too much audio you have a major problem: lag.

Take for example the default jttl_audio settings (I think these are default, anyway).

The main buffer is set to hover between 16384 and 32768 bytes (on my machine, I bump the upper limit to 65536 to avoid problems). Each buffer SDL buffer is 4096 bytes. Let's assume best case scenario of one SDL buffer being empty, and a sample rate of 48khz. Then the lag can be calculated as ((32768+4096)/(bytes per sample: 4))/48000 (sampling frequency) = 0.192 seconds. This might not seem like a whole lot, but it is... 1/5 of a second already will cause lip sync issues. In addition, the N64 itself adds a bit of lag, as well as sound output. In reality, I estimate the situation on my own computer to be much worse, with about 0.5 s lag.

The obvious solution I see here is to write an audio plugin that uses a blocking-call based output, like libao. Here, when the N64 calls our function with audio, we just pass it right into the audio output function (which hopefully uses the audio driver / sound card's buffer, therefore bypassing both the plugin's buffer and SDL's buffer). In addition, most of these libraries have a blocking call, which should keep our program from spitting out audio too fast. I'm not totally sure if libao's functions are blocking or not.



Any thoughts? I'm very tempted to write this plugin but I want to make sure my efforts are not a waste of time.

[Pyromanik]

Unless your computer is reaaaaaly slow, you shouldn't really have to worry about blocking issues... I know mine empties the hardware buffer faster than the N64 can fill it.

Resampling is a better answer, and JttL already does that.

With writing directly to the hardware buffer you have no control over what happens if an access fails. With blocking mode (to my knowledge) either the sound data would be dumped leaving the user with a skip in the audio, or the whole program would slow until the sound card catches up and allows the data into it's buffer.

This is why there is also a software buffer.
This also helps when a hardware buffer is too small for one 'chunk' to be written to it in whole (my onboard sound for example gives me all kinds of hell).

From what I understand, blocking mode is more to ensure that an audio stream does not overwrite itself in the ring buffer (for example when playing from a file), not to regulate generated audio that comes in on the fly.




The technique you're referring to there with 'chunks' is double buffering, and is very doubtful the source of your problem.

[Azimer]

I've b
The obvious solution I see here is to write an audio plugin that uses a blocking-call based output, like libao. Here, when the N64 calls our function with audio, we just pass it right into the audio output function (which hopefully uses the audio driver / sound card's buffer, therefore bypassing both the plugin's buffer and SDL's buffer). In addition, most of these libraries have a blocking call, which should keep our program from spitting out audio too fast. I'm not totally sure if libao's functions are blocking or not.

Any thoughts? I'm very tempted to write this plugin but I want to make sure my efforts are not a waste of time.

You are wasting you time. I know this thread is two months old but I have just come acrossed it.

[vinipsmaker]

The libao is cross-plataform too and have modules for pulseaudio, oss, alsa, esd and others. I think this can help the plugin use the same code for more time. Another thing: The pandora (openpandora.org) folk plan use the Mupen64plus as base for their N64 emulator, and if this can help the speed, can help the pandora folk. The pandora will be the most powerfull game handheld device console, but yet will be very less powerful than a PC.