It depends what version of Project64 you're running.
If you're using anything before 1.7, you should turn on audio HLE through the RSP plugin configuration (Options :: RSP Settings... or something to that effect). "Send alists to audio plugin" or whatever.
If you're using Project64 2.1, just check the box for audio HLE in the main options settings.
Either way, you'll need to install an HLE audio plugin. Currently that means Azimer's audio plugin. Check the Apollo section of this board for the latest download. Make sure you switch plugin to that for audio.
As a fallback alternative to the above, if you want more accuracy and less speed (than HLE method), there's another RSP emulator I wrote that should fix the issue with Project64's default sound plugin, if you don't want to change any of those settings I just mentioned. Just replace $Project64/Plugin/rsp.dll (or Plugin/RSP/RSP 1.7.0.9.dll if that exists?) with
this RSP emulator here to use that instead of the faulted interpreter that Project64 ships with for RSP.