This is a common frame buffer problem. Perfect emulation of N64 frame buffers is still yet to be accomplished. Some graphics plugins have options that make the graphics appear normal, but have very slow framerates as a side effect. Project64 1.6, when running with Jabo's most recent plugin appears to have a partial fix for this (not sure if it's the emulator, the plugin or both). It's still not perfect, but you can see most of the picture and it runs at normal speed.