I reccomend 128 to 256MB ram. For example, trying to load Zelda: OoT or MoM on 32mb ram isn't going to happen, or, trying to load RE2 on 64mb ram isn't gonna happen.
1. Go to the carpet where do yo have PJ64 installed.
2. You'll se the file Project64.exe, right click on it and make click on properties
3. On Compatibility, that option is visible
The compability mode doesn't really matter. It doesn't run better, and it doesn't run any worse. However, if this problem does occour, the solution is to use the compability mode.