In my opinion the most flexible controller is the Xbox 360 USB controller. Not only is it of higher quality than any gamepad you'll buy in a store, but it's buttons can work well for any console you want really. For the N64 I generally use the A and X buttons as AB, because that's how they're laid out on the N64 controller, then I use the right analog stick for C Buttons (it's really not bad at all) right trigger as R, left trigger as Z, and LB (left shoulder) as L, since L is useless in most situations. However like P_025, I use different configurations for different games because sometimes certain layouts can be really annoying.
However I really think that the N64 controller is one of the best controllers there has ever been, and if you have one of those it will kick the shit out of your other gamepads, at least for N64 games. I just don't want to shell out for an adapter.