Is Possible if you have patience
If you don´t want to use an adaptoid or something like that you can go to this site
www.emulatronia.com, is in spanish but it can be traslated, there is a section "conecta tu pad al PC" it means: connect your pad to the pc, is a figure of a psx pad icon, click it and after that you can see a 4 pads one of themis nintendo 64 pad: you can find a good guide how make a circuit in a test-board, is a little bit tricky, and only support a controller, the conecction uses the I/O Port. Using this method the N64 pad does´nt support force feedback (rumble pak).
The same connection for the psx only uses a 5 zenner polarizator, and no need test-board and support force feedback.