There are several parts about the X64 version that made me go back to the standard XP Pro VLE:
1: As was said, driver support wasn't the best. I couldn't find any for my TV Card for the life of me, couldn't update most of my motherboard drivers due to most drivers only working with XP Pro.
2: Had some issues overall with a couple of games I frequently play. I couldn't get Final Fantasy VII PC working correctly with high res stuff from the Qhimm website, for example.
I have a 64 bit processor and all, but I really didn't see any sort of improvement over the standard XP Pro VLE edition. A new background, a optimazation or two, but nothing that would prompt me to want to stay with the "upgraded" OS. I'll wait for progress on Vista before I upgrade (because right now it's buggy as hell, haha).