Was meaning to comment a while back but didn't get the chance.
Firstly, there is a document which some of us have posted in, which contains the builds, mostly the ones that work. The crucial part is MB + CPU and pretty much all ATI cards work, although if you want to be successful and starting with a new build and don't want to experiment, you should definitely use the build that appears the most in that document.
Second, having the hardware decided on, either by sticking to what you have or getting something based on above mentioned doc, you need to get the software running. As some have already mentioned, 5.1 has broken USB passthrough (at least). So either use 5.0.x or 5.5 (I haven't tried this one yet). The easy way to experiment is to backup your VMs and use a USB stick to test the various esx versions/patches.
At the same time, you need to take into account some restrictions, the pciHole is one, and sometiems setting pciPassthru(x=0..n).msiEnabled = false helps as well (not just for GPU but other cards that you passthrough and error when installing drivers (I had to do this for a pcie usb card (or was it a sata card?).and my main passed through GPU card.
good luck with it.
PS: I'm busy lately trying to jump start a new project, protecting/controlling private/personal information through the use of virtual profiles.If anybody is interested, please check it out and follow the project on twitter, FB, WP, etc. Any comments/suggestions welcome on those sites so we don't pollute this topic Image may be NSFW.
Clik here to view.
(project home page: http://www.virtualprofile.org )
Sorry for the small advertisement, it's hard to get something new off the ground with no funds :|
