What I'm doing is using USB over CAT5 adapters (keyboard/mouse don't need to be USB 2.0, so you can get away with the cheaper adapters). So, for example, I pass my video over CAT6 using a HDMI-->CAT6 adapter, and then I've run another line of CAT6 that I pass USB over and terminate it in a powered USB hub. Off that hub, I can run the mouse, keyboard, USB drives, and whatever else I need. At the moment, I have 4 VMs running like this: my son's computer (full on gaming, and I also run a second monitor dropping displayport to VGA and then passing the VGA over CAT6), my wife's computer (general use and light gaming), and two XBMC VMs. It takes a minimum of 2 nodes to do this, as you need a PCI-e slot, obviously, for each video card.
Currently, I'm running 5.0 as there are some proven issues with passthrough with 5.1. As a test, I made a clone of my USB stick for a node and then upgraded that node to 5.1, and the VMs using passthrough on that node no longer worked. Just plugged in the clone of the USB stick and went back to normal.
This month has seen some fairly large changes to the lab (just added a homemade standalone 32TB SAN for the lab, for example), which I'll be updating the site soon (been SO busy), but this basic setup has remained the same. I'm sure there are other ways to do it, but I've found this works wonderfully for my needs.