Here is the upgrade path from 3.5 to 4.0/4.1
VMware KB: Upgrading ESX/ESXi 3.5 or 4.0 to ESX/ESXi 4.1
You can build a new server for vCenter and migrate the database over (so you don't lose any historical data) or just start from scratch... When I went from vCenter 2.5 to 4.1, we had to start over as our 2.5 database had become corrupted. You lose the historical performance data, permissions, folder, etc... but the VMs and the hosts stay intact.