How did you identify that the packets hitting L2 switch, is it a guess because the file transfer takes more time to complete?
When the VMs are on same host and same port group, the packets will be transferred at the kernel level, it wont hit the physical layer (vmnic), so NIC speed doesnt matter here.
What is the speed you are getting, are you using vmxnet3 or E1000 for VMs ?
if the VMs are windows , you can tweak the network card settings inside guest to improve the IO speed.