When configuring a LAG in vSphere on a distributed switch, the default load balancing method is Source and Destination IP, Port, VLAN.
The upstream physical switch will also be configured with a load balancing method, and the LAG in vSphere needs to be configured to use the same method.
VMware vSphere ESXi
If the upstream physical switch is configured with a different load balancing algorithm than the LAG in vSphere, it is expected that traffic will not be distributed correctly across the uplinks at any given time.
Reconfigure the LAG in vSphere to use the same load balancing method as is configured on the upstream physical switch.