Beware of L2 changes when using bonding driver and mode 6 (ALB)
I would preface this with the fact that I am not able (do not want) to reproduce the issue, but after discussions with our very experienced network guy (20 yrs) and another senior analyst, this is what we came up with. Further, this could just be due to an old version of OpenE, or issues with our old switch stack. If you would like to discuss otherwise, I am happy to hear from the experts. TL;DR: If you have a 'whitebox' SAN that uses the Linux bonding driver in ALB mode, DO NOT make any kind of L2 changes on the SAN switch without shutting everything down first. Also, patch your ESXi like ASAP. We have an old legacy SAN (OpenE), it has two NICs that each connect to our old Netgear switch stack (one per switch). Up until now, no issues with the unit. Earlier this week I was adding a LAG to the switch stack as part of our new SAN deployment - the LAG did not utilize any active ports, and for a few hours seemed to be functioning just fine. Then I get a...