There are o lot of debates about using LAG (MLAG) vs No LAG for Compute, Management and Edge VMware nodes. Short answer to this dilemma is – It depends:) But in this article I want to make a bit clarification for each solution and make pros and cons for them. The explanation will be done in VCDX way.
Using LAG/MLAG
Decision:
- Use channel aggregation on ToR switches in order to connect your Compute, Management and Edge nodes
Justification:
- Low bandwidth ISL. Traffic that enters downlink on the switch will not traverse (expect servers that are connected using one link on one switch) ISL ad will exit on the same switch.Advanced mechanism to control link status on Layer 2 using LACP.
- LACP control plane has on board special BPDU packet (fast/slow) in order to exchange information between aggregation devices and also to control status between peers. This control packets can help detect problems on Layer 1/2 and also problems on switch itself (e.g physical link is up, but switch cannot process packet due to software or hardware error)
- Higher port capacity utilization and ability to use different load balancing hashing algorithms
Impact:
- Higher complicated design, configuration and operational procedures
Using No LAG
Decision:
- Use No MLAG/LAG on ToR switches in order to connect your Compute, Management and Edge nodes
Justification:
- Ability to physically separate different types of traffic – Data, Storage, vMotion and control on ESXi level load balancing for each traffic – Active/Active, Active/Standby
- Simplified design and independence from network infrastructure/hardware. No need to additionally config network equipment and also ability to use different vendors i ToRs pair.
- Operational overhead due to necessity of support different MLAG implementation. Each vendor has its own MLAG technology (VPC, MCLAG, MLAG, VLAG)
Impact:
- High bandwidth ISL utilization, more port needed in order to satisfy capacity requirements.