One of the most common questions we receive regarding VMware vSphere environments (and virtualization environments in general) is how to set up virtual networking.
While a misconfigured environment can still work properly, issues still arise with scalability, security, and ease of configuration. (This applies to almost all hypervisor solutions, but this post focuses on VMware vSphere environments.) Here’s my two cents on designing a virtual network using VMware vSphere.
The core choice for how to configure your environment revolves around virtual LAN (VLAN) tagging.
As a quick refresher, VLANs can take a single switch or group of switches and divide them into subgroups that are in completely separate broadcast domains. Each port on a switch can be configured as either an access port with a single VLAN ID (where the device connected to it doesn’t require any configuration), or a trunk port with one or multiple VLAN IDs (relying on the connected device to choose what VLAN ID a packet will traverse over).
The three possible methods of configuring vSphere networking
- External Switch Tagging (EST)
- Virtual Switch Tagging (VST)
- Virtual Guest Tagging (VGT)
Each method moves the stage at which a packet has its VLAN tagged or untagged.
External Switch Tagging (EST)
With EST, the vmnics (physical NICs in the ESXi host) are configured as access ports on the switch. As packets enter the switch from the host NIC, they are tagged with the appropriate VLAN and sent to their appropriate destination for that VLAN (a gateway, for example). As traffic flows towards the host, packets are untagged by the switch and presented as standard untagged packets for the host.
The idea behind this setup is that each NIC (or pair of NICs, for redundancy) is assigned its own specific VLAN. EST is the simplest to configure because it only uses access ports on the physical switch and no configuration is needed on the hosts (besides matching vmnics numbers to physical ports on the host and switch). The primary problem with EST, however, is its scalability. The more VLANs you require, the more physical NICs you need on each host. The main drawback to this approach is scalability. Switchports are expensive, and you can only add so many NICs into a given host.
Virtual Switch Tagging (VST)
VST solves this limitation by moving the tagging and untagging of traffic down to the ESXi host’s vSwitch. On the physical switch, ports are configured as trunk ports containing all VLANs that need to be accessed from any VM on the host or cluster. On the vSwitch, port groups are created for every VLAN that is required. For example, if there were VMs needing access on VLANs 5, 8, and 22, then three port groups will be created on the same vSwitch. Each one would specify a specific VLAN. In this setup, all tagged traffic destined for the ESXi host will remain tagged by the physical switch. This tagged traffic will then be untagged by the ESXi host as they proceed down to the VM itself. By having the vSwitch handle the untagging of traffic, individual VMs do not need to be aware of what VLAN is it on, and therefore the VMs just work without any additional configuration out of the box.
VST requires more configuration on the physical switch and on the ESXi host to be sure that all vmnic ports on the same vSwitch are required with the exact same configuration. If these configurations aren’t inline, intermittent packet loss could occur. This packet loss is the result of traffic sometimes leaving one (properly configured) vmnic or a different (improperly configured) vmnic, depending on how the switch is set up. When properly configured, however, VST can make an environment incredibly flexible. VST also gives admins a clear picture of how traffic is flowing between the physical and virtual network. Unlike EST, adding port groups (and thus, VLANs) is incredibly easy and scalable. Distributed switches can support 10,000 port groups, which is well over the capacity that the majority of environments will ever use.
VST topologies are by far the most common when deploying most VMware environments. The reason for this is because they strike a good balance between scalability (no reliance on physical switchports and NICs to expand the number of VLANs in a network) and ease of use (no need to configure every VM to specify what VLAN it is a part of).
Virtual Guest Tagging (VGT)
VGT sends tagged traffic all the way down to the guest VM. In this instance, the physical switch is configured as a trunk port, similar to VST, and the vSwitch port groups are configured for the special VLAN ID 4095. In order for traffic to pass end-to-end, the guest OS needs to be able to support 802.1q VLAN tags and should be configured for the specific VLAN that the guest needs to communicate on.
VGT is best when used in Lab environments, where perhaps a server administrator needs to be able to change VLANs on a guest without the hassle of changing it in vSphere. VGT can also be used if there is not a dedicated or knowledgeable VMware administrator on staff, and moving between VLANs is easier in-guest. Still, VGT can create an issue when it comes to vision into the network. It can become difficult to determine which VMs are associated with which VLANs, and even poses security concerns, as a VM can be set to a different VLAN without the virtualization administrator knowing about it.
SCTG’s approach
At SCTG, we standardize on VST deployments on all internal cloud, Enterprise Cloud, and Private Cloud deployments. This gives us the flexibility and scalability we need to provide customers with a vast array of options for their deployments. Choosing VST over EST not only increases the speed in which we can deploy new VLANs, it keeps switchport costs down, allowing us to pass those savings on to customers. While VGT would give us similar speed and cost savings, it creates too much in-guest customization, has the potential to be not as secure, and makes it very hard to perform root cause analyses should they be needed.
