Software-defined technology has changed the way that we looking at everything from an enterprise infrastructure standpoint. In thinking about software-defined storage and the way it handles failures, there are new concepts and ways of thinking about the storage infrastructure. VMware vSAN is a powerful software-defined storage solution that provides an extremely scalable, powerful, and resilient platform for running enterprise production workloads. There are a couple of concepts we have to get our heads around when it comes to the way vSAN handles failures – accessibility and availability. There are many underlying concepts and terminology that help to understand the different scenarios that affect a virtual machine residing on a vSAN datastore and how these scenarios are handled.
Let’s take a look at understanding VMware vSAN accessibility and availability and see how these two fundamental concepts are handled within the VMware vSAN environment.
VMware vSAN Object Store Design
VMware vSAN provides a modern storage design underneath the hood of the technology. VMware vSAN is a specialized type of “shared-nothing” object storage that represents virtual machines as objects and subcomponents on the vSAN datastore. One of the tremendous advantages of an object store is that availability can be defined on a per-object basis. The entire structure of how VMware vSAN deals with making virtual machines resilient between hosts is architected from these underlying objects.
Each of these objects consists of one or more components. This is determined by the size of the objects and the storage policy that is assigned to the object. A component is bound by the maximum size of 255 GB. Any object that is larger than 255GB is split into multiple components. This helps with many things including balancing capacity
Within VMware vSAN, a virtual machine is represented by objects. Within a VMware vSAN environment, the virtual machines are compromised of the following objects:
- VM Home, which contains virtual machine configuration files and logs, e.g., VMX file
- Virtual machine swap
- Virtual disk (VMDK)
- Delta disk (snapshot)
- Performance database
The objects above are the basic building blocks of vSAN storage for every virtual machine residing on vSAN. A vSAN object contains components that is determined by the configuration of the Virtual Machine Storage Policy. The VMware vSAN Storage Policy determines how these components are created across the vSAN landscape.
Below is a look at vSAN Default Storage Policy found in VMware vSphere vSAN 6.7 Update 1. As you can see there are various ways to configure the Failures to tolerate.
- No Data Redundancy
- 1 failure – RAID 1 (Mirroring)
- 1 failure – RAID 5 (Erasure Coding)
- 2 failures – RAID 1 (Mirroring)
- 2 failures – RAID 6 (Erasure Coding)
- 3 failures – RAID 1 (Mirroring)
The different failures to tolerate settings affect how the components are used to ensure the resiliency level defined. In the above configuration, the simplest form of redundancy is the NumberOfFailuresToTolerate=1 which means objects are mirrored/replicated. Each replica is considered to be a component of the vSAN object store. Any NumberOfFailuresToTolerate that is greater than 1 means that striping is involved to stripe data across multiple disks. This means that each stripe is considered to be an object.
There are specialized objects that help to make up the VMware vSAN components and objects that determine availability and accessibility.
- Replicas
- Witness component
A special type of object is a replica object. The number of replica objects created is determined by the setting specified in the vSAN Storage Policy that is used to set the resiliency of the storage objects for vSAN. The replica’s purpose is to allow the virtual machine to continue to run in the event of a failure in part of the vSAN physical infrastructure. Prior to vSAN 6.0, more than 50 percent of the vSAN objects must be accessible before the virtual machine can run. This is referred to as quorum. This quorum model changed starting with vSAN 6.0 in that now each component gets a vote. Now, more than 50% of all available votes is needed for quorum. The replicas which are part of this vote are part of the mechanism that achieves this accessibility so that vSAN virtual machines can be highly available and resilient to failures.
In a two-node configuration, each host would have a replica of the virtual machine components so that if a host fails, the virtual machine can still be available.
Wait, we need more than 50% of the votes right?
This is accomplished with the other specialized component, called the Witness component.
What is the witness component?
A witness component is provisioned for the purposes of quorum or as a tiebreaker in the case of a network partition or some other “split-brain” scenario. The witness component is placed on the third host. In a two-node configuration, there is a specialized witness appliance that serves the purpose of placing the witness components for virtual machines. The witness component is not a copy of the data for a virtual machine, but rather a special, very small, virtual machine metadata.
The various components and quorum mechanism that is baked into vSAN ensures the virtual machine data is accessible and there are enough resources in place to allow the virtual machine to come online. With the replicas in place that align with the configured storage policy as well as the special witness component, vSAN provides an extremely resilient solution where the infrastructure has the best chance possible for the “more than 50% of the votes” to be successful.
Absent vs Degrade Failure States in vSAN
VMware vSAN has two Failure States for components that determine how vSAN treats the failures. These are Absent and Degraded. These are viewed very differently by vSAN. The difference lies in the permanence from a failure perspective that vSAN determines exists with the failure. Thinking about a hard drive “failure” from a vSAN perspective, a drive that is simply “unplugged” and vSAN thinking it may or may not come back with data intact, is marked as Absent. An Absent state is viewed as potentially transitive. VMware vSAN will wait for 60 minutes in a state of Absent components before it will start a rebuild. The 60-minute timer allows the transient condition to potentially clear up before the rebuild is initiated.
The Degraded failure state means that vSAN views the failure as permanent. A drive that vSAN views as having a permanent failure will be marked as Degraded and the rebuild process for the underlying vSAN components will begin immediately and not wait on the 60-minute timer to be exhausted. A note here regarding the timer is, it is configurable. The 60-minute threshold is simply a recommended default but can be adjusted as needed.
In order for the rebuild mechanism to take place, vSAN must find a host to satisfy the placement requirements for rebuilding the components. This means, for example, a mirror set must be placed on two different hosts. Additionally, space requirements must be met in terms of free space on the remaining healthy disks to satisfy the rebuild requirements. Below, the vSAN environment is in a state of reduced availability with no rebuild since a mirror is down and there are no sufficient resources to complete a rebuild of the vSAN mirror.
Below, after a host was brought back online, the mirror was able to rebuild and the Placement and Availability status is now showing as Healthy for all objects. By ensuring the required components and replicas of the virtual machine are in place, the availability of the vSAN platform and the virtual machine is ensured. Even with failures as mentioned, the vSAN infrastructure is able to make the virtual machine highly available.
Takeaways
The powerful and resilient object storage platform that vSAN is built on allows for an extremely resilient solution on which to run production workloads. The object storage of vSAN makes use of an object representation of a virtual machine that allows effectively specifying resiliency at the object level. Objects are made up of special components including replicas and witness components ensure that more than 50% of the votes are available to ensure the virtual machine objects are accessible. The special features of vSAN ensure both accessibility and availability of virtual machines running on top of the vSAN infrastructure. This is made possible by aligning the replicas and witness components with the specified storage policy configuration and the size of the objects. The software-defined accessibility and availability mechanisms provide organizations with an extremely robust, next-generation storage solution that is easily scalable, resilient, and performant.
Follow our Twitter and Facebook feeds for new releases, updates, insightful posts and more.
