When configuring storage for a Hyper-V cluster, there are certainly many options that need to be considered. In Windows Server 2012 a new file system was introduced called ReFS or Resilient File System. ReFS have been touted as the “next generation” file system from Microsoft that will eventually replace NTFS.
However, in the early versions of ReFS, it was a bit limited and the features and compatibility from vendors and other integrations were slow to adopt ReFS support. Windows Server 2012 didn’t see much adoption with ReFS from customers.
However, with Windows Server 2016 and especially with Hyper-V, Microsoft has included tremendous enhancements to ReFS version 2 such as Accelerated VHDX Operations.
In this post, we will take a look at Windows Server 2016 Hyper-V ReFS vs NTFS and which file system customers may want to use in various situations.
Learn more: How to create a Windows Server 2016 Hyper-V cluster?
Windows Server 2016 ReFS Version 3.1
What exactly is ReFS?
The ReFS files system, as mentioned, stands for “Resilient File System” and contains many new resiliency features that are contained within the file system.
The goals as stated by Microsoft with the ReFS file system is to maximize availability, scalability, and the integrity of the file system. Interestingly, with ReFS, you no longer have to run the chkdsk command utility that checks for errors in the file system and corrects those.
ReFS can perform online checks of the file system in real time and correct errors on the fly. This is done with a checksum mechanism that allows Windows to detect if there are any inconsistencies in the data and have the ability to recover from those inconsistencies.
A few notable details about the ReFS 3.1 file system:
- Has a maximum volume size of 1 Yottabyte or 1 trillion terabytes
- Uses file metadata to which is used to protect the data integrity of the file system
- This is used when extending a disk or zeroing out new blocks on a disk
- This is how the extreme performance enhancements of ReFS comes into play. Operations are performed on the metadata and not the actual data on disk such as merging checkpoints, etc
- NTFS would take minutes to do the same operation as it is actually zeroing out blocks instead of metadata
- This is referred to as the Sparse VDL technology
- Windows Server 2016 contains the new block cloning technology found in ReFS
In traditional block cloning, the file system is instructed to copy a range of file bytes on behalf of an application, where the destination file may be the same as, or different from the source file.
With block cloning in Windows Server 2016 ReFS, however this operation is performed as a simple metadata operation rather than reading from and writing to file data.
ReFS allow multiple files to share the same logical clusters, copy operations are simply remapping operations of file regions to another separate physical location.
There are a few block clone functionality restrictions and limitations – a few of which are the following:
- The cloned region must be less than 4GB in length
- The maximum number of file regions that can map to the same physical region is 8175
- Source and destination regions must not overlap
- Source and destination regions must be on the same ReFS volume
- If source is a sparse file, the destination must be sparse as well
- The ReFS volume must be formatted with Windows Server 2016 and if clustering is in use, the Clustering Functional Level must be Windows Server 2016 at the time of format of the volume
With Windows Server 2016 ReFS 3.1, there are specific Hyper-V use cases. It has the enhanced feature set called Accelerated VHDX Operations which improve VHDX and VHD operations such as the following:
- Creating a new VHDX or VHD disk – This operation when using NTFS could take multiple minutes to complete. With ReFS, it can take only a matter of seconds to provision a large VHDX or VHD disk.
- When merging Hyper-V Checkpoints
- Backups that are based on production checkpoints
ReFS have several limitations to make note of that come into play when thinking about various use cases.
- You cannot use ReFS for the system drive of the boot drive for a Windows operating system. When you perform an installation of the Windows operating system, you will not see the option to use the ReFS file system for formatting during the install.
- There are no options for deduplication currently with ReFS. This may be an enhancement in a future revision of ReFS, however, for now, this is not possible with ReFS. If you have a specific use case for deduplication in your environment, then you will by necessity need to stick with the NTFS file system.
- Disk quotas are another NTFS feature that is not supported with ReFS.
Windows Server 2016 Hyper-V ReFS Considerations
When configuring a Microsoft Windows Server 2016 Hyper-V cluster that is making use of Shared Storage, you want to configure cluster shared volumes or CSVs. When using CSVs, multiple Hyper-V hosts can access the same files on storage at the same time.
This was a major step forward back in the Windows Server 2008 days when CSVs were introduced.
The reason we want to introduce CSVs at this point is that you certainly want to consider your Hyper-V architecture when deciding when and where to utilize ReFS file system.
Since Windows Server 2016 ReFS has many exciting new features and is the brand new “next generation” file system, you might think you need to use it in any Windows Server 2016 Hyper-V environment.
However, there is a major reason not to use ReFS in cluster shared volumes. In CSV Hyper-V architecture, you always want to use NTFS as the file system. Why? Even though ReFS is supported in Windows Server 2016 Hyper-V, when it is used for Cluster Shared Volumes it always runs in file system redirection mode which sends all I/O over the cluster network to the coordinator node for the volume. In deployments utilizing NAS or SAN, this can dramatically impact CSV performance. When utilizing cluster shared volumes you want to always make use of NTFS as the preferred file system in production environments in this configuration.
In contrast, it is highly recommended to utilize ReFS with Storage Spaces Direct which is locally attached storage on each Hyper-V node. ReFS doesn’t impact performance in this configuration because RDMA network adapters are required when using Storage Spaces Direct.
What is RDMA?
RDMA is remote direct memory access. It is basically zero-copy networking which enables the network adapter to transfer data directly to or from application memory, eliminating the need to copy data between application memory and the data buffers in the operating system. This bypasses any bottleneck that may be imposed by the CPU, caches, or other hardware in between. When utilizing the RDMA network adapters with Storage Spaces Direct there is no shared storage as such so all access is essentially redirected across the nodes utilizing the power of the RDMA network adapters.
A note on performance and cluster size with ReFS or NTFS.
It is best practice with Hyper-V to format the volume with 64 KB block size for optimal performance. With Hyper-V, the partition alignment is handled automatically in recent versions of the hypervisor, so this is something the administrator does not have to worry about manually doing.
Windows Server 2016 Hyper-V offers organizations a very robust solution that contains a tremendous number of options in regards to storage and other configurations. With Windows Server 2016 Hyper-V, there is a valid choice in choosing the file system for the Hyper-V configuration.
ReFS 3.1 that is contained in Windows Server 2016 contains some really great features and functionality that are extremely appealing for Hyper-V environments. However, there are definitely cautions with ReFS. In today’s Hyper-V configurations, unless you are making use of a Storage Spaces Direct configuration, it is recommended to use NTFS when utilizing shared storage with cluster shared volumes (CSVs). This is due to the redirected traffic that happens with clustered shared volumes formatted with ReFS in a Hyper-V cluster. This can potentially kill performance in this configuration.
However, using ReFS in a Storage Spaces Direct Hyper-V cluster configuration is the preferred file system of choice since RDMA networking in Storage Spaces Direct offsets the redirected I/O path when using ReFS. ReFS provides some really amazing advantages by using the metadata operations instead of actual file operations. This includes block cloning, sparse VDL and other Advanced VHDX Operations. One thing is for sure, Windows Server 2016 Hyper-V is the most capable and robust Hyper-V to date and the storage options are no exception.Like what you read? Rate us