How to identify unaligned IO on LUNs?
Applies to
- ONTAP 9.x
- SAN
- FlexPod
Procedure
LUNs as storage devices by host operating system
To understand I/O alignment, it is important to know how LUNs are used as storage devices by the host operating system:
- Traditional block size advertised by hard disks is 512 bytes.
- NetApp LUNs use a larger block size to store data while continuing to advertise smaller 512 byte blocks to the host for compatibility purposes.
- Advertised blocks are referred to as logical blocks and underlying storage block is referred to as a physical block.
- NetApp LUN stores data in 4 KB physical blocks, which yields eight 512 byte logical blocks per physical block.
- Host operating system might begin a Read or Write (I/O) operation at any logical block.
- However, when I/O begins at a logical block that is not at the start of a physical block, then I/O is said to be misaligned.
- I/Os are aligned only when they begin at a logical block; that is the first logical block in a physical block. I/O aligns to the physical block boundary.
Identifying I/O alignment for LUNs
- Data ONTAP contains a heuristic detection mechanism that automatically identifies misaligned I/O to LUNs.
- The heuristic checks the work by observing the position within a physical block where I/O begins, as well as the I/O length.
- The heuristic’s output is one of the following:
- indeterminate
- aligned
- misaligned
- partial writes
Indeterminate state | Only applied when the LUN has not received enough I/O to decide on alignment. |
Partial writes state |
Identifies LUNs that receive the majority of I/Os smaller than 4kB.
|
- ► Click to see the example showing I/O as misaligned
-
- Read and Write alignment histograms record the percentage of I/O that begins at each of the eight positions within a physical block.
- Here it shows that 100% of read and write I/Os are starting in the eighth position; also known as a histogram bucket.
*> lun alignment show /vol/luns/my_lun
Multiprotocol type: linux
Alignment: misaligned
Write alignment histogram percentage: 0, 0, 0, 0, 0, 0, 0, 100
Read alignment histogram percentage: 0, 0, 0, 0, 0, 0, 0, 100
Partial writes percentage: 0
Partial reads percentage: 0 - ► Click to see the next example showing a LUN the I/Os as aligned.
-
- The Read and Write alignment histograms show 100% of the I/Os are beginning in the first position, which is aligned to the physical block boundary.
*> lun alignment show /vol/luns/my_lun
Multiprotocol type: windows
Alignment: aligned
Write alignment histogram percentage: 100, 0, 0, 0, 0, 0, 0, 0
Read alignment histogram percentage: 100, 0, 0, 0, 0, 0, 0, 0
Partial writes percentage: 0
Partial reads percentage: 0 - ► Click to see the final example showing the partial writes state on a LUN
-
- Here you see some percentage of total I/O being accounted for in the Partial writes and Partial reads fields.
- The exact amount of partial writes and reads are dependent on the application performing the I/O and the nature of the workload on that application.
- Percentages might not add up to 100; this is expected and does not indicate an issue with the heuristic calculation.
*> lun alignment show /vol/luns/my_lun
Multiprotocol type: linux
Alignment: partial-writes
Write alignment histogram percentage: 0, 0, 0, 0, 0, 0, 0, 0
Read alignment histogram percentage: 1, 6, 1, 3, 0, 4, 6, 6
Partial writes percentage: 0
Partial reads percentage: 68
Contributing factors to I/O alignment for LUNs
- The way a host operating system uses a LUN varies greatly by OS type, partition scheme, file system and application.
- For the majority, the key contributing factor is the partition scheme of the host OS and can also be influenced by file system or application/database.
- As a rule of thumb, choose the Data ONTAP LUN OS type that most closely matches the host operating system to the OS type.
Notes:
Examples: |
Boot LUNs |
|
---|---|
Database Log LUNs |
|
The following table provides additional guidance.
In some circumstances, custom partition tables might be required.
- Some Data ONTAP LUN OS types use an offset known as a prefix, to enable the default partition scheme, used by the associated host operating system to be aligned.
- For LUNs with a prefix value greater than 0, a custom partition might create misaligned I/O.
LUN ostype |
Prefix (bytes) |
Prefix (sectors) |
Operating System |
---|---|---|---|
windows |
32,256 |
63 |
Windows 2000, 2003 (MBR format) |
windows_gpt |
17,408 |
34 |
Windows 2003 (GPT format) |
windows_2008 |
0 |
0 |
Windows 2008 and later |
hyper_v |
0 |
0 |
Windows 2008 Hyper-V abd later |
linux |
0 |
0 |
All Linux distributions * |
xen |
0 |
0 |
Citrix XenServer |
vmware |
0 |
0 |
VMware ESX* |
solaris |
1MB |
2048 |
Solaris * |
solaris_efi |
17,408 |
34 |
Solaris * |
hpux |
0 |
0 |
HP-UX |
aix |
0 |
0 |
AIX |
* Indicates additional consideration might be required; see the Additional Information below.
* Note in ONTAP 9.8 and higher, all LUN types use a 0 prefix and suffix to facilitate transitioning to NVME. Luns created prior to 9.8 are still supported but cannot be transitioned to NVME namespaces.
Additional Information
Additional considerations for Linux
- Linux distributions offer a variety of ways to use a LUN, including as raw devices for databases, volume managers and file systems.
- It is not necessary to create partitions on a LUN, when used as a raw device or as a physical volume in a volume manager.
- If the LUN is used without a volume manager, then a good practice is to partition the LUN, such that it has one partition, which begins at an aligned offset; that is a sector of an even multiple of eight logical blocks.
- For details on creating aligned partitions of Linux, see How to create aligned partitions in Linux for use with NetApp LUNs, VMDKs, VHDs and other virtual disk containers.
Additional considerations for VMware ESX/ESXi
Typically, unaligned I/O seen on a type VMware LUN is due to VMDK alignment issues introduced by the guest OS partitioning of its disk.
Additional considerations for Solaris
- Solaris offers a wide variety of ways to use a LUN, including many different file systems and volume managers.
- No special partitioning is necessary for Solaris; however, it is important to know when to use either solaris or solaris_efi LUN ostype.
- You should use the solaris_efi LUN ostype only for LUNs larger than 990GB that will be formatted with UFS. The
solaris_efi
LUN ostype is used to offset the default starting location of the first partition after the EFI disk label, which is sector 34. - You should use the solaris LUN ostype for all other configurations, including SVM, VxVM, ZFS and UFS smaller than 990GB.
- On Solaris versions 10u8 and newer, where ZFS is being used, a LUN ostype of solaris should be used along with adding physical-block-size:4096 to the host's
sd.conf
file (ssd.conf
for Solaris x86).
I/O alignment for files
- I/O alignment for files works exactly like it does for LUNs.
- Typically, misaligned I/O for files occurs with those files used as virtual disks or disk images accessed via NAS protocol.
- It is therefore important that virtual disks have their partitions aligned to file block boundaries for optimal I/O throughput.