How to identify unaligned IO on LUNs?

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Views:: 9,763

Visibility:: Public

Votes:: 13

Category:: ontap-9

Specialty:: SAN

Last Updated:

Applies to

ONTAP 9.x
SAN
FlexPod

Procedure

LUNs as storage devices by host operating system

To understand I/O alignment, it is important to know how LUNs are used as storage devices by the host operating system:

Traditional block size advertised by hard disks is 512 bytes.
NetApp LUNs use a larger block size to store data while continuing to advertise smaller 512 byte blocks to the host for compatibility purposes.
Advertised blocks are referred to as logical blocks and underlying storage block is referred to as a physical block.
NetApp LUN stores data in 4 KB physical blocks, which yields eight 512 byte logical blocks per physical block.

LUNs as storage devices by host operating system

Host operating system might begin a Read or Write (I/O) operation at any logical block.
However, when I/O begins at a logical block that is not at the start of a physical block, then I/O is said to be misaligned.

Unaligned IO

I/Os are aligned only when they begin at a logical block; that is the first logical block in a physical block. I/O aligns to the physical block boundary.

I/O aligns to the physical block boundary

Identifying I/O alignment for LUNs

Data ONTAP contains a heuristic detection mechanism that automatically identifies misaligned I/O to LUNs.
The heuristic checks the work by observing the position within a physical block where I/O begins, as well as the I/O length.
The heuristic’s output is one of the following:
- indeterminate
- aligned
- misaligned
- partial writes

Indeterminate state Only applied when the LUN has not received enough I/O to decide on alignment.

Partial writes state

Identifies LUNs that receive the majority of I/Os smaller than 4kB.

Typically, this state applies to LUNs used by database transaction or redo logs.
This state does not indicate an issue; it is used to separate situation from misaligned state.
As workloads change, it might be necessary to reset the alignment heuristics by running lun stats –z command.

► Click to see the example showing I/O as misaligned

Read and Write alignment histograms record the percentage of I/O that begins at each of the eight positions within a physical block.
Here it shows that 100% of read and write I/Os are starting in the eighth position; also known as a histogram bucket.

*> lun alignment show /vol/luns/my_lun Multiprotocol type: linux Alignment: misaligned Write alignment histogram percentage: 0, 0, 0, 0, 0, 0, 0, 100 Read alignment histogram percentage: 0, 0, 0, 0, 0, 0, 0, 100 Partial writes percentage: 0 Partial reads percentage: 0

► Click to see the next example showing a LUN the I/Os as aligned.

The Read and Write alignment histograms show 100% of the I/Os are beginning in the first position, which is aligned to the physical block boundary.

*> lun alignment show /vol/luns/my_lun Multiprotocol type: windows Alignment: aligned Write alignment histogram percentage: 100, 0, 0, 0, 0, 0, 0, 0 Read alignment histogram percentage: 100, 0, 0, 0, 0, 0, 0, 0 Partial writes percentage: 0 Partial reads percentage: 0

► Click to see the final example showing the partial writes state on a LUN

Here you see some percentage of total I/O being accounted for in the Partial writes and Partial reads fields.
The exact amount of partial writes and reads are dependent on the application performing the I/O and the nature of the workload on that application.
Percentages might not add up to 100; this is expected and does not indicate an issue with the heuristic calculation.

*> lun alignment show /vol/luns/my_lun Multiprotocol type: linux Alignment: partial-writes Write alignment histogram percentage: 0, 0, 0, 0, 0, 0, 0, 0 Read alignment histogram percentage: 1, 6, 1, 3, 0, 4, 6, 6 Partial writes percentage: 0 Partial reads percentage: 68

Contributing factors to I/O alignment for LUNs

The way a host operating system uses a LUN varies greatly by OS type, partition scheme, file system and application.
For the majority, the key contributing factor is the partition scheme of the host OS and can also be influenced by file system or application/database.
As a rule of thumb, choose the Data ONTAP LUN OS type that most closely matches the host operating system to the OS type.

Notes:

Data ONTAP might report I/O misalignments on properly aligned LUNs.
These misalignment warnings can be disregarded as long as you are confident LUN is correctly provisioned and partitioning table is correct.

Examples:

Boot LUNs	These are targets for various logs and other smaller writes. Small writes done will look like partial writes, which in turn looks like misaligned IO. They can be safely ignored. Most boot LUNs will have some number of partial writes. Therefore, might look misaligned even though the LUN geometry and LUN type are correct. If you know the LUN geometry is correct, ignore the misalignment.
Database Log LUNs	LUNs that are used as the log location for a Database application, such as Oracle (redo log) or Microsoft's SQL Server (transaction log), can see random distribution of unaligned write I/O across the alignment histogram. This is normal and expected behavior for this type of I/O. In cases where we know LUN is being used as a DB log location, and have confirmed that correct lun type, and offset are being used, no further corrective actions are possible or necessary.

Boot LUNs

These are targets for various logs and other smaller writes.
Small writes done will look like partial writes, which in turn looks like misaligned IO. They can be safely ignored.
Most boot LUNs will have some number of partial writes. Therefore, might look misaligned even though the LUN geometry and LUN type are correct.
If you know the LUN geometry is correct, ignore the misalignment.

Database Log LUNs

LUNs that are used as the log location for a Database application, such as Oracle (redo log) or Microsoft's SQL Server (transaction log), can see random distribution of unaligned write I/O across the alignment histogram.
This is normal and expected behavior for this type of I/O.
In cases where we know LUN is being used as a DB log location, and have confirmed that correct lun type, and offset are being used, no further corrective actions are possible or necessary.

The following table provides additional guidance.

In some circumstances, custom partition tables might be required.

Some Data ONTAP LUN OS types use an offset known as a prefix, to enable the default partition scheme, used by the associated host operating system to be aligned.
For LUNs with a prefix value greater than 0, a custom partition might create misaligned I/O.

LUN ostype	Prefix (bytes)	Prefix (sectors)	Operating System
windows	32,256	63	Windows 2000, 2003 (MBR format)
windows_gpt	17,408	34	Windows 2003 (GPT format)
windows_2008	0	0	Windows 2008 and later
hyper_v	0	0	Windows 2008 Hyper-V and later
linux	0	0	All Linux distributions *
xen	0	0	Citrix XenServer
vmware	0	0	VMware ESX*
solaris	1MB	2048	Solaris *
solaris_efi	17,408	34	Solaris *
hpux	0	0	HP-UX
aix	0	0	AIX

* Indicates additional consideration might be required; see the Additional Information below.

* Note in ONTAP 9.8 and higher, all LUN types use a 0 prefix and suffix to facilitate transitioning to NVME. Luns created prior to 9.8 are still supported but cannot be transitioned to NVME namespaces.

Additional Information

Additional considerations for Linux

Linux distributions offer a variety of ways to use a LUN, including as raw devices for databases, volume managers and file systems.
It is not necessary to create partitions on a LUN, when used as a raw device or as a physical volume in a volume manager.
If the LUN is used without a volume manager, then a good practice is to partition the LUN, such that it has one partition, which begins at an aligned offset; that is a sector of an even multiple of eight logical blocks.
For details on creating aligned partitions of Linux, see How to create aligned partitions in Linux for use with NetApp LUNs, VMDKs, VHDs and other virtual disk containers.

Additional considerations for VMware ESX/ESXi

Typically, unaligned I/O seen on a type VMware LUN is due to VMDK alignment issues introduced by the guest OS partitioning of its disk.

Additional considerations for Solaris

Solaris offers a wide variety of ways to use a LUN, including many different file systems and volume managers.
No special partitioning is necessary for Solaris; however, it is important to know when to use either solaris or solaris_efi LUN ostype.
You should use the solaris_efi LUN ostype only for LUNs larger than 990GB that will be formatted with UFS. The solaris_efi LUN ostype is used to offset the default starting location of the first partition after the EFI disk label, which is sector 34.
You should use the solaris LUN ostype for all other configurations, including SVM, VxVM, ZFS and UFS smaller than 990GB.
On Solaris versions 10u8 and newer, where ZFS is being used, a LUN ostype of solaris should be used along with adding physical-block-size:4096 to the host's sd.conf file (ssd.conf for Solaris x86).

I/O alignment for files

I/O alignment for files works exactly like it does for LUNs.
Typically, misaligned I/O for files occurs with those files used as virtual disks or disk images accessed via NAS protocol.
It is therefore important that virtual disks have their partitions aligned to file block boundaries for optimal I/O throughput.