CONTAP-749608: GDD Advanced Files larger than 25 GB might experience intermittent higher write latencies

Some writes to a large GDD file may end up in a livelock loop, causing those writes to experience additional latency before they complete.
This issue happens because writes need to establish some metadata state for the part targeted by the write to optimize for subsequent I/O, and for large files they also need to establish this metadata state on neighboring parts.
The amount of extra latency experienced by an affected write depends on node hardware and the amount of concurrent cluster workload.

Exposure Conditions:
In order for a write to be impacted, a number of conditions must be present.

The targeted GDD file has to have more than 25 parts such that an optimization engages to mark the metadata state not just for the targeted part, but for neighboring parts as well.
Additionally, the write is only impacted if the targeted part already has the metadata state marked on it, but its neighbor(s) do not.
The targeted part must be on a volume that is not the volume hosting the GDD Catalog. Once the targeted part and its neighbors have the metadata state marked on them, subsequent writes cannot be impacted.

Additional Symptoms:

This issue can be intermittent because every 20 minutes, a metadata operation (like an NFS GetAttr) may wipe away the I/O optimizing metadata state for all parts of the file. This exposes new writes to the issue, until again the targeted part and its neighbors have the metadata state re-established on them.
With a typical FlexGroup stripe width of 1 GB, a file would have to be more than 25 GB in size before it has more than 25 parts and is exposed to this issue. Larger files may not be impacted, if one or more of their parts were created with a larger than normal stripe width, but this should be rare.
WAFL includes a failsafe mechanism to detect and break the livelock. If the write loops 20,000 times, WAFL takes an alternate code path that breaks the loop and allows the write to proceed. Therefore, the maximum additional latency for an affected write is the time required to loop 20,000 times and trigger the failsafe mechanism.
Reads are also exposed; however WAFL typically decides to mark this I/O-optimizing metadata state during reads only once per hour, so reads are less likely to encounter the issue.