Latency and disk utilization increases such as ONTAP shutdown
Applies to
Any FAS model with Flash Cache (PAM or NVMe SSD)
Issue
ONTAP may report higher latency following a reboot, shutdown, or upgrade. EMS logs "
extCache.warming
" or "extCache.aggregate.invalidate
" are logged, such as:Sun Jul 26 09:41:38 -0500 [clus1-02: cf_giveback: extCache.aggregate.invalidate:info]: WAFL external cache: Started deferred invalidations for 2 aggregates. Sun Jul 26 09:41:38 -0500 [clus1-02: ec_exempt_worker_thread: extCache.aggregate.invalidate:info]: WAFL external cache: Starting deferred invalidation for aggregate in map slot 4. Sun Jul 26 09:41:38 -0500 [clus1-02: ec_exempt_worker_thread: extCache.aggregate.invalidate:info]: WAFL external cache: Starting deferred invalidation for aggregate in map slot 3. Sun Jul 26 09:41:41 -0500 [clus1-02: ec_exempt_worker_thread: extCache.aggregate.invalidate:info]: WAFL external cache: Completed invalidation for aggregate in map slot 4 in 3 seconds. Sun Jul 26 09:41:41 -0500 [clus1-02: ec_exempt_worker_thread: extCache.aggregate.invalidate:info]: WAFL external cache: Completed invalidation for aggregate in map slot 3 in 3 seconds. Sun Jul 26 09:41:41 -0500 [clus1-02: ec_exempt_worker_thread: extCache.aggregate.invalidate:info]: WAFL external cache: All pending deferred aggregate invalidations have completed. Total elapsed time: 3 seconds.
Also latency may be seen in System Manager or Active IQ Unified Manager. Here are a couple examples from a system that just went an ONTAP upgrade on July 26, and you can see right at midnight July 27 the latency is slightly higher:
ActiveIQ shows the latency increasing in the "Aggregate Ops" Cluster Component, or basically increased latency from the hard drives since the percentage of reads from Flash Cache are less.