High performance impact due to Motherboard issues on AFF-A300
Applies to
- ONTAP 9
- AFF-A300
Issue
- 100% CPU utilization after replacing the motherboard on the impaired node.
- All the volumes from the same node report high latency from DATA sub-system :
cluster::*> qos statistics volume latency show -vserver <vserver-name> -volume testvolume01_cluster1
Workload ID Latency Network Cluster Data Disk QoS Max QoS Min
--------------- ------ ---------- ---------- ---------- ---------- ---------- ---------- ----------
-total- - 9.35ms 750.00us 0ms 7.50ms 1083.00us 0ms 0ms
testvolume01_.. 6564 56.11ms 4.04ms 0ms 51.59ms 482.00us 0ms 0ms
-total- - 7.68ms 918.00us 0ms 5.75ms 1005.00us 0ms 0ms
testvolume01_.. 6564 50.03ms 6.07ms 0ms 43.55ms 411.00us 0ms 0ms
-total- - 7.93ms 560.00us 0ms 6.75ms 616.00us 0ms 0ms
testvolume01_.. 6564 56.79ms 2.67ms 0ms 53.75ms 370.00us 0ms 0ms
-total- - 9.73ms 867.00us 0ms 7.68ms 1174.00us 0ms 0ms
testvolume01_.. 6564 54.80ms 5.03ms 0ms 49.49ms 284.00us 0ms 0ms
-total- - 10.19ms 569.00us 0ms 7.95ms 1.67ms 0ms 0ms
- CPU utilization on the node reports >90 to 100% utilization on all CPU cores even without any major workload :
cluster::> node run -node <node-name> -command sysstat -m -c 30 -s 1
ANY AVG CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 CPU8 CPU9 CPU10 CPU11 CPU12 CPU13 CPU14 CPU15
100% 95% 96% 95% 96% 96% 95% 95% 96% 95% 95% 95% 95% 95% 95% 95% 96% 95%
100% 98% 97% 98% 98% 97% 98% 97% 98% 97% 98% 98% 97% 98% 98% 97% 98% 98%
100% 89% 91% 88% 90% 88% 89% 89% 88% 88% 90% 88% 89% 88% 87% 89% 92% 86%
100% 99% 99% 99% 99% 99% 99% 99% 99% 99% 99% 99% 99% 99% 99% 99% 99% 99%
100% 96% 96% 95% 95% 96% 95% 95% 95% 95% 95% 96% 96% 95% 96% 96% 96% 95%
100% 97% 96% 97% 96% 97% 95% 98% 97% 96% 96% 96% 97% 97% 96% 96% 97% 98%
cluster::> node run -node <node-name> -command sysstat -x 1
CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP
in out read write read write age hit time
84% 4455 0 0 4724 1800 101604 89417 24 0 0 43s 89% 100%
86% 3805 0 0 3933 1942 101654 103804 67209 0 0 50s 93% 100%
90% 2937 0 0 3028 1534 82590 95313 15497 0 0 45s 94% 100%
93% 3361 0 0 3470 1464 83808 72377 36 0 0 48s 92% 11%
93% 3596 0 0 3734 2643 84973 65356 4977 0 0 47s 91% 42%
91% 3462 0 0 3571 1246 74254 59665 24 0 0 47s 91% 0%
91% 3912 0 0 4031 2062 75502 65672 24 0 0 48s 90% 38%
91% 3844 0 0 3950 4401 138075 82362 3455 0 0 48s 93% 26%
94% 3742 0 0 3851 1097 60757 78074 36 0 0 49s 91% 0%
94% 3616 0 0 3746 1342 137586 88379 0 0 0 51s 93% 0%
93% 3662 0 0 3828 2253 59392 96558 24 0 0 52s 96% 0%
95% 3624 0 0 3857 1247 58682 93645 0 0 0 50s 97% 0%