StorageGRID node CPU higher than other nodes
Applies to
StorageGRID pre-11.7.0.8
Issue
- Individual storage node showing higher CPU than others after initiating SnapMirror operation
- Placing affected node in Maintenance Mode, switches high CPU to a different node
- Load balancers dedicated to SnapMirror operation show TCP connection tracking utilization of 100%
- There is an increase in nf_conntrack
- Other symptoms may include...
- large amounts of packet loss exhibited through ping tests to the active gateway hosting the Load Balancer Endpoint.
- Individual storage nodes showing higher S3 traffic/rates then their peers.
Yellow (dc2h2) metric below highlights the individual node taking on more S3 requests.