SolidFire storage full,repeated garbage collection failure (GCAborted) due to a thread resource error (boost::thread_resource_error)
Applies to
- NetApp Element OS 12.3.2
- NetApp HCI
- Garbage Collection (GC)
Issue
- The Element cluster entered a read-only storage full state and Garbage Collection failed
(GCAborted). - Cluster reported following alerts and gcEvent
Alerts:
blockClusterFull - Cluster capacity is completely consumed. Volumes are read-only and new connections are not permitted until additional capacity is available. Add additional capacity or free up capacity immediately.blockServiceTooFull - A Block Service is using 100% of the available space and space is getting critically low. Reads and writes can become disabled if this condition persists. You should immediately delete and purge volumes and snapshots or add more nodes.gcEvent:
GCAborted - {'err': 'xUnknownException', 'lastGCGeneration': 1763161200, 'lastStartSuccessful': True, 'paramCaller': 'OnAllSSStartResponses','paramGCGeneration': 1763161200, 'paramServiceID': 254}GCAborted - {'err': 'xGCAlreadyRunning', 'lastGCGeneration': 1763964824, 'lastStartSuccessful': True, 'paramCaller': 'OnAllSSGCStatusResponses','paramGCGeneration': 1763964824, 'paramServiceID': 254}