Object store unavailable causing replication failure
Applies to
- ONTAP 9
- Storage Grid
- ONTAP Select
Issue
- Object store unavailable messages seen in ems logs
Mon Mar 27 06:32:57 +0000 [OntapData-01: OscLowPriThreadPool: object.store.unavailable:EMERGENCY]: Unable to connect to the object store "Object_Store" from node eyyyyyy4-821c-498c-a82a-bxxxxxxxxxxd. Reason: Operation Timedout.
Mon Mar 27 06:33:42 +0000 [OntapData-01: OscLowPriThreadPool: object.store.available:notice]: Able to connect to the object store "Object_Store" from node eyyyyyy4-821c-498c-a82a-bxxxxxxxxxxd.
Mon Mar 27 12:59:21 +0000 [OntapData-01: OscLowPriThreadPool: object.store.unavailable:EMERGENCY]: Unable to connect to the object store "Object_Store" from node eyyyyyy4-821c-498c-a82a-bxxxxxxxxxxd. Reason: Operation Timedout.
Mon Mar 27 13:00:06 +0000 [OntapData-01: OscLowPriThreadPool: object.store.available:notice]: Able to connect to the object store "Object_Store" from node eyyyyyy4-821c-498c-a82a-bxxxxxxxxxxd
- Reason for Object store unavilable is seen to be Operation Timedout
OntapData::*> aggr object-store show -instance
Aggregate Name: aggr1
ONTAP Name for this Object Store Config: Object_Store
UUID of the Bin: ayyyyyy0-bd25-11ea-a4fc-0xxxxxxxxxxd
Bin Number: 1
Availability of the Object Store: unavailable
Reason why Object Store is Unavailable: Operation Timedout
Object Store Config Id in Dblade: 1
S3 Client Handle in Cbin: 0
SMDB Error for Partially Failed get() or next(): -
Bin is being Created: false
Is Bin Deleted?: false
UUID of the Aggregate: eyyyyyy4-821c-498c-a82a-bxxxxxxxxxx8
UUID of the Object Store Configuration: 9yyyyyye-bd25-11ea-a4fc-0xxxxxxxxxxd
Type of the Object Store Provider: SGWS
Cached Value of Used Space: 98.67TB
License Space Used Percent: -
Threshold for Reclaiming Unreferenced Space: 40%
Aggregate Fullness Threshold Required for Tiering: 50%
Name Format Version: 0
Object Store Mirror Type: primary
This object store is in mirror degraded mode: -
Force Tiering with no Mirror in a MetroCluster Configuration: false
Allow a degraded cloud mirror to be swapped with an unavailable primary object store: -
Pending Operation on the Bin: none
True if this object store is the primary destination for IO: true
The name of the Cluster to which the bin belongs: OntapData
Modify Count for the Row: 0
- Network ping/traceroute to object store via intercluster IP fails