Skip to main content
NetApp Knowledge Base

Object store unavailable causing replication failure

  1. Last updated
  2. Save as PDF
Views:
286
Visibility:
Public
Votes:
0
Category:
ontap-9
Specialty:
ontapselect
Last Updated:

Applies to

  • ONTAP 9
  • Storage Grid
  • ONTAP Select
  • AWS

Issue

  • Object store unavailable messages seen in ems logs

Mon Mar 27 06:32:57 +0000 [OntapData-01: OscLowPriThreadPool: object.store.unavailable:EMERGENCY]: Unable to connect to the object store "Object_Store" from node eyyyyyy4-821c-498c-a82a-bxxxxxxxxxxd. Reason: Operation Timedout.
Mon Mar 27 06:33:42 +0000 [OntapData-01: OscLowPriThreadPool: object.store.available:notice]: Able to connect to the object store "Object_Store" from node eyyyyyy4-821c-498c-a82a-bxxxxxxxxxxd.
Mon Mar 27 12:59:21 +0000 [OntapData-01: OscLowPriThreadPool: object.store.unavailable:EMERGENCY]: Unable to connect to the object store "Object_Store" from node eyyyyyy4-821c-498c-a82a-bxxxxxxxxxxd. Reason: Operation Timedout.
Mon Mar 27 13:00:06 +0000 [OntapData-01: OscLowPriThreadPool: object.store.available:notice]: Able to connect to the object store "Object_Store" from node eyyyyyy4-821c-498c-a82a-bxxxxxxxxxxd

  • Reason for Object store unavilable is seen to be Operation Timedout

OntapData::*> aggr object-store show -instance

                                   Aggregate Name: aggr1
          ONTAP Name for this Object Store Config: Object_Store
                                  UUID of the Bin: ayyyyyy0-bd25-11ea-a4fc-0xxxxxxxxxxd
                                       Bin Number: 1
                 Availability of the Object Store: unavailable
           Reason why Object Store is Unavailable: Operation Timedout
                 Object Store Config Id in Dblade: 1
                         S3 Client Handle in Cbin: 0
  SMDB Error for Partially Failed get() or next(): -
                             Bin is being Created: false
                                  Is Bin Deleted?: false
                            UUID of the Aggregate: eyyyyyy4-821c-498c-a82a-bxxxxxxxxxx8
           UUID of the Object Store Configuration: 9yyyyyye-bd25-11ea-a4fc-0xxxxxxxxxxd
                Type of the Object Store Provider: SGWS
                       Cached Value of Used Space: 98.67TB
                       License Space Used Percent: -
      Threshold for Reclaiming Unreferenced Space: 40%
Aggregate Fullness Threshold Required for Tiering: 50%
                              Name Format Version: 0
                         Object Store Mirror Type: primary
     This object store is in mirror degraded mode: -
Force Tiering with no Mirror in a MetroCluster Configuration: false
Allow a degraded cloud mirror to be swapped with an unavailable primary object store: -
                     Pending Operation on the Bin: none
True if this object store is the primary destination for IO: true
The name of the Cluster to which the bin belongs: OntapData
                         Modify Count for the Row: 0

  • Network ping/traceroute to object store via intercluster IP might also fail
  • Packet traces collected on the intercluster IPs also point to network related issues

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.