Skip to main content
NetApp Knowledge Base

ONTAP S3 object store shows unavailable after bucket resize

  1. Last updated
  2. Save as PDF
Views:
1,436
Visibility:
Public
Votes:
0
Category:
ontap-9
Specialty:
core
Last Updated:

Applies to

  • ONTAP 9
  • ONTAP S3

Issue

  • The ONTAP System Manager reports “Cloud tier is not available”.
  • ONTAP S3 object store is unavailable:

Cluster::> object-store show -fields object-store-availability,object-store-unavailable-reason
  (storage aggregate object-store show)
aggregate        object-store-name             object-store-availability object-store-unavailable-reason
---------------- ----------------------------- ------------------------- -------------------------------
netapp1_01         FabricPool-LocalS3             unavailable               Operation Timedout
netapp1_02         FabricPool-LocalS3             available                 -
2 entries were displayed.

  • ONTAP S3 object store is full and the following alerts are reported in the event logs:
[Node-01: OscLowPriThreadPool: object.store.full:alert]: Failed to write to object store FabricPool-LocalS3 because it is out of space.
[Node-01: OscLowPriThreadPool: object.store.unavailable:EMERGENCY]: Unable to connect to the object store "FabricPool-LocalS3" from node d98XXXXX-fXX1-1XX9-8XX3-d0XXXXXXXXXd. Reason: Operation Timedout.
  • ONTAP S3 object store was resized:
cluster::> object-store-server bucket modify -bucket ObjectStore -size NewSize
[Job 1111] Job succeeded: Successful
  • Even after the resize, some nodes are unable to connect to the object store:
[cluster: OscLowPriThreadPool: object.store.unavailable:EMERGENCY]: Unable to connect to the object store "FabricPool-LocalS3" from node UUID. Reason: Operation Timedout.
 
cluster::*> storage aggregate object-store show -aggregate netapp1_01 -fields provider-type,object-store-availability,object-store-unavailable-reason
aggregate         object-store-name object-store-availability object-store-unavailable-reason provider-type
----------------- ----------------- ------------------------- ------------------------------- -------------
AggrName          FabricPool-LocalS3 unavailable                 Operation Timedout             ONTAP_S3

  • From the SKTRACE.GZ section in the ASUP, it can be found that ONTAP is unable to make a connection with the object store:

2023-04-12T07:05:49Z 49022153676645474    [3:0] OSC_ERR:  logCmd:342 HEAD Cmd ID:56 obj-name: failed with OSC error: 2
2023-04-12T07:05:49Z 49022153676666186    [3:0] OSC_ERR:  handleExecCommand:457 ConnPool 0xfffff70079f95000 Conn create error 5
 

 

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.