Object-store unavailable on a single node after reboot due to connection unavailable
Applies to
- ONTAP 9
- FabricPool
- Object-storage
- Intercluster-LIF
- Certificates
Issue
- After a node reboot or upgrade, object-store becomes unavailable only on one node
- The object-store shows available on all other nodes
- Logs similar to those shown below may be seen
EMS:
[node-01: OscLowPriThreadPool_3: object.store.unavailable:EMERGENCY]: Unable to connect to the object store "FP-Object-store" from node <node-uuid>. Reason: Connection unavailable.
[node-01: ktlsd: ktls.failed:notice]: "The TLS connections have failed several times with remote host '10.10.10.10' in IPspace 'xxxx', for which the latest reason given is: OpenSSL: error:0A000086:SSL routines::certificate verify failed."
Sktrace:
[0:0] CLOUD_BIN_ERR: object_store_config_ksmf_try_create: save=1, objStoreClientStoreSetup() failed for s3 handle -1fqdn="FP-Object-store", with error 105 (No IP-address found for given hostname)
...
[1:0] OSC_ERR: logCmd:371 HEAD CmdId 66 obj-name: failed with OSC error: Inter-cluster LIF not configured
...
[7:0] OSC_ERR: logCmd:371 GET CmdId 500 obj-name: 2d4ff701-c538-4ce3-84d8-4ae8b073014f/RelRootInfo_00000552_2d4ff701-c538-4ce3-84d8-4ae8b073014f failed with OSC error: Key not found