Switchover simulation fails due to object store unreachable after SGWS upgrade
Applies to
- ONTAP 9
- FabricPool
- FAS
- AFF
- StorageGrid
Issue
- Metrocluster switchover simulation fails:
cluster::> metrocluster operation show
Operation: switchover-simulate
State: failed
Start Time: 7/29/2022 09:20:32
End Time: 7/29/2022 09:20:48
Errors: Failed to validate the node and cluster components before the switchover operation.
cluster (non-overridable veto): Cannot verify availability of the object store from node node_name Reason: Wrong port or server is not reachable.
- Object-store mirrors go unavailable.
cluster::> storage aggregate object-store show
Aggregate Object Store Name Availability Mirror Type
-------------- ----------------- ------------- -----------
aggr_01 cluster-mirror unavailable mirror
aggr_01 cluster-primary available primary
2 entries were displayed.
cluster::> storage aggregate object-store show -object-store-name cluster-mirror -instance
Aggregate Name: aggr_01
ONTAP Name for this Object Store Config: cluster-mirror
Availability of the Object Store: unavailable
Reason why Object Store is Unavailable: Connection unavailable
Cached Value of Used Space: 137.4TB
- Event logs report the object store unavailable due to timeout errors:
[node_name: OscLowPriThreadPool: object.store.unavailable:EMERGENCY]: Unable to connect to the object store "cluster-mirror" from node 210xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx36e. Reason: Operation Timedout.- The issue starts post upgrading the StorageGrid Webscale (SGWS) from 11.5.0.8 to 11.6.0.2
- After the upgrade, the SGWS firmware does not reflect correctly on the ONTAP end:
cluster::> storage aggregate object-store config show -instance (Output truncated for brevity)
Object Store Configuration Name: cluster-mirror
Type of the Object Store Provider: SGWS
Server Field Returned in HTTP Header: StorageGRID/11.5.0.8