FabricPool volume move stuck/ fail due to operation timeout from storageGRID
Applies to
- FabricPool
- ONTAP OS 9.12
- StorageGRID OS 11.5.0
Issue
- Volume move from AFF gets stuck/ fails due to operation timed out while reading object from StorageGRID:
OSC_ERR: logCmd:368 GET CmdId: 1265 obj-name: <Object_path> failed with OSC error: Operation Timedout
- StorageGRID Gateway node
nginx-gw
access.log
shows error:
[error] 23431#23431: *1031715311 connect() failed (111: Connection refused) while connecting to upstream, client: <Gateway_node>, server: _, request: "GET /<Object_path> HTTP/1.1", upstream: "https://<Storage_node>:18082/<Object_path>", host: "<Endpoint>:10443"
499
errors increment on gateway nodes as Volume move starts:
Grid manager > Nodes > gateway node > Load Balancer > Error Response Rate graph
: