CVO upgrade fails due to the failure of data LIFs' migration
Applies to
Cloud Volume ONTAP (CVO)
Issue
CVO upgrade fails due to the failure of data LIF's migration.
Example:
node::> cluster image show-update-progress
Estimated Elapsed
Update Phase Status Duration Duration
-------------------- ----------------- --------------- ---------------
Pre-update checks completed 00:10:00 00:01:00
Data ONTAP updates paused-on-error 01:32:00 19:57:18
Details:
Node name Status Status Description
-------------------- ----------------- --------------------------------------
node-01 waiting
node-02 failed Error: Failed to migrate data LIFs
from node "node-02".
Action: Migrate all of the data LIFs
using the "network interface
migrate-all -node node-02" command.
5 entries were displayed.
Status: Paused - An error occurred in "Data ONTAP updates" phase. The update cannot continue until the error has been resolved.
Resolve all issues, then use the "cluster image resume-update" command to resume the update. node::> network interface migrate-all -node node-02Error: command failed: Failed to migrate one or more LIFs away from node "node-02". Use the "network interface show
-curr-node node-02" command to review the status of any remaining LIFs on that node.node::> event log show
Time Node Severity Event
------------------- ---------------- ------------- ---------------------------
9/17/2020 02:26:58 node-02 ERROR vifmgr.extRoute.noConn: Failed to verify connectivity with the mediator se
rver, IP 10.135.26.11 (error Failed to verify connectivity to the mediator. The system encountered a timeout waiting for the
mediator to complete the operation. Retry the command. If the problem persists, contact technical support personnel for assi
stance.).vifmgr.log via ASUP
00000003.00192c6f 0e9d7d9e Thu Sep 17 2020 02:27:03 +00:00 [kern_vifmgr:info:8150] [0x808e43600] [FloatingIpMgrImpl::postOperationsToMediator] Failed to update external routing information; failed to send POST msg to mediator, URI: http://10.135.26.11:3080/v1/clusters/2911f4de-1987-11ea-a11d-111111/route_tables/update_all, request: {"IPs":[{"ip":"10.135.26.22","dest":"eni-0614d0e1b8a4a4a4","op":"replace"}]}, status: Timeout was reached, err: Failed to update external route tables for IP "10.135.26.22". The system encountered a timeout waiting for the mediator to complete the operation. Retry the command. If the problem persists, contact technical support personnel for assistance., server response: