SMHV application-consistent backups fail
Applies to
- SnapManager for HyperV (SMHV) 2.1.4
- SnapDrive (SDW) 7.1.5
- Windows 2012 R2 HyperV Cluster with 3 nodes
Issue
Application Consistent backups fail and below errors can be observed in SnapDrvDBG.log:
06/08 12:48:04.615: ProcessID= 11432 ThreadID= 12 CreateDomainChannel for HostRegistrationManagement Service thrown exception e= Could not connect to net.tcp://server2/SnapManager/HyperV/HostRegistrationMgmtService/v_10. The connection attempt lasted for a time span of 00:00:21.0028903. TCP error code 10060: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.0.0.2:808.
06/08 12:48:04.617: ProcessID= 11432 ThreadID= 12 Web Service Client channel was unable to connect to the HostRegistrationManagementService instance on machine server3.
06/08-12:54:22.060 PID:6976 TID:6436 filer.cpp@1127 CFiler::static_system_get_version: unable to get filer version for 'SVM_Hyper-V'. Error: Can't connect to host (err=-11001).
06/08-12:54:22.061 PID:6976 TID:6436 filer.cpp@1128 Failed to get Data ONTAP version running on the storage system SVM_Hyper-V.
Error Description: Can't connect to host (err=-11001). HRESULT 0xc0040321.
06/08-12:54:22.061 PID:6976 TID:6436 filer_set.cpp@13 filer_t::get_network_name: start.
06/08-12:54:22.061 PID:6976 TID:6436 filer_set.cpp@19 filer_t::get_network_name: end. return '10.0.0.1'
06/08-12:54:22.061 PID:6976 TID:6436 System.cpp@959 CSystem::get_filer_live_ip(): found ip address '10.0.0.1' for filer 'SVM_Hyper-V'
From windows event viewer, additional errors can be seen as a consequence of the slowness of the windows intra node communication:
“Failed to rename the Snapshot copy ({CE2345CF-C718-4389-9893-00000}_backup) of the LUN to the new Snapshot copy name (test_cluster1_server3_05-30-2021_12.11.17_backup).” –“Error code : An attempt to rename Snapshot copy from '{CE2345CF-C718-4389-9893-00000}_backup' to 'test_cluster1_server3_05-30-2021_12.11.17_backup' on volume 'lun__server3' failed on the storage system '10.0.0.1'. Error code: 15661. Error description: entry doesn't exist
[6/8/2021 12:55:50 PM, (20)] The shadow copy provider timed out while holding writes to the volume being shadow copied. This is probably due to excessive activity on the volume by an application or a system service. Try again later when activity on the volume is reduced.
[6/8/2021 12:55:50 PM, (20)] Error: Vss Requestor - Backup Components failed.The shadow copy provider timed out while holding writes to the volume being shadow copied. This is probably due to excessive activity on the volume by an application or a system service. Try again later when activity on the volume is reduced.