Skip to main content
NetApp Knowledge Base

SCV - Unable to find snapshot for SnapVault or SnapMirror update

Views:
4,584
Visibility:
Public
Votes:
3
Category:
snapcenter
Specialty:
snapx
Last Updated:

Applies to

SnapCenter Plug-in for VMware vSphere (SCV) 4.5 and above

Issue

  • SnapMirror or SnapVault operations fails with one the following messages:

The error on source side:
Unable to find Snapshot (RG|Policy_YYYY-MM-dd_HH:mm:ss.0ms) on volume (primary-volume) from storage (primary-SVM-fqdn).
Failed to attach the label to the specified Snapshot copy.(RG|Policy_YYYY-MM-dd_HH:mm:ss.0ms). Invalid response returned by Set-NcSnapshot: 0

The error on destination side:
Error: SnapVault update failed for the relationship [ source-SVM : source-volume ==> destination-SVM : destination-volume]
with error: Snapshot copy 'RG|Policy_YYYY-MM-dd_HH:mm:ss.0ms' not found on the SnapVault destination destination-SVM : destination-volume

  • SCV is unable to complete SnapMirror or SnapVault operations successfully and will not show as having secondary backups for these failures.
  • SCV backup completes with Warning:

Mirror failed: SnapMirror update failed for the relationship [<SOURCE_PATH> ==> <DESTINATION_PATH>] with error: Snapshot copy '<SNAPSHOT>' not found on the SnapMirror destination <DESTINATION_PATH>

  • wfe_<JOB_ID>.log shows message:

DEBUG SnapCenter --- [nio-8080-exec-4] c.n.w.c.BackupController                 : Message: SnapMirror update failed for the relationship [<SOURCE_PATH> ==> <DESTINATION_PATH>] with error: Snapshot copy '<SNAPSHOT>' not found on the SnapMirror destination <DESTINATION_PATH>

Cause

  • Snapshot cannot be found on source side if:
    • Snapshot is deleted by ONTAP autodelete on the volume, in which case, the solution below will not help (adjust Volume settings instead)
    • Storage system's volume database is not updated fast enough after taking the snapshot, seen often in MCC environment or with LS Mirror on SVM root
  • Snapshot cannot be found on destination side if:
    • SnapVault or SnapMirror update transfer failed, in which case, the solution below will not help (check ONTAP's SnapMirror logs on the destination)
  • SnapVault or SnapMirror update takes longer than (by default) two hours to complete, and the snapshot will appear later:
    • After triggering the SnapVault or SnapMirror update, the plugin will check and wait repeatedly for the transfer to end and the snapshot to appear on the destinatio
    •  The check is attempted 120 times with pauses of 1 minute before giving up and displaying the error message.

Solution

  • Update or add the below configuration parameters by either RestAPI or manually:

"SNAPSHOT_CHECK_RETRY": "120",
"SNAPSHOT_CHECK_TIMEOUT": "60000",
"SnapshotDiscoverRetryIntervalSecond": "10000",
"SnapshotDiscoverMaxRetryExist": "9",

"SnapmirrorRetry": "360",
"SnapmirrorTimeout": "300000",
"MaxRetrySnapmirrorHistory": "300",
"RetryIntervalSecondSnapmirrorHistory": "4",

  • SCV 4.5 and later: Take a backup of the file:appsettings-storage.json before you make any changes

  1. From vSphere Web Interface, open the Console
  2. Login with maint user
  3. From Main Menu, press  4
  4. From Support and Diagnostic Menu, press 2
  5. Press 'y'  to enter diagnostic shell
  6. Type 'sudo su'
  7. Edit the appsettings-storage.json file by typing  vi /opt/netapp/protectionservice/publish/storage/appsettings-storage.json
  8. With SCV 4.6 and later, to use the entries add the following in the AppSettings of the appsettings-storage.json at the top:"UseAppSettings": “true”,
  9. Add following keys inside the AppSettings section (timeouts are in milliseconds)

"SNAPSHOT_CHECK_RETRY": "120",

"SNAPSHOT_CHECK_TIMEOUT": "60000",

"SnapshotDiscoverRetryIntervalSecond": "10000",

"SnapshotDiscoverMaxRetryExist": "9",

"SnapmirrorRetry": "360",

 "SnapmirrorTimeout": "300000",

"MaxRetrySnapmirrorHistory": "300",

"RetryIntervalSecondSnapmirrorHistory": "4",  

  1. Shown here are the defaults that are not normally visible, you may need to adjust the values from the shown defaults to match the duration of longest normal SnapVault or SnapMirror transfer. Instead of increasing just the retries, change the timeout to 10 or 15 minutes (600000, 900000 resp.) and adjust the retries accordingly. This saves on the number of checks and logging (CPU usage).

10. Save the changes and exit the vi editor (press Escape from editing and enter : , w , and q on the keyboard)

11. Exit the diagnostic shell by typing 'Exit' at the command prompt

12. Type 'b' to go back to the Main Menu

13. From Main Menu, press 1

14. From the Application Configuration Menu, press 3 to stop the Services

15. From the Application Configuration Menu, press 2 to start the Services

Note:

SNAPSHOT_CHECK_RETRY defines the maximum number of retries that are performed to discover the Snapshot copies on the secondary location.

SNAPSHOT_CHECK_TIMEOUT defines the wait or sleep period for every retry.

The above default is to wait for one minute before checking again, the total is two hours by default.

Additional Information

Parent topic: Resolution guide for SnapMirror/SnapVault issues in SnapCenter

Note: The SNAPSHOT_CHECK_ variables are meant for the destination Snapshot appearance and the SnapshotDiscovery* for primary Snapshots

 

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

Scan to view the article on your device