SCV fails to restore VMs by "Single File Snapshot Restore Failed" and "Volume is full or Volume crossed autodelete threshold"

Applies to

•  SnapCenter Plug-in for VMware vSphere (SCV) 4.2
• NetApp Data Broker 1.0
• ONTAP 9.3

Issue

•  SCV fails to restore VMs with exception "Single File Snapshot Restore Failed" and "Volume is full or Volume crossed autodelete threshold".

• Message "Single File Snapshot Restore Failed" is returned as a result of SmRestoreFileResponse in vmcontrol_<JOB_ID>.log.

DEBUG comnr004 --- [p32374789-61341] c.n.a.s.c.SnapManagerRestoreServiceImpl  : <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<SmRestoreFileResponse>
    <Job>
        <Id>0</Id>
        <StartTime xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
        <EndTime xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
        <IsCancellable>false</IsCancellable>
        <IsRestartable>false</IsRestartable>
        <IsCompleted>false</IsCompleted>
        <IsVisible>true</IsVisible>
        <IsScheduled>false</IsScheduled>
        <PercentageCompleted>0</PercentageCompleted>
        <Status>Running</Status>
        <Priority>None</Priority>
        <Tasks/>
        <ParentJobID>0</ParentJobID>
        <EventId>0</EventId>
        <JobTypeId xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
        <ObjectId>0</ObjectId>
        <PluginCode>NONE</PluginCode>
        <HostId>0</HostId>
        <RoleId xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
    </Job>
    <Result>
        <_message>Single File Snapshot Restore Failed:  Failed to perform Single File SnapRestore.
Failed to restore file from Snapshot copy <SNAPSHOT_NAME> for volume "<VOLUME_NAME>" on Vserver "<VSERVER_NAME>". Reason: Volume is full or Volume crossed autodelete threshold. 
.  Failed to perform Single File SnapRestore.
Failed to restore file from Snapshot copy <SNAPSHOT_NAME> for volume "<VOLUME_NAME>" on Vserver "<VSERVER_NAME>". Reason: Volume is full or Volume crossed autodelete threshold. 
. </_message>
        <_errorCode>-1</_errorCode>
        <ErrorRecords/>
    </Result>
    <TotalCount>0</TotalCount>
    <DisplayCount>0</DisplayCount>
</SmRestoreFileResponse>

ERROR comnr004 --- [p32374789-61341] c.n.a.s.c.SnapManagerRestoreServiceImpl  : [JobId:<JOB_ID>] Restore failed with exception: Single File Snapshot Restore Failed:  Failed to perform Single File SnapRestore.
Failed to restore file from Snapshot copy <SNAPSHOT_NAME> for volume "<VOLUME_NAME>" on Vserver "<VSERVER_NAME>". Reason: Volume is full or Volume crossed autodelete threshold. 
.  Failed to perform Single File SnapRestore.
Failed to restore file from Snapshot copy <SNAPSHOT_NAME> for volume "<VOLUME_NAME>" on Vserver "<VSERVER_NAME>". Reason: Volume is full or Volume crossed autodelete threshold. 
. 
 

• The following error is shown in wfe_<JOB_ID>.log.

ERROR comnr004 --- [pool-3-thread-1] c.n.w.r.SCVRestoreTasksServiceImpl       : Exception while calling restore on SCV: Restore failed with exception: Single File Snapshot Restore Failed:  Failed to perform Single File SnapRestore.
Failed to restore file from Snapshot copy <SNAPSHOT_NAME> for volume "<VOLUME_NAME>" on Vserver "<VSERVER_NAME>". Reason: Volume is full or Volume crossed autodelete threshold. 
.  Failed to perform Single File SnapRestore.
Failed to restore file from Snapshot copy <SNAPSHOT_NAME> for volume "<VOLUME_NAME>" on Vserver "<VSERVER_NAME>". Reason: Volume is full or Volume crossed autodelete threshold. 
 

• The following error is returned as a result of snapshot-restore-file ZAPI in ONTAP /mroot/etc/log/mlog/audit.log.

<netapp version='1.0' xmlns='http://www.netapp.com/filer/admin'><snapshot-restore-file>   <path><FILE_PATH></path>   <snapshot><SNAPSHOT_NAME></snapshot>   <volume><VOLUME_NAME></volume> </snapshot-restore-file></netapp>  :: Pending
Failed to restore file from <SNAPSHOT_NAME> for volume "<VOLUME_NAME>" on Vserver "<VSERVER_NAME>". Reason: Volume is full or Volume crossed autodelete threshold.  :: ONTAPI :: Error

• EMS event wafl.sis.clone.create.failed is detected from ONTAP side.