XCP sync performance degrades over time when lots of files deleted on source
Applies to
- XCP 1.5
- XCP 1.6
- XCP 1.6P1
Issue
- When a large number (Million) of files were deleted from the source volume/qtree after initial copy the XCP sync took lots of hours to complete.
- Some of the errors seen in the XCP logs:
XCP logs :
Reopener (Reopener) (47012 isChild False) 'lnn63f1 tcp 2049 nfs3 c0' WARNING: reconnected in 0.007s after 1 try during 5s
xcp: 17,700 unknown, 17,706 retries, 5.40M reviewed, 61,770 checked at source, 5.42M gone, 1,532 dir.gone, 5.42M file.gone, 10 modifications, 2.93M removed, 61,760 reindexed, 1.99M errors, 1.21 GiB in (0/s), 2.98 GiB out (0/s), 5d11h
Reopener (Reopener) (47012 isChild False) 'lnn63f1 tcp 2049 nfs3 c0': (47012): new Reopener for lnn63f1 tcp 2049 nfs3 c0 is Child False (requests 1)
Reopener (Reopener) (47012 isChild False) 'lnn63f1 tcp 2049 nfs3 c0' WARNING: reconnecting to resend 1 pending request to lnn63f1 tcp 2049 nfs3 c0 (isChild False)
Reopener (Reopener) (47012 isChild False) 'lnn63f1 tcp 2049 nfs3 c0': resending nfs3 READ
removeT1 WARNING: error removing 'filer1.netapp.com:/vol1/tree1/slm.dir/fil12856.file' (sync batch): nfs3 REMOVE 'fil12856.file' in 'filer1.netapp.com:/vol1/tree1/slm.dir': nfs3 error 2: no such file or directory
removeT1 WARNING: error removing 'filer1.netapp.com:/vol1/tree1/slm2.dir/fil17268.file' (sync batch): nfs3 REMOVE 'fil17268.file' in 'filer1.netapp.com:/vol1/tree1/slm2.dir': nfs3 error 2: no such file or directory