Cloud sync failed with multiple errors when copying files from NFS to S3
Applies to
- Cloud sync
- AWS S3
Issue
- Cloud sync fails to copy multiple files when performing copy from NFS to S3.
- Data broker reports multiple errors like
"TimeoutError: socket hang up" , "Error: write EPIPE", "RequestTimeout:"
[WARN] [TRANSFERRER] [1361] [32677] [64137xxxxxxxxxx] RETRYING copy from: {"local":{"path":"mnt/64137xxxxxxxxx/test_src/test1/folder1"}} to: {"s3":{"bucket":"test-input","tags":[],"provider":"s3","storageClass":"STANDARD","region":"eu-xx-1","path":"test1/folder1"}} entry: {"path":"test0_test2","type":"FILE","size":1165827801088,"mode":33184,"uid":207,"gid":205,"mtimeSec":1677791421,"creationTimeSec":1677791421,"inode":955xxxx} TimeoutError: socket hang up
[WARN] [TRANSFERRER] [1361] [32677] [64137xxxxxxxxx] RETRYING copy from: {"local":{"path":"mnt/64137xxxxxxxxx/test_src/test1/folder1"}} to: {"s3":{"bucket":"test-input","tags":[],"provider":"s3","storageClass":"STANDARD","region":"eu-xx-1","path":"test1/folder1"}} entry: {"path":"test0_test2","type":"FILE","size":1221348679680,"mode":33184,"uid":207,"gid":205,"mtimeSec":1677792320,"creationTimeSec":1677792320,"inode":955xxxxx} Error: write EPIPE
[WARN] [TRANSFERRER] [1362] [1968] [64137xxxxxxxxx] RETRYING copy from: {"local":{"path":"mnt/64137xxxxxxxxx/test_src/test1/folder1"}} to: {"s3":{"bucket":"test-input","tags":[],"provider":"s3","storageClass":"STANDARD","region":"eu-xx-1","path":"test1/folder1"}} entry: {"path":"test0_test2","type":"FILE","size":1181281681408,"mode":33184,"uid":207,"gid":205,"mtimeSec":1677792475,"creationTimeSec":1677792475,"inode":955xxxxx} RequestTimeout: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.