Intermittent S3 multipart upload operations fail
Applies to
- ONTAP 9
- S3
- Multipart Upload Operations
- Quality Of Service (QOS)
- Flexgroup
- Performance
Issue
- Intermittent S3 multipart upload operations fail with client-side errors such as:
Put "https://10.1.1.251/zone2/zoneinformation/data": context deadline exceeded
- Additional symptoms:
- S3 clients see timeouts when writing to storage
- Debugged
SKTRACE.logshows the connection is closed by the by the client after waiting too long:
- Debugged
- S3 clients see timeouts when writing to storage
S3_CMD_Err: n0x1556a3: [1] Connection closed by client: Remote 10.1.1.100:44144 <-> Local 10.1.1.251:443
-
- Most multipart PUTs succeed
- QOS is enabled on the storage
- High QoS throttling and latency observed in ONTAP performance archives during workload bursts (up to 1.1GB/s & 20 IOPS)
- The FlexGroup volume is configured with only one constituent
