Client application reports intermittent ingest issue with StorageGRID 11.5.0.3 or higher
Applies to
- NetApp StorageGRID 11.5.0.3 or higher
- Bucket consistency set to Available
Issue
- After applying hotfix 11.5.0.3 or higher, StorageGRID might report a sharp increase in both
404 HTTP
errors and503 HTTP
errors, which can cause intermittent ingest failures in Client application. - Example: FabricPool performs a GET immediately after a PUT in its ingest workflow. The following error in the
bycast.log
corresponds to each503
error.
Oct 21 14:34:04 dc1h1 ADE: |12597648 2463375992 S3RQ %CEA 2021-10-21T14:34:04.044625| NOTICE 1354 401c06c1dfbc708d S3RQ: EVENT_PROCESS_CREATE - connection=1634825219859407 method=GET bytes=0-31 name=</dc1/ab41a77f-7e7f-4e48-b917-effc3f7c1338/06137e93_0000000000c9afe7_ab41a77f-7e7f-4e48-b917-effc3f7c1338> auth=<V4> clientIP=<10.0.0.1>
Oct 21 14:34:04 dc1h1 ADE: |12597648 2463375996 OBDR CSRT 2021-10-21T14:34:04.065348| NOTICE 0419 401c06c1dfbc708d OBDR: Metadata request failed for unknown 'dc1/ab41a77f-7e7f-4e48-b917-effc3f7c1338/06137e93_0000000000c9afe7_ab41a77f-7e7f-4e48-b917-effc3f7c1338', object temporarily unavailable as it may have been modified
Oct 21 14:34:04 dc1h1 ADE: |12597648 2463375992 S3RQ %DED 2021-10-21T14:34:04.065541| NOTICE 0090 401c06c1dfbc708d S3RQ: S3 error response: RequestId=1634825219859407, TraceId=401c06c1dfbc708d, Resource=/dc1/ab41a77f-7e7f-4e48-b917-effc3f7c1338/06137e93_0000000000c9afe7_ab41a77f-7e7f-4e48-b917-effc3f7c1338, HTTP Status Code=503, X-Forwarded-For: 10.0.0.1, ErrorMsg=ServiceUnavailable, ErrorType=Client, CustomErrorMessage={<none>}, Details={<none>