404 errors seen in Load Balancer error rate graph
Applies to
- StorageGRID 11.8 and above
- Commvault
Issue
- 404 Errors reported in the Load Balancer Error rate graph.

- 404 errors seen against HEAD requests in
endpoint-access.log
:
[2025-04-02T17:09:58+00:00] 4.33.234.9 "HEAD /bucket/8PQ1E2_04.12.2022_18.52/CV_MAGNETIC/V_5122760/CHUNK_429554362/SFILE_CONTAINER_012.compact2.SIZE HTTP/1.1" 404 213 565 0.004 "256f1bd5-cc29-4b0e-a86d-ef5ba88402b2" "IP:18086" "FQDN:4443"
- HEAD request is performed and immediately afterwards you can see the metadata request failure for unknown reason in the
bycast.log
.
Apr 2 17:09:58 node ADE: |12826914 3151662980 S3RQ %CEA 2025-04-02T17:09:58.472826| NOTICE 1255 3cf4315496c151bf S3RQ: EVENT_PROCESS_CREATE - connection=1743613595419306 method=HEAD name=</bucket/8PQ1E2_04.12.2022_18.52/CV_MAGNETIC/V_5122760/CHUNK_429554362/SFILE_CONTAINER_012.compact2> auth=<V4> clientIP=<IP>
Apr 2 17:09:58 node ADE: |12826914 3151662981 OBDR ^RDY 2025-04-02T17:09:58.476433| NOTICE 0404 3cf4315496c151bf OBDR: Metadata request failed for unknown '/bucket/8PQ1E2_04.12.2022_18.52/CV_MAGNETIC/V_5122760/CHUNK_429554362/SFILE_CONTAINER_012.compact2'
- Two HEAD requests followed by the PUT.
Apr 2 17:09:58 node ADE: |12826914 3151662977 S3RQ %CEA 2025-04-02T17:09:58.469387| NOTICE 1255 968a82837eb53977 S3RQ: EVENT_PROCESS_CREATE - connection=1743613595419306 method=HEAD name=</bucket/8PQ1E2_04.12.2022_18.52/CV_MAGNETIC/V_5122760/CHUNK_429554362/SFILE_CONTAINER_012.compact2.SIZE> auth=<V4> clientIP=<IP>
Apr 2 17:09:58 node ADE: |12826914 3151662980 S3RQ %CEA 2025-04-02T17:09:58.472826| NOTICE 1255 3cf4315496c151bf S3RQ: EVENT_PROCESS_CREATE - connection=1743613595419306 method=HEAD name=</bucket/8PQ1E2_04.12.2022_18.52/CV_MAGNETIC/V_5122760/CHUNK_429554362/SFILE_CONTAINER_012.compact2> auth=<V4> clientIP=<IP>
Apr 2 17:10:18 node ADE: |12826914 3151669891 S3RQ %CEA 2025-04-02T17:10:18.585100| NOTICE 1255 af2f1401955e23ad S3RQ: EVENT_PROCESS_CREATE - connection=1743613595419306 method=PUT name=</bucket/8PQ1E2_04.12.2022_18.52/CV_MAGNETIC/V_5122760/CHUNK_429554362/SFILE_CONTAINER_012.compact2> auth=<V4> clientIP=<IP>