Multiple 504 errors due to high delete workload

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Views:: 199

Visibility:: Public

Votes:: 0

Category:: storagegrid

Specialty:: sgrid

Last Updated:

Applies to

StorageGRID 11.3
HAproxy

Issue

HAproxy times out before StorageGRID does, resulting in 504 errors.

{"host":"fully.qualified.host.name","ident":"haproxy","pid":12345,"time":"5/May/2021:18:24:20+0100","haproxy":{"conn":{"act":250,"fe":249,"be":248,"srv":25},"queue":{"backend":0,"srv":0},"time":{"tq":0,"tw":0,"tc":0,"tr":-1,"tt":50003},"termination_state":"sH--","retries":0,"network":{"client_ip":"w.x.y.z","client_port":12345,"frontend_ip":"a.b.c.d","frontend_port":443},"ssl":{"version":"-","ciphers":"-"},"request":{"method":"PUT","uri":"\/bucket\/key","protocol":"HTTP/1.1","header":{"host":"-","xforwardfor":"-","referer":"-"}},"name":{"backend":"grid-be","frontend":"grid-fe","server":"haproxy_1"},"response":{"status_code":504,"header":{"xrequestid":"-"}},"bytes":{"uploaded":49564,"read":262}}

Note that the 504 Error originates from HA proxy, 504 is not a valid S3 error response.