StorageGRID client observes slow GET response time and load balancer 499 timeout
Applies to
- StorageGRID 11.6.0.12 or earlier
- StorageGRID 11.7.0.5 or earlier
- History of ADC Storage Node decommissioning
Issue
- Client reports very slow GET response times
audit-sum
script can be used on/var/local/audit/export
to summarize recent activity
/var/local/audit/export # audit-sum audit.log
message group count min(sec) max(sec) average(sec)
============= ===== ======== ======== ============
SDEL 13355085 0.005 64.642 9.890
SGET 1763885 0.001 34346.196 19.277
SHEA 1077045 0.000 1.252 0.005
SPUT 4812323 0.003 50.772 0.029
Note: The audit-sum
queries has possibility to impact StorageGRID operations as described in Use audit-sum tool
Diagnostic
report by Run diagnostics shows large amounts of 499 timeouts forLoad balancer - request timeout