Experiencing slowness when accessing StorageGRID Objects
Applies to
- StorageGRID 11.x
- StorageGRID deployment with an external load balancer.
- NSX F5 load balancer
Issue
- Observing latency accessing the objects from the StorageGRID.
- There are no alarms related to node down / slow performance reported on the StorageGRID.
- Audit logs show the object accessed is responding quickly.
root@PrimaryAdminNode $ tail /var/local/log/export/audit.log | grep 'TIME\|SGET\|SPUT'
202x-0x-11T21:05:46.260002 [AUDT:[RSLT(FC32):SUCS][CNID(UI64):1649711110894001][TIME(UI64):81600][SAIP(IPAD):"10.2xx.11.xx6"][HTRH(CSTR):"{}"][S3AI(CSTR):"05888264359773466568"][SACC(CSTR):"ahmxx"][S3AK(CSTR):"SGKHWjnEyvucUR-NT0wMLd8miVQbZTinNmQcBxaNIQ=="][SUSR(CSTR):"urn:sgws:identity::05888264359773466568:root"][SBAI(CSTR):"05888264359773466568"][SBAC(CSTR):"ahmxx"][S3BK(CSTR):"objectlockno"][S3KY(CSTR):"test/A4863.tmp"][CBID(UI64):0xA93681444A89572E][UUID(CSTR):"8DBEB7EA-EE05-4389-ADB3-C87F6C562320"][CSIZ(UI64):65536][MTME(UI64):1649711146178617][AVER(UI32):10][ATIM(UI64):1649711146260002][ATYP(FC32):SPUT][ANID(UI32):12046747][AMID(FC32):S3RQ][ATID(UI64):15176828704627503237]]
202x-0x-11T21:05:46.385211 [AUDT:[RSLT(FC32):SUCS][CNID(UI64):1649711112216796][TIME(UI64):80809][SAIP(IPAD):"10.2xx.11.xx6"][HTRH(CSTR):"{}"][S3AI(CSTR):"05888264359773466568"][SACC(CSTR):"ahmxx"][S3AK(CSTR):"SGKHWjnEyvucUR-NT0wMLd8miVQbZTinNmQcBxaNIQ=="][SUSR(CSTR):"urn:sgws:identity::05888264359773466568:root"][SBAI(CSTR):"05888264359773466568"][SBAC(CSTR):"ahmxx"][S3BK(CSTR):"objectlockno"][S3KY(CSTR):"test/A4864.tmp"][CBID(UI64):0xBE2FC422E0C0CCD2][UUID(CSTR):"630CA0F3-AB63-4F80-AE3A-6236F7AA6628"][CSIZ(UI64):65536][MTME(UI64):1649711146304581][AVER(UI32):10][ATIM(UI64):1649711146385211][ATYP(FC32):SPUT][ANID(UI32):12046747][AMID(FC32):S3RQ][ATID(UI64):2877178465283031656]]