StorageGRID S3 uploads fail intermittently with Internal server 500
Applies to
- StorageGRID 11.5
- FileNet Application
Issue
When using FileNet Application S3 operations to the GRID failed for some of the objects :
2023-02-02 02:48:00,259 com.<xxxx> ERROR - DMS11132I: <xxxx> Document Import robot failed to commit batch:<xxxx>
2023-02-02 02:48:00,264 <xxxx> ERROR - <xxxx>: FNRCC0243E: CONTENT_ASA_NOT_ENOUGH_REPLICAS_WRITTEN: Not enough replicas were written to satisfy the synchronous upload requirement, written 0, required 1. Area Id: [{<xxxx>}]; Device Id: [{<xxxx>}]; Content Id: [{<xxxx>}]; HashSeq: [<xxxx>]; Successful replicas: []; Failed replicas: [{<xxxx>} : <xxxx>: FNRCC0317E: CONTENT_S3_OPERATION_FAILED: S3 storage device [CreateContent] call failed for Object: <xxxx>] errorStack={at <xxxx>(Redirector.java:392)
StorageGRID bycast-err.log reports:
Jan 30 08:06:57 <xxxx> ADE: |12468221 0472863229 S3RQ POPD 2023-01-30T08:06:57.108038| ERROR 1880 <xxxx> S3RQ: S3 request error: connection=<xxxx>, Resource=<xxxx>, HTTP Status Code=500, ErrorMsg=InternalError, ErrorType=Client CustomErrorMessage={<none>}, Details={asyncPop error: no producer}