S3 job is failing to read the data from a bucket with "HTTP Status Code=500 ErrorMsg=InternalError"
Applies to
- NetApp StorageGRID
- External Load Balancer / Gateway (NGINX)
- Spark Jobs
Issue
- Spark job on the client end is failing to read the objects from the StorageGRID bucket with "
HTTP Status Code=500 ErrorMsg=InternalError
". - The error message can be found in
bycast.log
in the log bundle or under/var/local/log
directory. The failure detail{
indicates the client connection was lost while the StorageGRID node was returning the data to the client.asyncPush error: no consumer
}
Sep 19 09:11:38 <NODE> ADE: |12561386 2430411499 S3RQ PSHD 2022-09-19T09:11:38.901474| ERROR 1880 94aa61294d2c0aae S3RQ: S3 request error (after response): connection=1663578683183367, Resource=/bucket/xxxx/optimized/production/lte/1234567890.snappy, HTTP Status Code=500, ErrorMsg=InternalError, ErrorType=Client CustomErrorMessage={<none>}, Details={asyncPush error: no consumer}