Setting object versioning on a S3 bucket fails by HTTP 500

Applies to

• StorageGRID
• S3
• One or more storage nodes are unavailable

Issue

• Setting object versioning on a S3 bucket fails by HTTP 500 Internal Server error.
• bycast.log shows Cassandra UnavailableExceptin with "Cannot achieve consistency level ALL" by PUT request for versioning.

Example:

NOTICE   1154 S3RQ: EVENT_PROCESS_CREATE - connection=1615515505231245 method=PUT namespace=<> path=</> name=</<S3 BUCKET>?versioning> auth=<V4> clientIP=<IP>
ERROR    0236 S3RQ:  Cassandra UnavailableException (Failed to execute cql: UPDATE storagegrid.buckets SET versioning = 1 WHERE bucket = '<S3 BUCKET>' - Cassandra Driver Error(Unavailable):'Cannot achieve consistency level ALL')
NOTICE   0137 S3RQ: S3 error response: RequestId=1615515505231245, Resource=/<S3 BUCKET>?versioning, HTTP Status Code=500, X-Forwarded-For: '<IP>', ErrorMsg=InternalError, ErrorType=Client, CustomErrorMessage={<none>}, Details={Failed to execute cql: UPDATE storagegrid.buckets SET versioning = 1 WHERE bucket = '<S3 BUCKET>' - Cassandra Driver Error(Unavailable):'Cannot achieve consistency level ALL'}