503 Service Unavailable error on COSBench during performance test to StorageGRID
Applies to
- StorageGRID
- COSBench
Issue
- COSBench reports
503 Service Unavailable
error during performance test against StorageGRID.
com.intel.cosbench.api.storage.StorageException: HTTP/1.1 503 Service Unavailable
at com.intel.cosbench.api.swift.SwiftStorage.createObject(SwiftStorage.java:226)
at com.intel.cosbench.driver.operator.Writer.doWrite(Writer.java:98)
at com.intel.cosbench.driver.operator.Writer.operate(Writer.java:79)
at com.intel.cosbench.driver.operator.AbstractOperator.operate(AbstractOperator.java:76)
at com.intel.cosbench.driver.agent.WorkAgent.performOperation(WorkAgent.java:197)
at com.intel.cosbench.driver.agent.WorkAgent.doWork(WorkAgent.java:177)
at com.intel.cosbench.driver.agent.WorkAgent.execute(WorkAgent.java:134)
at com.intel.cosbench.driver.agent.AbstractAgent.call(AbstractAgent.java:44)
at com.intel.cosbench.driver.agent.AbstractAgent.call(AbstractAgent.java:1)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: com.intel.cosbench.client.swift.SwiftException: unexpected return from server
at com.intel.cosbench.client.swift.SwiftClient.storeStreamedObject(SwiftClient.java:261)
at com.intel.cosbench.api.swift.SwiftStorage.createObject(SwiftStorage.java:217)
... 12 more
Bycast.log
shows error:
NOTICE 0040 7e42fcdb85c56f58 SWRQ: Swift error response: RequestId=1628154714382298, Resource=/v1/AUTH_23cea2664e424964b36465eca3fffbc2/xx/xx, ErrorCode=ServiceUnavailable, Details={Timestamp of object being replaced is later than current server time (time has gone backwards). Check NTP and check for occurrences of VMotion during this request.}