StorageGRID decommission paused at erasure coded data due to read-only volume
Applies to
StorageGRID
Issue
- StorageGRID node decommission pauses at Decommissioning Erasure Coding Data.
- After resuming erasure coding decommission, the job pauses again.
- Bycast logs on EC Leader show following EC Job Manager exception:
Caught exception 'ENFORCE failed: 'SUCS' == jobResult.value()' when running job <job ID>: Storage Node Decommissioning
- Bycast logs on Source node shows VCSCopy terminating (repeatedly):
VCPY: VCS not making progress
VCPY: Retry limit reached in VCS copy <VCS ID> to <Node ID>: Terminating VCSCopy
VCPY: Sending VCS copy response to process {<ID>} for VCS <VCS ID>, result: GERR.
- Bycast logs on target node show volume read only while trying to copy chunks into the VCS:
WARNING: PUT <ID> - failed with 503: volume read only
VCS notification (EVFL) sent to service id ECLD on node <Node ID> for VCSId <VCS ID>