Skip to main content
NetApp Knowledge Base

EC rebalance job failing with "Job needs retry"

Views:
29
Visibility:
Public
Votes:
0
Category:
storagegrid-webscale
Specialty:
sgrid
Last Updated:

Applies to

NetApp StorageGRID  11.7

Issue

When running EC balance job it fails without error. The bycast.log of the EC leader you see the following :
 
ba9de970532e9 ECJM: 16605481304316448032(rebalance 20): saving state. status: JOBSTATUS_PAUSED
Sep 27 08:34:06 FBRPSSTO004 ADE: |21977923 1592539400 ECJM ^RDY 2023-09-27T08:34:06.557857| WARNING  0067 a5bba9de970532e9 ECJM: Caught exception 'ENFORCE failed: !"Job needs retry."' when running job 16605481304316448032: Site Rebalance - Group ID 20.
Sep 27 08:34:06 FBRPSSTO004 ADE: |21977923 1592539400 ECJM ^RDY 2023-09-27T08:34:06.557933| ERROR    1083 a5bba9de970532e9 PROC: Exception: /build/src/modules/ErasureCoding/EC_JobManager_Module/SiteRebalanceJob.cc(410): Throw in function CXD_AtomContainer erasurecoding::SiteRebalanceJob::doRebalance()#012Dynamic exception type: boost::wrapexcept<std::runtime_error>#012std::exception::what: ENFORCE failed: !"Job needs retry."#012
Sep 27 08:34:06 FBRPSSTO004 ADE: |21977923 1568886980 ECJM #DON 2023-09-27T08:34:06.557935| NOTICE   0959 0fc4f7218174f516 ECJM: Received job completion message.
Sep 27 08:34:06 FBRPSSTO004 ADE: |21977923 1568886980 ECJM #DON 2023-09-27T08:34:06.557949| NOTICE   0965 0fc4f7218174f516 ECJM: Job 16605481304316448032 completed with result GERR.
Sep 27 08:34:06 FBRPSSTO004 ADE: |21977923 1568886980 ECJM #DON 2023-09-27T08:34:06.557956| NOTICE   1229 0fc4f7218174f516 ATOM: #DON:CONT:
Sep 27 08:34:06 FBRPSSTO004 ADE: |21977923 1568886980 ECJM #DON 2023-09-27T08:34:06.557965| NOTICE   1205 0fc4f7218174f516 ATOM:      RSLT:FC32:'GERR'
Sep 27 08:34:06 FBRPSSTO004 ADE: |21977923 1568886980 ECJM #DON 2023-09-27T08:34:06.557972| NOTICE   1205 0fc4f7218174f516 ATOM:      JBID:UI64:16605481304316448032
Sep 27 08:34:06 FBRPSSTO004 ADE: |21977923 1568886980 ECJM #DON 2023-09-27T08:34:06.557976| NOTICE   1233 0fc4f7218174f516 ATOM: END
Sep 27 08:34:06 FBRPSSTO004 ADE: |12320679 2877050429 S3RQ %CEA 2023-09-27T08:34:06.587048| NOTICE   1418 9e04d1c712a4942c S3RQ: EVENT_PROCESS_CREATE - con

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.