Skip to main content
NetApp Knowledge Base

System does not start after reboot with Unable to recover the local database of Data Replication Module

  1. Last updated
  2. Save as PDF
Views:
19,568
Visibility:
Public
Votes:
8
Category:
fas-systems
Specialty:
hw
Last Updated:

Applies to

  • All AFF and FAS platforms
  • ONTAP 9
  • ONTAP Select
  • Power outage, controller relocation, or during upgrade or other reboot
  • Improperly unfail a disk before fixing the underlying virtual disk file in ONTAP Select product

Issue

  • Console logs for node show:
******************************************************
* This is a serial console session. Output from this *
* session is mirrored on the SP console session.     *
******************************************************
***********************
**  SYSTEM MESSAGES  **
***********************
Internal error: Cannot open corrupt replicated database. Automatic recovery
attempt has failed or is disabled. Check the event logs for details. This node
is not fully operational. Contact support personnel for the root volume recovery
procedures.
  • Node may panic during boot:

Warning: previous shutdown was dirty, there is a possible loss of data.
May 28 00:43:33 [node1:wafl.root.content.changed:error]: Contents of the root volume 'vol0' might have changed. Verify that all recent configuration changes are still in effect.
PANIC  : NVRAM contents are invalid...
PANIC: NVRAM contents are invalid...   in SK process rc on release 9.10.1P5 (C) on Wed May 28 00:43:33 GMT 2025

  • EMS messages for node:
Mar 28 17:03:46 [NODE_B:rdb.recovery.failed:EMERGENCY]: Error: Unable to find a master. Unable to recover the local database of Data Replication Module: Management.
Mar 28 17:03:46 [NODE_B:spm.mgwd.process.exit:EMERGENCY]: Management Gateway (mgwd) subsystem with ID 1944 exited as a result of signal normal exit (0). The subsystem will attempt to restart.
  • Node boots and it is possible to log in but the cluster commands are not showing the correct outputs:
::> cluster show
Error: "show" is not a recognized command
 
::> set advanced
::*> cluster ring show
Error: "show" is not a recognized command
  • ROOT VOLUME NOT WORKING PROPERLY: RECOVERY REQUIRED error message is displayed during the boot/login phase.
  • Some bootarg values are stated for corrupt and/or recovery in the LOADER.

Example:

LOADER-B> printenv bootarg.rdb_corrupt
Variable Name        Value
-------------------- --------------------------------------------------
bootarg.rdb_corrupt  0500000000
 
LOADER-B> printenv bootarg.init.boot_recovery
Variable Name        Value
-------------------- --------------------------------------------------
bootarg.init.boot_recovery 80
 
Note1:  If bootarg is not set, it is displayed as undefined.
 
Note2: The same value(s) also can be checked via AutoSupport from KENV section.
 
Example:
 
bootarg.rdb_corrupt="5550055000"

 

 

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.