mgmt cluster app flapping on all nodes during ONTAP upgrade
Applies to
- ONTAP 9
- Automated Non-Disruptive Upgrade (ANDU)
Issue
- ONTAP upgrade pauses on error after first node boots on higher version
- Cluster app
mgmt
is switching between online and offline on both nodes every few seconds. - MGWD on upgraded node is spammed with:
ERR: rdb_glue: modify_imp:src/rdb/rdb_iterator.cc:1796 database 'audit_rdb' is not open.
- MGWD on non-upgraded node reports:
Sun Jan 1 2003 12:00:00 +00:00 [kern_mgwd:info:0001] 0x820edb400: SQL error: "INSERT INTO smf_schema_replicated(rowid, _epoch, _tid, [table-name], [field-pid], [sql-type], [sql-type-map-version], [smf-type], [rdb-type], [field-name], [is-list], [persistence-type]) VALUES (-1549789249282575363, 64, 2, 'userprofile_rdb', 23, 'INTEGER', 3, 'yes_no', 5, 'is-ldap-fastbind', 0, 2);" UNIQUE constraint failed: smf_schema_replicated.table-name, smf_schema_replicated.field-pid(19)
Sun Jan 1 2003 12:00:00 +00:00 [kern_mgwd:info:0001] 0x820edb400: 0: ERR: SQL_CONTEXT: execute_sql:src/sql_context.cc:834 SQL: failed on connection 0x83164d208: UNIQUE constraint failed: smf_schema_replicated.table-name, smf_schema_replicated.field-pid(19), txn: 'saveTxnChanges:smf_schema::save_schema',active_connection: 0x83164d208, active_thread: 0x820edb400, active_label: 'saveTxnChanges:smf_schema::save_schema', stmt: "INSERT INTO smf_schema_replicated(rowid, _epoch, _tid, [table-name], [field-pid], [sql-type], [sql-type-map-version], [smf-type], [rdb-type], [field-name], [is-list], [persistence-type]) VALUES (-1549789249282575363, 64, 2, 'userprofile_rdb', 23, 'INTEGER', 3, 'yes_no', 5, 'is-ldap-fastbind', 0, 2);"
Sun Jan 1 2003 12:00:00 +00:00 [kern_mgwd:info:0001] E [src/rdb/sql_local_unit.cc 5208 (0x820edb400)]: saveTxnChanges: failed to execute SQL: 'INSERT INTO smf_schema_replicated(rowid, _epoch, _tid, [table-name], [field-pid], [sql-type], [sql-type-map-version], [smf-type], [rdb-type], [field-name], [is-list], [persistence-type]) VALUES (-1549789249282575363, 64, 2, 'userprofile_rdb', 23, 'INTEGER', 3, 'yes_no', 5, 'is-ldap-fastbind', 0, 2);'.
Sun Jan 1 2003 12:00:00 +00:00 [kern_mgwd:info:0001] W [src/rdb/sql_local_unit.cc 5420 (0x820edb400)]: saveTxnChanges: abandoning due to INTERNAL_ERROR.