AIQUM MySQL database fails to startup due to corruption after unexpected shutdown during install/upgrade
Applies to
- Active IQ Unified Manager (AIQUM)9.x
- Windows platform
Issue
- AIQUM host Windows was unexpectedly rebooted during install/upgrade
- MySQL service crashes or doesn't start at all
- HTTP
404 - Not Foundis returned when accessing AIQUM GUI au.logindicatesServer returned HTTP status 404
ERROR [WrapperStartStopAppMain] c.o.s.a.f.m.CommunicationManager (CommunicationManager.java:555) - Server returned HTTP status 404.
ERROR [WrapperStartStopAppMain] c.o.s.a.f.m.CommunicationManager (CommunicationManager.java:561) - - Failed to get Summary SessionServer returned HTTP status 404. java.lang.RuntimeException: Server returned HTTP status 404.
at com.onaro.sanscreen.acquisition.framework.mgmt.CommunicationManager.getSummarySessionFromREST(CommunicationManager.java:556) ~[au-framework.jar:9.13.0-2023.06.J287]
jboss/server.logindicates connection to port 3306 fails
WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (pool-13-thread-1) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: IJ031084: Unable to create connection
at org.jboss.ironjacamar.jdbcadapters@1.5.3.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:364)
Caused by: java.sql.SQLNonTransientConnectionException: Could not connect to address=(host=localhost)(port=3306)(type=master) : Socket fail to connect to host:localhost, port:3306. Connection refused: connect
Caused by: java.sql.SQLNonTransientConnectionException: Socket fail to connect to host:localhost, port:3306. Connection refused: connect
at org.mariadb//org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:73) [mariadb-java-client.jar:]
...
Caused by: java.net.ConnectException: Connection refused: connect
error.logunder<AIQUM_INSTALL_DRIVE>:\ProgramData\MYSQL\MySQLServerData\Dataindicates corruption
Example1:
[Note] [MY-012551] [InnoDB] Database was not shutdown normally!
[Note] [MY-012552] [InnoDB] Starting crash recovery.
[Note] [MY-013086] [InnoDB] Starting to parse redo log at lsn = 175331412096, whereas checkpoint_lsn = 175331412106 and start_lsn = 175331411968
[Note] [MY-012550] [InnoDB] Doing recovery: scanned up to log sequence number 175331412106
[Note] [MY-013083] [InnoDB] Log background threads are being started...
[Note] [MY-012532] [InnoDB] Applying a batch of 0 redo log records ...
[Note] [MY-012535] [InnoDB] Apply batch completed!
[ERROR] [MY-011971] [InnoDB] Tablespace 'innodb_system' Page [page id: space=0, page number=5] log sequence number 175332036315 is in the future! Current system log sequence number 175331412106.
[ERROR] [MY-011972] [InnoDB] Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB redo log files. Please refer to http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html for information about forcing recovery
Exmaple2:
[ERROR] [MY-013183] [InnoDB] Assertion failure: fut0lst.ic:81:addr.page == FIL_NULL || addr.boffset >= FIL_PAGE_DATA thread 5944
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
- Windows Error Reporting (WER) reports
APPCRASHevent formysqld.exe - Windows event log indicates Event ID 6008 which means unexpected system shutdown
