Skip to main content
NetApp Knowledge Base

AIQUM MySQL database fails to startup due to corruption after unexpected shutdown

  1. Last updated
  2. Save as PDF
Views:
179
Visibility:
Public
Votes:
0
Category:
active-iq-unified-manager
Specialty:
om
Last Updated:

Applies to

  • Active IQ Unified Manager (AIQUM)
  • AIQUM host Windows was unexpectedly rebooted

Issue

  • mysqld service cannot startup
  • HTTP 404 - Not Found is returned by accessing AIQUM GUI
  • au.log indicates Server returned HTTP status 404

ERROR [WrapperStartStopAppMain] c.o.s.a.f.m.CommunicationManager (CommunicationManager.java:555) - Server returned HTTP status 404.
ERROR [WrapperStartStopAppMain] c.o.s.a.f.m.CommunicationManager (CommunicationManager.java:561) -  - Failed to get Summary SessionServer returned HTTP status 404. java.lang.RuntimeException: Server returned HTTP status 404.
    at com.onaro.sanscreen.acquisition.framework.mgmt.CommunicationManager.getSummarySessionFromREST(CommunicationManager.java:556) ~[au-framework.jar:9.13.0-2023.06.J287]
    at com.onaro.sanscreen.acquisition.framework.mgmt.CommunicationManager.waitForServerToFullyDeployed(CommunicationManager.java:450) ~[au-framework.jar:9.13.0-2023.06.J287]
    at com.onaro.sanscreen.acquisition.framework.mgmt.CommunicationManager.doStart(CommunicationManager.java:144) ~[au-framework.jar:9.13.0-2023.06.J287]
    at com.onaro.sanscreen.acquisition.framework.mgmt.FrameworkService.start(FrameworkService.java:75) ~[au-framework.jar:9.13.0-2023.06.J287]
    at com.onaro.sanscreen.acquisition.framework.mgmt.ServiceManager.doStart(ServiceManager.java:221) ~[au-framework.jar:9.13.0-2023.06.J287]
    at com.onaro.sanscreen.acquisition.framework.mgmt.FrameworkService.start(FrameworkService.java:75) ~[au-framework.jar:9.13.0-2023.06.J287]
    at com.onaro.sanscreen.acquisition.framework.Main.run(Main.java:66) ~[au-framework.jar:9.13.0-2023.06.J287]
    at com.onaro.sanscreen.acquisition.framework.Main.startUp(Main.java:163) ~[au-framework.jar:9.13.0-2023.06.J287]
    at com.onaro.sanscreen.acquisition.framework.Main.main(Main.java:127) ~[au-framework.jar:9.13.0-2023.06.J287]

  • jboss/server.log indicates connection to port 3306 fails

WARN  [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (pool-13-thread-1) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: IJ031084: Unable to create connection
    at org.jboss.ironjacamar.jdbcadapters@1.5.3.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:364)
    at org.jboss.ironjacamar.jdbcadapters@1.5.3.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:371)
    at org.jboss.ironjacamar.jdbcadapters@1.5.3.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:287)
    at org.jboss.ironjacamar.impl@1.5.3.Final//org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.createConnectionEventListener(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:1328)
    at org.jboss.ironjacamar.impl@1.5.3.Final//org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.getConnection(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:505)
..
Caused by: java.sql.SQLNonTransientConnectionException: Could not connect to address=(host=localhost)(port=3306)(type=master) : Socket fail to connect to host:localhost, port:3306. Connection refused: connect
    at org.mariadb//org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:73) [mariadb-java-client.jar:]
    at org.mariadb//org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:192) [mariadb-java-client.jar:]
    at org.mariadb//org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1346) [mariadb-java-client.jar:]
    at org.mariadb//org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:634) [mariadb-java-client.jar:]
    at org.mariadb//org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:150) [mariadb-java-client.jar:]
    at org.mariadb//org.mariadb.jdbc.Driver.connect(Driver.java:89) [mariadb-java-client.jar:]
    at org.jboss.ironjacamar.jdbcadapters@1.5.3.Final//org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:335)
    ... 24 more
Caused by: java.sql.SQLNonTransientConnectionException: Socket fail to connect to host:localhost, port:3306. Connection refused: connect
    at org.mariadb//org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:73) [mariadb-java-client.jar:]
    at org.mariadb//org.mariadb.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:183) [mariadb-java-client.jar:]
    at org.mariadb//org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.createSocket(AbstractConnectProtocol.java:255) [mariadb-java-client.jar:]
    at org.mariadb//org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.createConnection(AbstractConnectProtocol.java:512) [mariadb-java-client.jar:]
    at org.mariadb//org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1341) [mariadb-java-client.jar:]
    ... 28 more
Caused by: java.net.ConnectException: Connection refused: connect

  • error.log under <AIQUM_INSTALL_DRIVE>:\ProgramData\MYSQL\MySQLServerData\Data indicates corruption

Example1:

[Note] [MY-012551] [InnoDB] Database was not shutdown normally!
[Note] [MY-012552] [InnoDB] Starting crash recovery.
[Note] [MY-013086] [InnoDB] Starting to parse redo log at lsn = 175331412096, whereas checkpoint_lsn = 175331412106 and start_lsn = 175331411968
[Note] [MY-012550] [InnoDB] Doing recovery: scanned up to log sequence number 175331412106
[Note] [MY-013083] [InnoDB] Log background threads are being started...
[Note] [MY-012532] [InnoDB] Applying a batch of 0 redo log records ...
[Note] [MY-012535] [InnoDB] Apply batch completed!
[ERROR] [MY-011971] [InnoDB] Tablespace 'innodb_system' Page [page id: space=0, page number=5] log sequence number 175332036315 is in the future! Current system log sequence number 175331412106.
[ERROR] [MY-011972] [InnoDB] Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB redo log files. Please refer to http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html for information about forcing recovery

Exmaple2:

[ERROR] [MY-013183] [InnoDB] Assertion failure: fut0lst.ic:81:addr.page == FIL_NULL || addr.boffset >= FIL_PAGE_DATA thread 5944
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.

  • Windows Error Reporting (WER) reports APPCRASH event for mysqld.exe
  • Windows event log indicates Event ID 6008 which means unexpected system shutdown

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.