Unable to access AIQUM GUI due to error "Cannot set log file .\ib_logfile101 to size"
Applies to
- ActiveIQ Unified Manager 9.6P1+
- Windows
Issue
- Unable to access the AIQUM GUI (error 404).
- MySQL and OCIE services are down, OCIEAU is up.
- Deployment file dfm-app.war is in failed state.
- When we try to start MySQL we get the following error:
Windows could not start the MYSQL8 service On Local Computer. Error 1053•. The service did not respond to the Start or control request in a timely fashion.
- From the MySQL error logs we could see the below errors:
2023-10-25T09:16:57.692725Z 1 [Note] [MY-013575] [InnoDB] Creating log file .\ib_logfile101 2023-10-25T09:18:10.926777Z 1 [Note] [MY-012887] [InnoDB] Setting log file .\ib_logfile101 size to 189770 MB. Progress : 10% 2023-10-25T09:19:40.485390Z 1 [Note] [MY-012887] [InnoDB] Setting log file .\ib_logfile101 size to 189770 MB. Progress : 20% 2023-10-25T09:19:59.163682Z 1 [ERROR] [MY-012644] [InnoDB] Encountered a problem with file '.\ib_logfile101' 2023-10-25T09:19:59.167174Z 1 [ERROR] [MY-012645] [InnoDB] Disk is full. Try to clean the disk to free space. 2023-10-25T09:19:59.170650Z 1 [ERROR] [MY-012888] [InnoDB] Cannot set log file .\ib_logfile101 to size 189770 MB 2023-10-25T09:20:01.496534Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Generic error. 2023-10-25T09:20:01.544845Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine 2023-10-25T09:20:01.554295Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed. 2023-10-25T09:20:01.560308Z 0 [ERROR] [MY-010119] [Server] Aborting
- We tried to clear up data from MySQL data drive, and were able to free up about 50 GB and had around 90GB available.
- Then we dodeployed the failed deployment file and stopped OCIEAU and OCIE service and started the MySQL service. The service failed with the same error as above.
- Then we could see the MySQL data drive E:\ rapidly filling up, probably due to the file recovery of .\ib_logfile101 going on in the backend, after reaching 0GB available, we could see the recovery terminated and the E drive return to the previous state.