IOM software crash due to firmware failure
Applies to
- FAS2750
- IOM12E
Issue
- A crash occurs in the IOM module with an exception log after the SAS port goes down, accompanied by a chassisPower.degraded error.
[sas.port.down:debug]: SAS port "0b" went down.
[sas.adapter.debug:info]: params: {'adapterName': '0a', 'debug_string': 'SSP_EVENT 0xf on 0b.00.99, TAG 0x458c0042, INIT_TAG 0x1876: hard reset'}
[sas.adapter.debug:info]: params: {'adapterName': '0a', 'debug_string': 'Port 0: disabled 0, up 4, down 0: old state 3 --> new state 3'}
[sas.adapter.debug:info]: params: {'adapterName': '0a', 'debug_string': 'Port 1: Nominal speed changed: 11 --> 0'}
[sas.adapter.debug:info]: params: {'adapterName': '0a', 'debug_string': 'Port 1: disabled 0, up 0, down 4: old state 3 --> new state 1'}
[sas.adapter.debug:info]: params: {'adapterName': '0a', 'debug_string': 'Port 1: Minimum common speed changed: 11 --> 0'}
[ses.exceptionShelfLog:debug]: Retrieving Exception SES Shelf Log information on channel 0b IOM module B disk shelf ID 0.
[ses.IOMLogCrash:debug]: Encountered a crash in Exception Log on IOM module B SN [042007009113] FW 0201, channel 0b.00.99 shelf ID 0 shelf SN [*****].
[ses.IOMLogCrash:debug]: Encountered a crash in Exception Log on IOM module B SN [042007009113] FW 0201, channel 0b.00.99 shelf ID 0 shelf SN [*****].
[sla.shelf.message:debug]: params: {'type': 'SEVERITY', 'log': 'Startup type 5-Crash reset'}
[sla.shelf.message:debug]: params: {'type': 'SEVERITY', 'log': 'Failure info:PMC Firmware assert (err 1000e, d2 0, file ..\\..\\threadx_mips\\src\\osf\\osf_msg.c, line 546)'}
[monitor.chassisPower.degraded:alert]: Chassis power is degraded: Power Supply Status Critical: PSU1, PSU2.
[callhome.chassis.power:error]: Call home for CHASSIS POWER DEGRADED: Power Supply Status Critical: PSU1, PSU2.
[monitor.globalStatus.critical:EMERGENCY]: Power Supply Status Critical: PSU1, PSU2.
[monitor.chassisPowerSupplies.ok:info]: Chassis power supplies OK.
[monitor.globalStatus.ok:notice]: The system's global status is normal.
SP-IPMI logshows that the PSUs are all in normal status
PSU2 Bad FALSE
PSU1 Bad FALSE
PSU2 GOOD
PSU1 GOOD
PSU2 ON ON
PSU1 ON ON
- The
ENVIRONMENT logshows that the PSUs are all in normal condition.
Power Supply installed element list: 1, 2; with error: none
Power Supply information by element:
[1] Serial number: ***** Part number: 114-00148+F0
Type: 7D
Firmware version: 0111 Swaps: 0
[2] Serial number: ****** Part number: 114-00148+F0
Type: 7D
Firmware version: 0111 Swaps: 0
Voltage Sensor installed element list: 1, 2, 3, 4; with error: none
- Messages displayed in
SHELF-LOG-IOM.GZ:
Thu Jan 1 00:00:00 1970 ( 0+00:00:00.475); 02000027; U?; HAL; hal; 04; NDU Data Invalid.
Thu Jan 1 00:00:00 1970 ( 0+00:00:00.475); 02000093; U?; HAL; hal; 04; Module Reboot: Startup type 5-Crash reset
Thu Jan 1 00:00:00 1970 ( 0+00:00:00.475); 02000186; U?; HAL; hal; 04; Module Reboot: Latched power registers - EBOD = EF, ACP = EF
Thu Jan 1 00:00:00 1970 ( 0+00:00:00.475); 0200011E; U?; HAL; hal; 02; Canister CPLD POST passed
Thu Jan 1 00:00:00 1970 ( 0+00:00:00.475); 02000120; U?; HAL; hal; 02; Canister CPLD: V24
Thu Jan 1 00:00:00 1970 ( 0+00:00:00.656); 01000001; U?; CLI; cli; 02; Failed to bind CLI command: gpio_set_bit, status 4
Thu Jan 1 00:00:00 1970 ( 0+00:00:00.683); 0200013C; U?; HAL; hal; 02; Reboot after software crash.
Thu Jan 1 00:00:00 1970 ( 0+00:00:00.683); 0200013D; U?; HAL; hal; 02; Failure info:Cause: 00802808, PC: c00eacec, . Thread i2cipcwork 0
Thu Jan 1 00:00:00 1970 ( 0+00:00:00.683); 0200013E; U?; HAL; hal; 02; Failed firmware version: 0210, Code Image B
Thu Jan 1 00:00:00 1970 ( 0+00:00:00.683); 020001E3; U?; HAL; hal; 02; Sideband CPLD: V11