CFBMC-5427: AFF A400 and FAS8300 Multiple Internal Temp sensors are unreadable

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Views:: 204

Visibility:: Public

Votes:: 0

Category:: ontap-9

Specialty:: HW

Last Updated:

Issue

event log show reports multiple sensor-unreadable errors.

node1 ALERT nvmem.battery.notPresent: The NVMEM battery is not present. To prevent data loss, the system will shut down in 5 minutes.
node1 ERROR callhome.chassis.ps.degraded: Call home for CHASSIS POWER SUPPLY DEGRADED: PS 1
node1 EMERGENCY monitor.globalStatus.critical: Power Supply Status Critical: PSU2, PSU1.
node1 EMERGENCY callhome.chassis.overtemp: Call home for CHASSIS OVER TEMPERATURE SHUTDOWN
node1 EMERGENCY callhome.battery.failure: Call home for BATTERY (not present) CRITICAL.
node1 ERROR callhome.chassis.power: Call home for CHASSIS POWER DEGRADED: Power Supply Status Critical: PSU2, PSU1.
node1 ALERT monitor.chassisPower.degraded: Chassis power is degraded: Power Supply Status Critical: PSU2, PSU1.
node1 EMERGENCY monitor.shutdown.chassisOverTemp: Chassis temperature is too hot: Multiple Internal Temp sensors are unreadable. System will be shutdown in 10 minutes

Multiple sensors are unavailable or invalid, and the serial console on the node is unresponsive.

::*> node run -node node1 -command environment status chassis list-sensors
Node: node1
Sensor Name State Current Critical Warning Warning Critical
 Reading Low Low High High
-------------------------------------------------------------------------------------------------
NVDIMM0 Health GOOD
LED1 Temp not_available -- C 0 C 3 C 43 C 46 C
LED2 Temp not_available -- C 0 C 3 C 43 C 46 C
MP Temp1 not_available -- C 0 C 5 C 45 C 48 C
MP Temp3 not_available -- C 0 C 5 C 45 C 48 C
Mezz Temp1 invalid -- C 0 C 5 C 80 C 85 C
Mezz Temp2 invalid -- C 0 C 5 C 54 C 57 C
PSU1 VIN invalid -- mV 90480 mV 93600 mV 261040 mV 263120 mV
PSU1 VOUT invalid -- mV 11336 mV 11440 mV 12948 mV 13156 mV
PSU1 Curr IIN invalid -- mA 0 mA -- 9984 mA 12012 mA
PSU1 IOUT invalid -- mA 0 mA -- 130000 mA 132000 mA
PSU1 PIN invalid -- mW 7100 mW 14200 mW 1611700 mW 1796300 mW
PSU1 POUT invalid -- mW 7100 mW 14200 mW 1611700 mW 1796300 mW
PSU1 Inlet invalid -- C 0 C -- 58 C 63 C
PSU1 Hot invalid -- C 0 C -- 100 C 105 C
PSU1 FB Hot invalid -- C 0 C -- 100 C 105 C
PSU1 FAN not_available -- RPM 800 RPM 1200 RPM -- --
PSU2 VIN invalid -- mV 90480 mV 93600 mV 261040 mV 263120 mV
PSU2 VOUT invalid -- mV 11336 mV 11440 mV 12948 mV 13156 mV
PSU2 Curr IIN invalid -- mA 0 mA -- 9984 mA 12012 mA
PSU2 IOUT invalid -- mA 0 mA -- 130000 mA 132000 mA
PSU2 PIN invalid -- mW 7100 mW 14200 mW 1611700 mW 1796300 mW
PSU2 POUT invalid -- mW 7100 mW 14200 mW 1611700 mW 1796300 mW
PSU2 Inlet invalid -- C 0 C -- 58 C 63 C
PSU2 Hot invalid -- C 0 C -- 100 C 105 C
PSU2 FB Hot invalid -- C 0 C -- 100 C 105 C
PSU2 FAN invalid -- RPM 800 RPM 1200 RPM -- --
Fan1_1 not_available -- RPM 600 RPM 900 RPM -- --
Fan1_2 not_available -- RPM 600 RPM 900 RPM -- --
Fan2_1 not_available -- RPM 600 RPM 900 RPM -- --
Fan2_2 not_available -- RPM 600 RPM 900 RPM -- --
Fan3_1 not_available -- RPM 600 RPM 900 RPM -- --
Fan3_2 not_available -- RPM 600 RPM 900 RPM -- --
Fan4_1 not_available -- RPM 600 RPM 900 RPM -- --
Fan4_2 not_available -- RPM 600 RPM 900 RPM -- --

Fan sensors may alternatively report No Reading.

Fan1_1 | 70h | ns | 29.1 | No Reading
Fan2_1 | 71h | ns | 29.2 | No Reading
Fan3_1 | 72h | ns | 29.3 | No Reading
Fan4_1 | 73h | ns | 29.4 | No Reading
Fan1_2 | 74h | ns | 29.1 | No Reading
Fan2_2 | 75h | ns | 29.2 | No Reading
Fan3_2 | 76h | ns | 29.3 | No Reading
Fan4_2 | 77h | ns | 29.4 | No Reading

From baseboard management controller (BMC) CLI bmc log debug output, several sensor statuses are na.

[2023-07-04 20:27:26.617] CX5_Temp1 | na | degrees C | na | na | 0.000 | 5.000 | 80.000 | 85.000 | na
[2023-07-04 20:27:26.617] CX5_Temp2 | na | degrees C | na | na | 0.000 | 5.000 | 80.000 | 85.000 | na
[2023-07-04 20:27:26.617] LED1_Temp | na | degrees C | na | na | 0.000 | 3.000 | 43.000 | 46.000 | na
[2023-07-04 20:27:26.617] LED2_Temp | na | degrees C | na | na | 0.000 | 3.000 | 43.000 | 46.000 | na
[2023-07-04 20:27:26.617] MP_Temp1 | na | degrees C | na | na | 0.000 | 5.000 | 45.000 | 48.000 | na
[2023-07-04 20:27:26.667] MP_Temp3 | na | degrees C | na | na | 0.000 | 5.000 | 45.000 | 48.000 | na
[2023-07-04 20:27:29.016] Mezz_Temp1 | na | degrees C | na | na | 0.000 | 5.000 | 80.000 | 85.000 | na
[2023-07-04 20:27:29.016] Mezz_Temp2 | na | degrees C | na | na | 0.000 | 5.000 | 54.000 | 57.000 | na

BMC may automatically reboot to recover the sensor state.

Fri Sep 05 14:58:08 +0900 [node01: env_mgr: monitor.shutdown.chassisOverTemp:EMERGENCY]: Chassis temperature is too hot: Multiple Internal Temp sensors are unreadable. System will be shutdown in 10 minutes
Fri Sep 05 14:58:24 +0900 [node01: env_mgr: monitor.shutdown.cancel:notice]: Automatic shutdown sequence canceled.
Fri Sep 05 14:58:24 +0900 [node01: env_mgr: sp.reboot.sensor.unreadable:notice]: Rebooting BMC because one or more sensors are unreadable.
Fri Sep 05 14:59:14 +0900 [node01: env_mgr: callhome.chassis.overtemp:EMERGENCY]: Call home for CHASSIS OVER TEMPERATURE SHUTDOWN