"environment_sensors_iterator::next()" took longer than 50 seconds to complete

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Views:: 897

Visibility:: Public

Votes:: 0

Category:: aff-series

Specialty:: hw

Last Updated:

Applies to

AFF A800
ONTAP 9

Issue

When running system node environment sensors show, a node reports the following timeout error:

"environment_sensors_iterator::next()" took longer than 50 seconds to complete [from mgwd on node "node-01" (VSID: -1) to mgwd at 169.254.100.100]

In SP-MGMT-MLOG-TXT.GZ, errors reported:

[kern_servprocd:info:7949] Thrift: Sat Oct 19 21:55:00 2024 TSocket::open() connect() <Host: 192.0.2.82 Port: 50000>: Host is down

[kern_servprocd:info:7949] 0x809628500: 8803fd000070f9b4: ERR: Servprocd::CLI: sp_get_sensor_info_worker : TTX ERROR: 1 (connect() failed: Host is down) 

[kern_servprocd:info:7949] 0x809628500: 8803fd000070f9b4: ERR: Servprocd::bmc_client: bmc_get_sensor_info_imp: target_node=cluster-02, err=The Service Processor on node "cluster-02" is not reachable. Verify that the SP or BMC is online, verify that api-service is enabled on the SP or BMC, verify that the partner node is running, check if pings from SP or BMC to partner node work, check if hw-assist keep-alives are normal, check that network ports are configured correctly and are functional (up). Then, try the command again.

[kern_servprocd:info:7949] bmc_set_privlanconfig_to_chip:return err(-1)

[kern_servprocd:info:7949] bmc_set_privlanconfig_to_chip: netmask set failed

[kern_servprocd:info:7949] 0x808cd4e00: 0: ERR: Servprocd::SPConfig: manage_ipmi_tasks : Failed to set BMC priv lan.

[kern_servprocd:info:7949] 0x808cd4e00: 0: ERR: Servprocd::SPConfig: manage_ipmi_tasks : GET priv LANCONFIG : Network config from BMC 0.0.0.0 0.0.0.0

Error reported when updating SP Image:

Node_01::*> system service-processor image update -node Node_01 Note: Firmware update will need to reboot the SP on completion. If your console connection is through the SP, it will be disconnected Do you want to proceed with the firmware update ? {y|n}: y Error: command failed: Pre-update Check failed: SP IP address not available

storage failover hwassist test may or may not fail for this node.
It is possible to SSH to the BMC of the node.
If attempting ONTAP upgrade, ONTAP image validation will fail with Shelves and Sensors check error (Error: Validation check failed to execute properly.).
The steps in Before and after upgrade validation check fails to execute on shelves and sensors have been followed with no change to the above symptoms.