Intermittent unable to communicate with node alert on single node
Applies to
Issue
- Intermittent
'Unable to communicate with node'
alert on single node that self-resolves within minutes.
Dynip.log
from problematic node indicates:
[2023-08-22T14:24:52.795]DummyThread-1125493 - ERROR --Exception in send_node_update Traceback(most recent call last):
File"/usr/lib/python2.7/dist-packages/dynip/threads/heartbeat.py", line155, in send_node_update
response = send_request(msg_id, ipaddr,path, 'PUT', body)
File"/usr/lib/python2.7/dist-packages/dynip/threads/heartbeat.py", line116, in send_request
response = urlopen(request, timeout=10,context=_ssl_context)
File"/usr/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File"/usr/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File"/usr/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File"/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File"/usr/lib/python2.7/urllib2.py", line 1241, in https_open
context=self._context)
File"/usr/lib/python2.7/urllib2.py", line 1201, in do_open
r = h.getresponse(buffering=True)
File"/usr/lib/python2.7/httplib.py", line 1137, in getresponse
response.begin()
File"/usr/lib/python2.7/httplib.py", line 448, in begin
version, status, reason =self._read_status()
File"/usr/lib/python2.7/httplib.py", line 412, in _read_status
raise BadStatusLine("No status linereceived - the server has closed the connection")
BadStatusLine:No status line received - the server has closed the connection
Dynip.log
from other storage nodes indicates:
[2023-08-22T14:15:12.091]DummyThread-1123635 - WARNING -- Update to (NODE_NAME) (IP_ADDRESS) failed:('The read operation timed out',)
[2023-08-22T14:18:36.155]DummyThread-1123873 - INFO -- Received heartbeat from node (NODE_NAME) (IP_ADDRESS)