Ontap reports s3 tier unavailable TLS handshake timeout
- Views:
- 500
- Visibility:
- Public
- Votes:
- 0
- Category:
- storagegrid
- Specialty:
- sgrid
- Last Updated:
- 2/19/2024, 8:07:21 AM
Applies to
- StorageGRID 11.6
- Fabricpool Solution
- OnTap 9.10.1P7
- Redundant StorageGRID gateway nodes under a hardware load balancer.
- Legacy CLB service on StorageGRID gateway nodes.
Issue
OnTap Fabricpool reports in events:
Tue Jan 17 02:15:42 +0000 [eu-cfiler05-01: ktlsd: ktls.failed:notice]: "The TLS connections have failed several times with remote host '10.11.92.84' in IPspace '4294967295', for which the latest reason given is: TLS: Handshake timed out.
Tue Jan 17 02:17:03 +0000 [eu-cfiler05-01: OscHighPriThreadPool: object.store.unavailable:EMERGENCY]: Unable to connect to the object store "xx-s3" from node xxxxxx-709d-11ec-8271-d039ea380eaa. Reason: Internal server error.
Tue Jan 17 02:17:03+0000 [eu-cfiler05-01: OscLowPriThreadPool: object.store.available:notice]: Able to connect to the object store "xx-s3" from node xxxxxx-709d-11ec-8271-d039ea380eaa.
Checking the kernel.log on both the gateway nodes, we see:
Jan 17 01:03:46 localhost kernel: [5006170.113426] TCP: request_sock_TCP: Possible SYN flooding on port 8082. Sending cookies. Check SNMP counters.
Jan 17 02:36:29 localhost kernel: [5011732.576329] TCP: request_sock_TCP: Possible SYN flooding on port 8082. Sending cookies. Check SNMP counters.
The connection restores without any user intervention.