Skip to main content
NetApp Knowledge Base

What causes EMS "Nblade.nfsConnResetAndClose with reason OncRpcTcpInvalidRecordMarker"

Views:
796
Visibility:
Public
Votes:
2
Category:
ontap-9
Specialty:
nfs
Last Updated:

 

Applies to

  • ONTAP 9
  • NFS

Answer

  • EMS logs an error of type Nblade.invTcpRecordMarker
  • It will likely be accompanied by an error of type Nblade.nfsConnResetAndClose with reason Nblade.InvTcpRecordMarker

Example:
Nblade.invTcpRecordMarker: Invalid TCP record marker received from remote IP address x.x.x.x on local port xxxx. Vserver ID associated with this operation is x.
Invalid reason is TCP_FRAGMENT, invalid record marker is xxxxxxxxx.

Nblade.nfsConnResetAndClose: Shutting down connection with the client. Vserver ID is x; network data protocol is NFS; client IP address:port is x.x.x.x:xxx. local IP
address is x.x.x.x; reason is OncRpcTcpInvalidRecordMarker.

  • This error message is seen when a TCP connection is reset due to an invalid TCP record marker in an Open Network Computing/Remote Procedure Call request.
  • At the head of every NFS call is an RPC call, if this RPC call has the incorrect number of bits or is what the ONTAP would not expect from a client, ONTAP will reset the Client TCP connection.
  • That is why the second error Nblade.nfsConnResetAndClose will likely be seen with it.
  • The error message will specify which client is giving the filer the incorrect RPC header, which local TCP port it was using, the Vserver, and the record Marker.
  • Below are the different subtypes of invalid TCP record markers for ONC RPC headers.

Note : For all 3 below cases, investigate with the client vendor why the client is sending an invalid record marker.

  1. TCP_FRAGMENT: The client sent RPC TCP fragments which are not currently supported.

Example:
Nblade.invTcpRecordMarker:error]: Invalid TCP record marker received from remote
IP address x.x.x.x on local port xxxx. Vserver ID associated with this operation is x.
Invalid reason is TCP_FRAGMENT, invalid record marker is 4.

2. TCP_DATA_LENGTH_OVERFLOW: The record marker received was great than the maximum supported size (70KB pre 8.2. 300KB 8.2+, 1MB in 9.0+)

Example:
Nblade.invTcpRecordMarker:error]: Invalid TCP record marker received from remote
IP address x.x.x.x on local port xxxx. Vserver ID associated with this operation is x.
Invalid reason is TCP_DATA_LENGTH_OVERFLOW, invalid record marker is xxxxxxxxx.

3. TCP_DATA_LENGTH_UNDERFLOW: The record marker received was smaller than the minimum supported size (Record marker must be greater than or equal to 40 bytes)
Example:
Nblade.invTcpRecordMarker:error]: Invalid TCP record marker received from remote
IP address x.x.x.x on local port xxxx. Invalid reason is TCP_DATA_LENGTH_UNDERFLOW, invalid record marker is 24.

 

Additional Information

  • One possible reason for this message is that the client runs telnet xxx.xxx.xxx.xxx 2049 to confirm the accessibility of NFS.

Linux client : 
[root@RHELClient8 ~]# telnet 10.xx.yy.211 2049
Trying 10.xx.yy.211...
Connected to 10.xx.yy.211.
Escape character is '^]'.
^]
telnet> ^C
Connection closed by foreign host.

[root@RHELClient8 ~]# ifconfig -a
ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.zz.xx.79  netmask 255.255.255.0  broadcast 10.zz.xx.255

cdot_vsim8::> date ; event log show -event *Nblade*

Time                Node             Severity      Event
------------------- ---------------- ------------- ---------------------------
4/14/2021 16:55:48  node1            ERROR         Nblade.nfsConnResetAndClose: Shutting down connection with the client. Vserver ID is 2; network data protocol is NFS, Rpc Xid 0x0; client IP address:port is 10.zz.xx.79:33504. local IP address is 10.xx.yy.211; reason is OncRpcTcpInvalidRecordMarker.
4/14/2021 16:55:48  node1            ERROR         Nblade.invTcpRecordMarker: Invalid TCP record marker received from remote IP address 10.zz.xx.79 on local port 2049. Vserver ID associated with this operation is 2. Invalid reason is TCP_FRAGMENT, invalid record marker is 218824696.