Skip to main content
NetApp Knowledge Base

Trident driver takes minutes to start fully after its pod termination (ontap-san)

  1. Last updated
  2. Save as PDF
Views:
145
Visibility:
Public
Votes:
0
Category:
trident-kubernetes
Specialty:
snapx
Last Updated:

Applies to

  • Trident for Openshift/Kubernetes
  • ontap-san and ontap-san-economy drivers

Issue

When the kubernetes cluster still contains a lot of old ontap-san* Trident PVs (LUNs), during Trident upgrades or Trident driver POD restarts,  the node's main Trident driver startup can be relatively slowl, taking in the order of minutes before being available for changes or new deployments.
 
In the meantime, the registrar container times out several times and restarts every 30 seconds, for example:
I0525 22:34:53.934755   75245 main.go:150] "Version" version="v2.13.0"
I0525 22:34:53.934806   75245 main.go:151] "Running node-driver-registrar" mode=""
I0525 22:34:53.934811   75245 main.go:172] "Attempting to open a gRPC connection" csiAddress="/plugin/csi.sock"
I0525 22:34:53.934824   75245 connection.go:234] "Connecting" address="unix:///plugin/csi.sock"
I0525 22:35:03.935038   75245 connection.go:253] "Still connecting" address="unix:///plugin/csi.sock"
I0525 22:35:13.934973   75245 connection.go:253] "Still connecting" address="unix:///plugin/csi.sock"
I0525 22:35:23.935336   75245 connection.go:253] "Still connecting" address="unix:///plugin/csi.sock"
E0525 22:35:23.935453   75245 main.go:176] "Error connecting to CSI driver" err="context deadline exceeded"
 
What also may be seen is this message in the driver's (previous) events, causing the driver to automatically terminate and restart:
fatal error: concurrent map writes
 
When starting, the main driver is doing iSCSI session checks as part of iSCSI self-healing, showing all paths for any of the visible LUNs this way:

time="2025-06-11T06:35:19Z" level=debugmsg="Found iSCSI host/session." hostNumber=3 logLayer=csi_frontendrequestID=556030d2-f64e-40b9-9e07-92d2c0453152 requestSource=InternalsessionNumber=5 workflow="plugin=activate"

time="2025-06-11T06:35:19Z" level=debugmsg="Found iSCSI host/session." hostNumber=4 logLayer=csi_frontendrequestID=556030d2-f64e-40b9-9e07-92d2c0453152 requestSource=InternalsessionNumber=6 workflow="plugin=activate"

time="2025-06-11T06:35:19Z" level=debugmsg="Found iSCSI host/session." hostNumber=5 logLayer=csi_frontendrequestID=556030d2-f64e-40b9-9e07-92d2c0453152 requestSource=InternalsessionNumber=7 workflow="plugin=activate"

time="2025-06-11T06:35:19Z" level=debugmsg="Found iSCSI host/session." hostNumber=6 logLayer=csi_frontendrequestID=556030d2-f64e-40b9-9e07-92d2c0453152 requestSource=InternalsessionNumber=8 workflow="plugin=activate"

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.