Trident PVC mount fails with Autogrow and Concurrency
Applies to
- NetApp Trident 26.06.1
- ONTAP 9.18.1P6 (Cluster-Mode)
- Kubernetes v1.35.5 (OpenShift 4.22.x)
- StorageClass: ontap-nvme-block (autogrowPolicy enabled)
Issue
Pods in the openshift-monitoring namespace fail to start and remain stuck in CreateContainerConfigError due to repeated errors mounting PVCs backed by ONTAP via Trident.
Observed symptoms:
- Multiple NetApp Trident and ONTAP alerts:
volume "trident_pvc_<uuid>" is full (using or reserving 91% of space and 0% of inodes) - On the ONTAP system, the volume/filesystem appears under threshold (not full).
- Kubernetes pod events (example):
Warning Failed kubelet Error: failed to prepare subPath for volumeMount "<namespace>-k8s-db" of container - Kubelet logs:
error resolving symlinks in "/var/lib/kubelet/pods/.../volumes/kubernetes.io~csi/pvc-<uuid>/mount": lstat ...: input/output error - Pod status:
<namespace>-k8s-0 5/6 CreateContainerConfigError
<namespace>-k8s-1 5/6 CreateContainerConfigError - Trident node logs (decisive error):
fatal error: concurrent map writesgithub.com/netapp/trident/frontend/csi/node_helpers/kubernetes.(*helper).AddPublishedPath/go/src/github.com/netapp/trident/frontend/csi/node_helpers/kubernetes/plugin.go:214 - Repeated warnings/errors in Trident node logs:
Failed to add volume(tvp) to effectivePolicyCache ... already exists in cacheFailed to add volume(tvp) to assorter ... already exists in assorterProcessedCount=0 ... errorCount=N ... during autogrow reconciliation
- Affected PVCs:
<namespace>-k8s-db-prometheus-k8s-0(pvc-name)<namespace>-k8s-db-prometheus-k8s-1(pvc-name)
- Red Hat support confirmed issue is isolated to storage backend and not a permissions/configuration issue on the Kubernetes side.
