PODs come online again very slowly during Kubernetes node upgrades
Applies to
Trident for Kubernetes/Openshift (23.01.0 to 24.06.0)
Issue
When upgrading Kubernetes nodes, the PODs are slow in coming online, hitting client rate limitation messages on volume operations (detach/attach), when Trident attempts to update Kubernetes about the volume (un)publication:
level=error msg="error saving volume publication record" error="client rate limiter Wait returned an error: context deadline exceeded" logLayer=core requestID=<REQUEST_ID> requestSource=CSI workflow="controller=publish"
level=error msg="error saving volume publication record" Method=ControllerPublishVolume Type=CSI_Controller logLayer=csi_frontend requestID=<REQUEST_ID> requestSource=CSI workflow="controller=publish"
level=error msg="GRPC error: rpc error: code = Unknown desc = error saving volume publication record" logLayer=csi_frontend requestID=<REQUEST_ID> requestSource=CSI