CVO in degraded state in BlueXP after ONTAP upgrade
Applies to
- Cloud Volumes ONTAP (CVO)
- Amazon Web Services (AWS)
- Mediator
- HA pair
- BlueXP
Issue
After ONTAP upgrade , CVO becomes degraded in BlueXP canvas even though CVO looks healthy from ONTAP CLI.
From server.log :
2024-11-01 15:15:17,645 UTC ERROR [Working Environment ] [xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx] [xxxxxxxx ] [x@x.com] (oncloud-akka.actor.default-dispatcher-209) [AwsHaDescribeFlowExecutor:126] Error while retrieving ha properties for working environment Vsa Working Environment [id: VsaWorkingEnvironment-xxxxxx, name: xxxxxxxx, ip: x.x.x.x,x.x.x.x]com.netapp.oncloud.simplicator.client.common.SimplicatorBadRequestException: Ask timed out on [Actor[akka://simplicator/user/IO-HTTP/host-connector-36#-xxxxxxxxxx]] after [30000 ms]. Sender[null] sent message of type "spray.http.HttpRequest"
2024-11-01 15:16:20,370 UTC WARN [Update all CVOs statuses ] [xxxxxxxx] [ ] [System ] (oncloud-akka.actor.default-dispatcher-184) [AwsHaClusterStatusCalculator:85] Calculated DEGRADED status with reason: Failed to establish HTTP connection to the mediator
From mediator.log :
[2024-11-01 15:14:28.73856] [xxx] [info] Received response:([{"message": "Connect timeout on endpoint URL: \"https://ec2.us-east-1.amazonaws.com/\"", "code": "<class 'botocore.exceptions.ConnectTimeoutError'>"}, {"message": "Unable to to determine vpc for either RT(xxx-xxxxxxxxxxxxx) or ENI(xxx-xxxxxxxxxxxxx) - Likely due to lack of permissions(check IAM role and if configured assumed role)", "code": 400}]) from mediator_route_update