Volume inaccessible due to connectivity issue between Azure CVO and Azure Key Vault

Last updated

Feb 15, 2025
Save as PDF
Share
1. Share
2. Tweet
3. Share

Views:: 307

Visibility:: Public

Votes:: 0

Category:: cloud-volumes-ontap-cvo

Specialty:: cloud

Last Updated:: 2/15/2025, 1:48:10 PM

Applies to

Cloud Volumes ONTAP (CVO)
Microsoft Azure
Azure Key Vault (AKV)

Issue

1. Key access errors associated with AKV in EMS logs as below:

MonJan 22 03:53:09 +1000 [node-01: mgwd: ekmip.akv.volOffline:alert]:The Vserver has been blocked and any encrypted volumes belonging to Vserver "vservername" have been taken offline due to keyaccess errors associated with the key-id https://kv-storage-avs-4.vault.azure.net/keys/cvoname-vserver-nve owned by Azure Key Vault https://kv-storage-avs-4.vault.azure.net/

2. Volumes are taken offline due to key inaccessible:

Mon Jan 22 03:53:01 +1000 [node-01: vv_config_worker04: wafl.vvol.offline:info]: Volume 'vol01@vserver:b715xxxx-00xx-11xx-9xxx-00xx4892xxxx' has been set temporarily offline

Mon Jan 22 03:53:01 +1000 [node-01: vv_config_worker20: wafl.vvol.offline:info]: Volume 'vol02(1)@vserver:b715xxxx-00xx-11xx-9xxx-00xx4892xxxx' has been set temporarily offline

3. Verify you see Error: IO in KMIP2-Client.GZ file as below:

00000022.000dd8b8 0x4xxx3d Wed Jan 31 2024 05:56:09 +10:00 [kern_kmip2_client:info:8183] [Jan 31 05:56:09]: 0x80axx5f00: 0: ERR: kmip2::kmipCmds::KmipConnection: [cryptsoftErrorCb]:93: Error: distro/kmip_ssl.c: 4557: error: 10: msg: kmip_ssl_conn_do_handshake:login.microsoftonline.com

00000022.000dd8b9 0x4xxx3d Wed Jan 31 2024 05:56:09 +10:00 [kern_kmip2_client:info:8183] [Jan 31 05:56:09]: 0x80axx5f00: 0: ERR: kmip2::kmipCmds::KmipConnection: [cryptsoftErrorCb]:93: Error: distro/kmip_io.c: 229: error: 10: msg: KMIP_xmit:KMIP_send

00000022.000dd8ba 0x4xxx3d Wed Jan 31 2024 05:56:09 +10:00 [kern_kmip2_client:info:8183] [Jan 31 05:56:09]: 0x80axx5f00: 0: ERR: kmip2::kmipCmds::KmipConnection: [cryptsoftErrorCb]:93: Error: src/kmip_cmd.c: 2648: error: 10: msg: KMIP_CMD_xmit:KMIP_xmit

00000022.000dd8bb 0x4xxx3d Wed Jan 31 2024 05:56:09 +10:00 [kern_kmip2_client:info:8183] [Jan 31 05:56:09]: 0x80axx5f00: 0: ERR: kmip2::kmipCmds::KmipConnection: [cryptsoftErrorCb]:93: Error: src/AKV/kmip_akv_cmd.c: 839: error: 10: msg: akv_do_login

00000022.000dd8bc 0x4xxx3d Wed Jan 31 2024 05:56:09 +10:00 [kern_kmip2_client:info:8183] [Jan 31 05:56:09]: 0x80axx5f00: 0: ERR: kmip2::kmipCmds::KmipConnection: [cryptsoftErrorCb]:93: Error: src/AKV/kmip_akv_cmd.c: 1408: error: 10: msg: cmd_akv_encrypt

00000022.000dd8bd 0x4xxx3d Wed Jan 31 2024 05:56:09 +10:00 [kern_kmip2_client:info:8183] [Jan 31 05:56:09]: 0x80axx5f00: 0: ERR: kmip2::kmipCmds::KmipConnection: [cryptsoftErrorCb]:93: Error: src/tables/kmip_cloud_cmd.cc: 84: error: 11: msg: KMIP_get_data

00000022.000dd8be 0x4xxx3d Wed Jan 31 2024 05:56:09 +10:00 [kern_kmip2_client:info:8183] [Jan 31 05:56:09]: 0x80axx5f00: 0: ERR: kmip2::tables::kmip_akv_cmd: [getSmdbError]:109: Error: IO

4. Verifiy service_reachability is failed during the issue using command:

::*> azure check