CONTAPEXT-4200: Proxmox VE iSCSI DMMP LUN path remains in "active ghost" state when SMas relations goes Out-Of-Sync and comes back to InSync
Issue
When SnapMirror active sync Consistency Group (CG) goes into Out-of-Sync (OOS) state, the Proxmox VE iSCSI DMMP LUN path gets into "failed faulty" state on the paths coming from secondary site controller nodes, which expected behavior. However when the CG comes back to InSync state, the Proxmox VE iSCSI multipath does not return to "active ready" state, but rather remain in "active ghost" state on the LUN paths coming from secondary site controller nodes. The LUN paths that are in "active ghost" state are deemed unusable for IO activities.
The below is the multipath -ll command output from the Proxmox hypervisor CLI during the event.
A) Multipath status while In-Sync:
LUN paths from both sites are in "active ready" state.
B) After CG Out-of-Sync:
After CGs go into Out-of-Sync state, the LUN paths from the secondary site controller nodes will be in "failed faulty" state. The LUN paths from the primary site remain active and running and usable for IO activity.
C) After CGs come back to InSync:
After CGs come back to InSync state, the LUN paths from secondary site comes to "active ghost" instead of "active ready" state.
Applications can still use paths from the primary site which are in "active ready" state, and there shall be no disruption to IO.
The below is the multipath -ll command output from the Proxmox hypervisor CLI during the event.
A) Multipath status while In-Sync:
LUN paths from both sites are in "active ready" state.
3600a09803831504c4f5d2f4a7278546e dm-17 NETAPP,LUN C-Mode
size=50G features='3 queue_if_no_path pg_init_retries 50' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 16:0:0:70 sdcq 69:224 active ready running
| `- 18:0:0:70 sddo 71:96 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
|- 7:0:0:70 sdw 65:96 active ready running
|- 8:0:0:70 sdai 66:32 active ready running
|- 9:0:0:70 sdau 66:224 active ready running
|- 10:0:0:70 sdbg 67:160 active ready running
|- 15:0:0:70 sdce 69:32 active ready running
`- 17:0:0:70 sddc 70:160 active ready running
B) After CG Out-of-Sync:
After CGs go into Out-of-Sync state, the LUN paths from the secondary site controller nodes will be in "failed faulty" state. The LUN paths from the primary site remain active and running and usable for IO activity.
3600a09803831504c4f5d2f4a7278546e dm-17 NETAPP,LUN C-Mode
size=50G features='3 queue_if_no_path pg_init_retries 50' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=0 status=enabled
| |- 16:0:0:70 sdcq 69:224 active ready running
| `- 18:0:0:70 sddo 71:96 active ready running
`-+- policy='service-time 0' prio=10 status=active
|- 7:0:0:70 sdw 65:96 failed faulty offline
|- 8:0:0:70 sdai 66:32 failed faulty offline
|- 9:0:0:70 sdau 66:224 failed faulty offline
|- 10:0:0:70 sdbg 67:160 failed faulty offline
|- 15:0:0:70 sdce 69:32 active ready running
`- 17:0:0:70 sddc 70:160 active ready running C) After CGs come back to InSync:
After CGs come back to InSync state, the LUN paths from secondary site comes to "active ghost" instead of "active ready" state.
3600a09803831504c4f5d2f4a7278546e dm-17 NETAPP,LUN C-Mode
size=50G features='3 queue_if_no_path pg_init_retries 50' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=50 status=enabled
| |- 16:0:0:70 sdcq 69:224 active ready running
| `- 18:0:0:70 sddo 71:96 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
|- 7:0:0:70 sdw 65:96 active ghost running
|- 8:0:0:70 sdai 66:32 active ghost running
|- 9:0:0:70 sdau 66:224 active ghost running
|- 10:0:0:70 sdbg 67:160 active ghost running
|- 15:0:0:70 sdce 69:32 active ready running
`- 17:0:0:70 sddc 70:160 active ready running Applications can still use paths from the primary site which are in "active ready" state, and there shall be no disruption to IO.