ONTAP Select node reboots unexpectedly after Long CP and vNVRAM flush delay

• NetApp ONTAP Select
• ONTAP 9

• ONTAP Select node reboots unexpectedly with panic-string: received completion for unknown cmd in process irqXXX: nvme0
• The device referred is nvmeX, in confgurations without NVMe backend usually nvme0
• ONTAP side log sequence leading to the panic:

Sat Jul 02 03:10:28 +0200 [node-01: ctlg_flxlg_mirror: vnvram.dma.long.wait:alert]: vNVRAM flush taking over 10 seconds.
Sat Jul 02 03:10:29 +0200 [node-01: wafl_exempt03: wafl.cp.toolong:error]: Aggregate aggr0 experienced a long CP.
Sat Jul 02 03:10:30 +0200 [node-01: irq282: nvme0: cf.fm.localFwTransition:debug]: params: {'progresscounter': '1031', 'newstate': 'SF_DUMPCORE', 'prevstate': 'SF_UP'}
Sat Jul 02 03:10:30 +0200 [node-01: irq282: nvme0: ha.panicInfoSent:notice]: Node successfully sent a panic information message to its HA partner. Partner name: . Partner system ID: 1234567890.
Sat Jul 02 03:10:30 +0200 [node-01: irq282: nvme0: sk.panic:alert]: Panic String: received completion for unknown cmd in process irq282: nvme0 on release 9.9.1P8 (C)

• ESXi side vmware.log sequence:

2022-07-02T01:10:30.121Z| vcpu-0| | I005: NVME-VMM: Controller level reset via CC.EN bit transition on nvme0
2022-07-02T01:10:30.121Z| vcpu-0| | I005: NVME-CORE: Doing a partial reset of controller regs and queues.
2022-07-02T01:10:33.353Z| vcpu-0| | I005: HBACommon: First write on scsi0:0.fileName='/vmfs/volumes/.../ontapselect-n02/ontapselect-n02.vmdk'