Skip to main content
NetApp Knowledge Base

ESXi datastores not accessible with error "device entered the All Paths Down state"

  1. Last updated
  2. Save as PDF
Views:
1,931
Visibility:
Public
Votes:
12
Category:
ontap-9
Specialty:
san
Last Updated:

Applies to

  • VMware ESXi all versions 
  • ONTAP 9

Issue

  • All paths down reported by host and lost access to storage
  • Error message(s) seen in the host logs:
    • cpu34:33313)StorageApdHandler: 982: APD Handle Created with lock[StorageApd-0x43041149xxb0]
    • [APDCorrelator]:[vob.storage.apd.start] Device or filesystem with identifier [x] has entered the All Paths Down state.
    • [APDCorrelator]:[esx.problem.storage.apd.start] Device or filesystem with identifier [x] has entered the All Paths Down state.
  • ESX  Datastores seeing the numerous Path redundancy to storage device naa.600a0980383x363644xxf41354x6773 degraded. Path vmxba1:C0:Tx44:L100 is down. Affected datastores: EsxHxxPrdVxiOs900_ushouxxor1011
  • Below login events for observed in EMS logs for the affected host initiators:

Tue Nov 14 07:08:14 -0600 [ushouxxtor10-09: fct_tpd_work_thread_0: scsitarget.fct.portLogin:notice]: Login at target FC port: '2a' by initiator port: '20:0x:0x:25:bx:02:a0:76' address 0xe80xx6. The target virtual port is: 'NetApp FC Target Port (LPe32000) ushoxxtor1011-EsxVxi:ushoxxtor10-0x_fx_lif_1'.
Tue Nov 14 07:08:14 -0600 [ushouxxor10-09: fct_tpd_work_thread_0: scsitarget.fct.portLogin:notice]: Login at target FC port: '4a' by initiator port: '20:0x:0x:2x:bx:02:a0:76' address 0xe80xx6. The target virtual port is: 'NetApp FC Target Port (LPe32000) ushoxxtor1011-EsxVxi:ushoustor10-0x_fx_lif_3

  • IO WQE errors with extended status 16 is observed in ems logs:

Tue Nov 14 07:08:19 -0600 [ushxxstor10-09: fct_tpd_work_thread_0: fcp.io.status:debug]: STIO Adapter:2b IO WQE failure, Handle 0x3, Type 8, S_ID: C8xx92, VPI: 279, OX_ID: 472A, Status 0x3 Ext_Status 0x16

  • From vobd host logs path changed from dead errors observed:

2023-11-14T14:16:43.461Z: [scsiCorrelator] 6940447943826us: [vob.scsi.scsipath.pathstate.on] scsiPath vmhba2:C0:T14x:L100 changed state from dead
2023-11-14T14:16:43.485Z: [scsiCorrelator] 6940447967687us: [vob.scsi.scsipath.add] Add path: vmhba2:C0:T16x:L100
2023-11-14T14:16:43.492Z: [scsiCorrelator] 6940447974351us: [vob.scsi.scsipath.pathstate.on] scsiPath vmhba2:C0:T16x:L100 changed state from dead
2023-11-14T14:16:43.540Z: [scsiCorrelator] 6940448022605us: [vob.scsi.scsipath.add] Add path: vmhba2:C0:T14x:L100
2023-11-14T14:16:43.551Z: [scsiCorrelator] 6940448033745us: [vob.scsi.scsipath.pathstate.on] scsiPath vmhba2:C0:T14x:L100 changed state from dead

  • Below steps can be verified that might trigger the APD state of the storage device: 
    • Possible fabric issues
    • Verify end to end connectivity
    • Verify all end port states
    • Check for underlying Performance issues

 

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.