StorageGRID Node Reports Unable to Communicate with Node Due to LDR Service Crash

• A StorageGRID storage node reported the alert “Unable to communicate with node.”
• The node became unreachable, and backup operations from Rubrik to NetApp failed.
• Investigation of the logs revealed repeated LDR service crashes with "Resources file store failed with E013".

/var/local/log/servermanager.log

2026-05-23 02:31:09 +0000 | ldr                       | ldr ended
2026-05-23 02:31:14 +0000 | ldr                       | Start attempt 4. Trying for 5 minutes.
2026-05-23 02:31:18 +0000 | ldr                       | using jemalloc from /lib/x86_64-linux-gnu/libjemalloc.so.2

bycast-err.log

May 23 02:30:51 <Node_Name> ADE: |12730234 2096489724 PATM ATSA 2026-05-23T02:30:51.089924| ERROR    0204 PATM: Resources file store failed with 'E013'
May 23 02:30:51 <Node_Name> ADE: |12730234 2096489724 PATM ATSA 2026-05-23T02:30:51.089929| CRITICAL 0205 PATM: Assertion Failed#012[/build/src/modules/AttrPersistence_Module/AttrPersistence_Module.cc:205 void (a
nonymous namespace)::AttrPersistenceStorage::dispatch(ade::Message &&)] ASSERT FAILED (status == 'SUCS') is false. []
May 23 02:30:51 <Node_Name> ADE: |12730234 2096489724 PATM ATSA 2026-05-23T02:30:51.095856| ERROR    0483 PATM:  0# sgutil::stacktrace::current[abi:cxx11]() in /usr/local/ldr/ldr
May 23 02:30:51 <Node_Name> ADE: |12730234 2096489724 PATM ATSA 2026-05-23T02:30:51.095869| ERROR    0483 PATM:  1# CXD_Debug_PrintStackCrawl() in /usr/local/ldr/ldr
May 23 02:30:51 <Node_Name> ADE: |12730234 2096489724 PATM ATSA 2026-05-23T02:30:51.095873| ERROR    0483 PATM:  2# 0x0000000001539604 in /usr/local/ldr/ldr
May 23 02:30:51 <Node_Name> ADE: |12730234 2096489724 PATM ATSA 2026-05-23T02:30:51.095876| ERROR    0483 PATM:  3# AttrPersistence_Module(ade::Message&&, ADE_Process_Storage*) in /usr/local/ldr/ldr
May 23 02:30:51 <Node_Name> ADE: |12730234 2096489724 PATM ATSA 2026-05-23T02:30:51.095880| ERROR    0483 PATM:  4# 0x0000000001C14784 in /usr/local/ldr/ldr
May 23 02:30:51 <Node_Name> ADE: |12730234 2096489724 PATM ATSA 2026-05-23T02:30:51.095883| ERROR    0483 PATM:  5# ade_details::SpawnContext::processEntry() in /usr/local/ldr/ldr
May 23 02:30:51 <Node_Name> ADE: |12730234 2096489724 PATM ATSA 2026-05-23T02:30:51.095886| ERROR    0483 PATM:  6# void boost::context::detail::fiber_entry<boost::context::detail::fiber_record<boost::context::fi
ber, ade_details::SpawnContext::CachingStackAllocator<boost::context::basic_protected_fixedsize_stack<boost::context::stack_traits> >, ade_details::SpawnContext::CachingStackAllocator(unsigned int, std::function<void
()>, unsigned int, bool)::{lambda(boost::context::fiber&&)#1}> >(boost::context::detail::transfer_t) in /usr/local/ldr/ldr
May 23 02:30:51 <Node_Name> ADE: |12730234 2096489724 PATM ATSA 2026-05-23T02:30:51.095891| ERROR    0483 PATM:  7# make_fcontext in /lib/x86_64-linux-gnu/libboost_context.so.1.74.0

After the LDR service crash, the node could not be communicated with and backups to StorageGRID were disrupted.