Skip to main content

NetApp_Insight_2020.png 

NetApp Knowledgebase

S3 GETs on LATM (last access time) enabled buckets may cause memory exhaustion and crash StorageGRID nodes

Views:
40
Visibility:
Public
Votes:
0
Category:
storagegrid
Specialty:
sgrid
Last Updated:

Applies to

  • StorageGRID 11.2.0.3 or earlier
  • StorageGRID 11.1.1.9 or earlier

Description

Only if Last Access Time (LATM) on a bucket is enabled may this article be helpful.

bucket_settings.png

Symptoms
  • S3 Client side slow performance
  • Storage Node Memory depletion
  • Possible Storage Nodes crashing (Blue state)
  • Possible Storage Node LDR service crashing (Blue state)
  • High CPU Usage on Storage Node
  • "Worst Case Blocking (MSBX)" over 100s
    • GMI -> Storage Node -> LDR -> Resources

 

Cause

When LATM is enabled for a bucket

  • S3 Request will attempt to push all object segments to ILM when OBDR is destroyed
  • The pushWithSegments looks up the segment UUID metadata to enqueue segments into ILM queue
  • As ADE kernel considers the OBDR dead, the C* operation response is never delivered to it and OBDR never exits

 

CUSTOMER EXCLUSIVE CONTENT

Registered NetApp customers get unlimited access to our dynamic Knowledge Base.

New authoritative content is published and updated each day by our team of experts.

Current Customer or Partner?

Sign In for unlimited access

New to NetApp?

Learn more about our award-winning Support