Skip to main content

This Site will be down for up to 3 hours on December 2, 2023 from 8 PM - 11 PM PST, to deploy an infrastructure update.

NetApp Knowledge Base

S3 GETs on LATM (last access time) enabled buckets may cause memory exhaustion and crash StorageGRID nodes

  1. Last updated
  2. Save as PDF
Views:
183
Visibility:
Public
Votes:
0
Category:
storagegrid
Specialty:
sgrid
Last Updated:

Applies to

  • StorageGRID 11.2.0.3 or earlier
  • StorageGRID 11.1.1.9 or earlier

Description

Only if Last Access Time (LATM) on a bucket is enabled may this article be helpful.

bucket_settings.png

Symptoms
  • S3 Client side slow performance
  • Storage Node Memory depletion
  • Possible Storage Nodes crashing (Blue state)
  • Possible Storage Node LDR service crashing (Blue state)
  • High CPU Usage on Storage Node
  • "Worst Case Blocking (MSBX)" over 100s
    • GMI -> Storage Node -> LDR -> Resources

 

Cause

When LATM is enabled for a bucket

  • S3 Request will attempt to push all object segments to ILM when OBDR is destroyed
  • The pushWithSegments looks up the segment UUID metadata to enqueue segments into ILM queue
  • As ADE kernel considers the OBDR dead, the C* operation response is never delivered to it and OBDR never exits

 

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.
Scan to view the article on your device