Skip to main content
NetApp Knowledge Base

Constantly high Java Heap Usage regardless of reboot on StorageGRID pre 11.1

Views:
325
Visibility:
Public
Votes:
0
Category:
storagegrid-webscale
Specialty:
sgrid
Last Updated:

Applies to

  • StorageGRID Webscale 10.3
  • StorageGRID Webscale 10.4
  • StorageGRID Webscale 11.0

Issue

Alarms of CAHP (Java Heap Usage) are constantly reported.

'top -b -n 1' in system_commands from lumberjack shows cassand+ is heavily consuming memory and CPU:

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
  652 cassand+  20   0 63.437g 0.011t 2.889g S 125.3 47.1   1355:59 java


'nodetool cfstats' in cassandra/system_commands from lumberjack shows there exists large row in key_prefix_by_bucket:

        Table: key_prefix_by_bucket
        SSTable count: 8
        Space used (live), bytes: 1880268774
        Space used (total), bytes: 1880268774
        Off heap memory used (total), bytes: 724800
        SSTable Compression Ratio: 0.3176106335841224
        Number of keys (estimate): 1024
        Memtable cell count: 2970
        Memtable data size, bytes: 808762
        Memtable switch count: 428
        Local read count: 4225600
        Local read latency: 6.692 ms
        Local write count: 86867
        Local write latency: 0.026 ms
        Pending tasks: 0
        Bloom filter false positives: 0
        Bloom filter false ratio: 0.00000
        Bloom filter space used, bytes: 10688
        Bloom filter off heap memory used, bytes: 10624
        Index summary off heap memory used, bytes: 384
        Compression metadata off heap memory used, bytes: 713792
        Compacted partition minimum bytes: 150
        Compacted partition maximum bytes: 1386179893    <<<< 1.4G
        Compacted partition mean bytes: 132014784
        Average live cells per slice (last five minutes): 1.0
        Average tombstones per slice (last five minutes): 0.0

 

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

Scan to view the article on your device