Skip to main content
NetApp Knowledge Base

Constantly high Java Heap Usage regardless of reboot

Views:
260
Visibility:
Public
Votes:
0
Category:
storagegrid-webscale
Specialty:
sgrid
Last Updated:

Applies to

  • StorageGRID Webscale 10.3
  • StorageGRID Webscale 10.4
  • StorageGRID Webscale 11.0

Issue

Alarms of CAHP (Java Heap Usage) are constantly reported.

'top -b -n 1' in system_commands from lumberjack shows cassand+ is heavily consuming memory and CPU:

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
  652 cassand+  20   0 63.437g 0.011t 2.889g S 125.3 47.1   1355:59 java


'nodetool cfstats' in cassandra/system_commands from lumberjack shows there exists large row in key_prefix_by_bucket:

        Table: key_prefix_by_bucket
        SSTable count: 8
        Space used (live), bytes: 1880268774
        Space used (total), bytes: 1880268774
        Off heap memory used (total), bytes: 724800
        SSTable Compression Ratio: 0.3176106335841224
        Number of keys (estimate): 1024
        Memtable cell count: 2970
        Memtable data size, bytes: 808762
        Memtable switch count: 428
        Local read count: 4225600
        Local read latency: 6.692 ms
        Local write count: 86867
        Local write latency: 0.026 ms
        Pending tasks: 0
        Bloom filter false positives: 0
        Bloom filter false ratio: 0.00000
        Bloom filter space used, bytes: 10688
        Bloom filter off heap memory used, bytes: 10624
        Index summary off heap memory used, bytes: 384
        Compression metadata off heap memory used, bytes: 713792
        Compacted partition minimum bytes: 150
        Compacted partition maximum bytes: 1386179893    <<<< 1.4G
        Compacted partition mean bytes: 132014784
        Average live cells per slice (last five minutes): 1.0
        Average tombstones per slice (last five minutes): 0.0

 

CUSTOMER EXCLUSIVE CONTENT

Registered NetApp customers get unlimited access to our dynamic Knowledge Base.

New authoritative content is published and updated each day by our team of experts.

Current Customer or Partner?

Sign In for unlimited access

New to NetApp?

Learn more about our award-winning Support