Skip to main content
NetApp Knowledge Base

How to resolve memory issues with Active IQ Unified Manager

  1. Last updated
  2. Save as PDF
Views:
5,601
Visibility:
Public
Votes:
5
Category:
active-iq-unified-manager
Specialty:
om
Last Updated:

Applies to

  • Active IQ Unified Manager (AIQUM)
  • OnCommand Unified Manager (OCUM)

Answer

  1. Determine the amount of memory allocated and reserved

  2. Determine the amount of reserved memory

    • Utilizing reserved memory should be full or none, as fractional reserves can cause issues

  3. Check AIQUM memory recommendations

Software Version System Requirements

Active IQ Unified Manager 9.6 and above

 VMware OVA - Virtual infrastructure or hardware system requirements
Microsoft Windows - Virtual infrastructure or hardware system requirements
Red Hat and CentOS Linux - Virtual infrastructure or hardware system requirements
OnCommand Unified Manager 9.5 and earlier Virtual infrastructure or hardware system requirements

  1. Review best practice recommendations for sizing

    1. See page 10 - "Unified Manager scalability"
    2. If you are more than halfway to the next tier, increase resources to the next tier to allow for growth.
      i.e. If you have 30 nodes, increase memory from 12GB to 60GB
    3. Alternately, add 1GB of memory for each node monitored in addition to the starting 12GB memory.
      i.e. If you have 30 nodes, increase memory from 12GB to 42GB

  2. Make memory adjustments as needed

    1. VMware Docs - Change the Memory Configuration
    2. VMware Docs - Allocate Memory Resources to a Virtual Machine in the VMware Host Client

  3. Confirm that no ballooning or swapping is occurring on the virtual machine.

    1. VMware Docs - Memory (Data Counters)
    2. VMware Docs - Memory (MB)

  4. Check for memory starved Unified Manager components 

  5. Linux only - Linux may stop Unified Manager processes to free memory

    1. On Linux platforms, check for OOM "Out of Memory" killer ending processes to free up memory. 
    2. In rare instances Red Hat and CentOS platforms kill Unified Manager services / processes to free up memory.
      1. How to troubleshoot Out of memory (OOM) killer in Red Hat Enterprise Linux?

Dec 8 22:00:18 um-hostname kernel: Out of memory: Kill process 45678 (java) score 228 or sacrifice child
Dec 8 22:00:18 um-hostname kernel: Killed process 45988 (sh) total-vm:122868kB, anon-rss:260kB, file-rss:0kB, shmem-rss:0kB
Dec 8 22:00:19 um-hostname systemd: mysqld.service: main process exited, code=killed, status=9/KILL
Dec 8 22:00:19 um-hostname systemd: Unit mysqld.service entered failed state.
Dec 8 22:00:19 um-hostname systemd: mysqld.service failed.
Dec 8 22:00:20 um-hostname systemd: mysqld.service holdoff time over, scheduling restart.
Dec 8 22:00:20 um-hostname systemd: Stopped MySQL Server.
Dec 8 22:00:20 um-hostname systemd: Starting MySQL Server...

Additional Information

Common memory issues include

  • Unified Manager application is running slow:
    • Services take a long time to come online.
    • WebGUI is slow.
    • Backups take a very long time ( and possibly fail ).
  • Unified Manager application CPU and/or memory utilization are extremely high; swap / page file is in use.
  • Unified Manager application experiences inconsistent behavior:
    • Services hanging.
    • Services restarting.
    • Services not starting at all (especially after upgrade)
    • Occasional issues with alert notifications.
    • Inventory / performance acquisition polls may not complete within their normal 15 minute / 5 minute cycles.
    • Other abnormal (unexplainable) inconsistent behavior when running in a virtual environment.
  • VMware ESXi displays one or more of the following:
    • Memory Ballooning
    • Swapping
    • Transparent Page Sharing
    • Compression
  • Scale Monitoring Messages
    • "Unified Manager Server Low On Memory"
      • Memory_low_on_ocum.PNG
  • Log indications:
    • MySQL "error.log" file:
      • 2020-04-12T04:02:10.036849Z 205 [Note] Aborted connection XXX to db: 'unconnected' user: 'jboss' host: 'localhost' (Failed on my_net_write())
      • 2020-04-12T08:50:42.808598Z 450 [Note] Got timeout reading communication packets
    • Unified Manager "server_mega.log" file:
      • 2020-04-12 06:11:01,221 ERROR [default task-4] c.n.s.s.a.OriginatorUpdateServlet (OriginatorUpdateServlet.java:82) - Transaction rolled back: javax.ejb.EJBTransactionRolledbackException: Transaction rolled back
    • Unified Manager "ocumserver.log" file: 
      • 2019-06-06 02:49:49,566 INFO [oncommand] [pool-22-thread-4] [c.n.d.e.p.e.b.UnifiedManagerMemoryEventTypeController] Scale Monitor: New Memory Event Created: event id=128123,eventSeverity=low, thresholdValue=5.0, currentUsageValue=6.690000057220459, totalMemorySizeValue=1.365997568E10, jvm=AuJvm
      • 2019-06-06 01:49:49,190 INFO [oncommand] [pool-22-thread-55] [c.n.d.e.p.e.b.UnifiedManagerMemoryEventTypeController] Scale Monitor: New Memory Event Created: event id=128094,eventSeverity=almostOut, thresholdValue=7.0, currentUsageValue=7.195000171661377, totalMemorySizeValue=1.365997568E10, jvm=AuJvm

NOTE: This is not an exhaustive list of logs indicating memory issues. Please be aware that NetApp Support will find other log lines that can also indicate memory issues.

 

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.