Skip to main content
NetApp Knowledge Base

OnCommand Unified Manager fails to purge "unified-manager.xxxxxx" binlogs

  1. Last updated
  2. Save as PDF
Views:
1,148
Visibility:
Public
Votes:
2
Category:
oncommand-unified-manager
Specialty:
legacy
Last Updated:

Applies to

  • OnCommand Unified Manager ( OCUM )
  • ActiveIQ Unified Manager (AIQUM)
  • All platforms

Issue

After upgrading Unified Manager, the available space in /data or your Windows install directory began to decrease abnormally fast over time. In the example below the /data filesystem is 93% used. 

diag@ocum:~$ df -TH
Filesystem               Type      Size  Used Avail Use% Mounted on
rootfs                   rootfs     64G   14G   47G  23% /
udev                     devtmpfs   11M     0   11M   0% /dev
tmpfs                    tmpfs     1.3G  173k  1.3G   1% /run
/dev/sda1                ext3       64G   14G   47G  23% /
tmpfs                    tmpfs     5.3M     0  5.3M   0% /run/lock
tmpfs                    tmpfs     5.2G     0  5.2G   0% /run/shm
/dev/sdd1                ext3       32G  123M   30G   1% /jail
/dev/mapper/data--vg-lv2 ext4      793G  698G   55G  93% /data

While attempting to upgrade Unified Manager one might run into the following message saying that there is not enough free space.
Example: Upgrading from Unified Manager 7.2P1 to Unified Manager 9.4.

Unable to perform operation due to available disk space limitations. Please add additional storage to this VM or contact technical support for assistance"

Note: There are other situations one might encounter this message.

Cause

Unified Manager fails to purge the MySQL binlogs after a backup failure, documented in BUG 1103982. Check the /opt/netapp/data/mysql and /opt/netapp/data directories for excessive unified-manager.xxxxxx binlogs. In the example below one will need to login as the diag user in order to check the directory as Unified Manager is installed on an OVA. For accessing the diag user please see the knowledge article How to access the OnCommand Virtual Machine DIAG shell.

diag@ocum:opt/netatpp/data$ ls
acquisition             unified-manager.000052  unified-manager.000129  unified-manager.000206  unified-manager.000283
auto.cnf                unified-manager.000053  unified-manager.000130  unified-manager.000207  unified-manager.000284
ddl_log.log             unified-manager.000054  unified-manager.000131  unified-manager.000208  unified-manager.000285
ib_buffer_pool          unified-manager.000055  unified-manager.000132  unified-manager.000209  unified-manager.000286
ib_logfile0             unified-manager.000056  unified-manager.000133  unified-manager.000210  unified-manager.000287
ib_logfile1             unified-manager.000057  unified-manager.000134  unified-manager.000211  unified-manager.000288
ibdata1                 unified-manager.000058  unified-manager.000135  unified-manager.000212  unified-manager.000289
ibtmp1                  unified-manager.000059  unified-manager.000136  unified-manager.000213  unified-manager.000290
management              unified-manager.000060  unified-manager.000137  unified-manager.000214  unified-manager.000291
mysql                   unified-manager.000061  unified-manager.000138  unified-manager.000215  unified-manager.000292
mysql_upgrade_info      unified-manager.000062  unified-manager.000139  unified-manager.000216  unified-manager.000293
netapp_model            unified-manager.000063  unified-manager.000140  unified-manager.000217  unified-manager.000294
netapp_model_view       unified-manager.000064  unified-manager.000141  unified-manager.000218  unified-manager.000295
netapp_performance      unified-manager.000065  unified-manager.000142  unified-manager.000219  unified-manager.000296
ocum                    unified-manager.000066  unified-manager.000143  unified-manager.000220  unified-manager.000297
ocum.err                unified-manager.000067  unified-manager.000144  unified-manager.000221  unified-manager.000298
ocum_data_collector     unified-manager.000068  unified-manager.000145  unified-manager.000222  unified-manager.000299
ocum_report             unified-manager.000069  unified-manager.000146  unified-manager.000223  unified-manager.000300
ocum_report_birt        unified-manager.000070  unified-manager.000147  unified-manager.000224  unified-manager.000301
ocum_view               unified-manager.000071  unified-manager.000148  unified-manager.000225  unified-manager.000302
opm                     unified-manager.000072  unified-manager.000149  unified-manager.000226  unified-manager.000303
performance_schema      unified-manager.000073  unified-manager.000150  unified-manager.000227  unified-manager.000304
sanscreen               unified-manager.000074  unified-manager.000151  unified-manager.000228  unified-manager.000305
scalemonitor            unified-manager.000075  unified-manager.000152  unified-manager.000229  unified-manager.000306
scrub                   unified-manager.000076  unified-manager.000153  unified-manager.000230  unified-manager.000307
sys                     unified-manager.000077  unified-manager.000154  unified-manager.000231  unified-manager.000308
unified-manager.000001  unified-manager.000078  unified-manager.000155  unified-manager.000232  unified-manager.000309
unified-manager.000002  unified-manager.000079  unified-manager.000156  unified-manager.000233  unified-manager.000310
unified-manager.000003  unified-manager.000080  unified-manager.000157  unified-manager.000234  unified-manager.000311
unified-manager.000004  unified-manager.000081  unified-manager.000158  unified-manager.000235  unified-manager.000312
unified-manager.000005  unified-manager.000082  unified-manager.000159  unified-manager.000236  unified-manager.000313
unified-manager.000006  unified-manager.000083  unified-manager.000160  unified-manager.000237  unified-manager.000314
unified-manager.000007  unified-manager.000084  unified-manager.000161  unified-manager.000238  unified-manager.000315
unified-manager.000008  unified-manager.000085  unified-manager.000162  unified-manager.000239  unified-manager.000316

Normally we'd expect to see 1-2 unified-manager.xxxxxx binlogs and a single unified-manager.index file. Failing to purge the unified-manager.xxxxxx binlogs can eat up available space over time. This example has been cut down for presentation purposes as there can be many more binlogs contained in this directory than is seen above. This behavior has also been observed on RedHat Enterprise Linux installations. In the event that Unified Manager is installed to a RedHat Enterprise Linux server please check the directory /opt/netapp/data in addition to /opt/netapp/data/mysql for unified-manager.xxxxxx binlog files.

Solution

  1. Take a snapshot before continuing.

  2. Login to Unified Manager via SSH/RDP.

Note: NetApp best practice is to take a cold VMware snapshot of the ActiveIQ Unified Manager host before making any changes to the MySQL database.

  1. Log into the ActiveIQ Unified Manager MySQL database. Use the knowledge article: How to access MYSQL within all OnCommand Unified Manager and OnCommand Performance Manager platforms
  2. At the prompt, set a database by typing the following:

use mysql;

  1. Purge the Logs from the MySQL backup utility by running each of the commands below that end with a semicolon; individually, in sequence:

FLUSH LOGS;

PURGE BINARY LOGS BEFORE CURRENT_TIMESTAMP;

exit

For example:

mysql> FLUSH LOGS;
Query OK, 0 rows affected (0.01 sec)
 
mysql> PURGE BINARY LOGS BEFORE CURRENT_TIMESTAMP;
Query OK, 0 rows affected, 1 warning (0.00 sec)
 
In the output provided, “Query OK”  indicates that the command executed properly.  
“0 rows affected”  is printed because the command that was run did not update any active MYSQL Tables or ‘rows’

 

Note: Running through this action plan will break the backup chain. The next backup will be a full backup.

Partner Notes

partnerNotes_text

Additional Information

additionalInformation_text

Internal Notes

internalNotes_text

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.