Skip to main content
NetApp Knowledge Base

How to manually rotate the logs in StorageGRID

Views:
417
Visibility:
Public
Votes:
1
Category:
storagegrid
Specialty:
sgrid
Last Updated:

Applies to

  • StorageGRID 7.5
  • StorageGRID 8.1
  • StorageGRID 8.5
  • StorageGRID 9.0

Description

The common location for the StorageGRID server debug logs is the /var/local/log/directory.

The most recent log messages are placed in the bycast.log file, while the older log messages are periodically rotated to the bycast.log.n file.

Note: This log format covers all the grid software, with the exception of the following:

  • Server Manager and Gridstat
  • Content Management System (CMS)
  • Network Management System - Web server and back end

Example:

In an instance, it was discovered that the /var/local directory ran out of space, causing some of the services to crash.

  • The following error was reported in the bycast-err.log:

Jun 14 13:40:53 vch-vgh-mt1 ADE: |20671179 19221 000010 ARNI ETEL 2011-06-14T13:40:53.530316| CRITICAL 0667 ARNI: Assertion Failed [libs/cxd/CXD_DiskQueue.cc:667 UINT32 CXD_DiskQueue_Enqueue(CXD_DiskQueue*, CXD_Memory*)] ASSERT FAILED ("CXD_DiskQueue library unable to write (ERR_SYS_NOSPC)" == false) is false

  • The following message was reported in the Kernel log: 

Sep 15 20:10:30 servername syslog-ng[2596]: io.c: do_write: write() failed (errno 28), No space left on device

  • The df-h command output displayed the following: 

Filesystem                         Size   Used   Avail   Use%   Mounted on
/dev/cciss/c0d0p1                  5.0G   2.5G   2.3G     52%   /
udev                               1014M  128K   1014M     1%   /dev
/dev/cciss/c0d0p3                  60G    60G    0       100%   /var/local
/dev/cciss/c0d1p1                  71G    69G    1.9G     98%   /var/local/mysql_ibdata
/dev/mapper/fsgvg-fsglv            64G    724K   64G       1%   /fsg
/dev/cciss/c0d3p1                  139G   29G    110G     21%   /var/local/rangedb/0

On checking the output of the du -hx --max-depth=1 command, the logs had stopped rotating correctly. The bycast.log was of 52GB size and the ha.log was of 6.2GB size. Both the files were deleted manually to clear the space.

The ls -ltr directory contained the following: 

-rw-r----- 1 root     root    108572147 Apr  8  2012 bycast.log.11.gz
-rw-r----- 1 root     root    108704513 Apr 29 11:00 bycast.log.10.gz
-rw-r----- 1 root     root    108611113 May 20 04:00 bycast.log.9.gz
-rw-r----- 1 root     root    108418726 Jun  9 18:00 bycast.log.8.gz
-rw-r----- 1 root     root    107182703 Jun 29 16:00 bycast.log.7.gz
-rw-r----- 1 root     root    104442374 Jul 20 16:00 bycast.log.6.gz
-rw-r----- 1 root     root    105747418 Aug 10 15:00 bycast.log.5.gz
-rw-r----- 1 root     root    105832058 Aug 31 18:00 bycast.log.4.gz
-rw-r----- 1 root     root    105523073 Sep 21 20:00 bycast.log.3.gz
-rw-r----- 1 root     root    105568607 Sep 28 22:00 bycast.log.2.gz
-rw-r----- 1 root     root     67548330 Oct  02 12:59 bycast-err.log.1
-rw-r----- 1 root     root     231233800 Oct 12 15:33 bycast-err.log
-rw-r----- 1 root     root   1075737021 Oct 02 22:00 bycast.log.1
-rw-r----- 1 root     root      52335833898 Oct 12 23:13 bycast.log

Sign in to view the entire content of this KB article.

New to NetApp?

Learn more about our award-winning Support

NetApp provides no representations or warranties regarding the accuracy or reliability or serviceability of any information or recommendations provided in this publication or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS and the use of this information or the implementation of any recommendations or techniques herein is a customer's responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.