What are ActiveIQ Unified Manager’s recording files and how to we parse them for troubleshooting?

Applies to

Active IQ Unified Manager (AIQUM)

Answer

Recording files are created at the time that a polling cycle is triggered within ActiveIQ Unified Manager to collected data from the Clustered Ontap Operating System.

• Recording files capture the raw API data that is actually received by ActiveIQ Unified Manager from the cluster.
• Foundation Recording files are created each time inventory Polling is performed.
  • Polling is once every 5 minutes by default.
• Performance Recording files are bundled into a ‘one recording per day’ zip file.
  • Performance Recording files will contain up to 5 polling cycle samples (Numbered folders) per recording file.
• Recording files provide one or more “Sample.log” file that records the status of the Polling Cycle success or failure.

Recording file Naming convention: LogType_CLUSTERNAME.corp.company.org_datasourceid_year_month_day_HR_MIN_SEC_XXX.zip
"Sample.log" file Naming convention: log_LogType_CLUSTERNAME.corp.company.org_91_year_month_day_HR_MIN_SEC_XXX.sample.log

• Recording “sample.log” is usually located within the “root” folder, however it can also reside within the Number folder; check both folders.

 

How to find Recording files within a Support Bundle
Within a Support Bundle, the recording files are located within folder: Support_bundle_DATE_HR_MIN_SEC\acquisition\recording

Looking for a Storage Performance Recording file that was created on the date that the issue was observed on.
EXAMPLE: netappstorageperformance_CLUSTER01.corp.company.org_01_2019_06_09_08_37_21_665.zip

• How to locate the Sample.log file
  • Check the root or numbered folder to identify the Sample.log file.
  • Open the sample.log file within any Text editor.   
  • This sample file will tell you whether the Performance Acquisition was successful, or failed.
  • If failed, the error will be captured within the sample.log file.

 

For convenience, here are examples of the recording file structure, both for netappfoundation and netappstorageperformance.

Netappfoundation:

netappfoundation_xx.xx.xx.xx_3_2019_05_30_11_51_22_243

├───1559242272012
│       description.xml
│       log_netappfoundation_xx.xx.xx.xx_91_2019_05_30_11_51_12_075_iterator_response_times.txt
│       log_netappfoundation_xx.xx.xx.xx_91_2019_05_30_11_51_12_076_zapi_attributes.txt
│       log_netappfoundation_xx.xx.xx.xx_91_2019_05_30_11_51_22_165_poll_time.txt
│       model_netappfoundation_xx.xx.xx.xx_91_2019_05_30_11_51_21_306.xml
│       netappdata_netappfoundation_xx.xx.xx.xx_91_2019_05_30_11_51_21_337_3.xml
│       recording_netappfoundation_xx.xx.xx.xx_91_2019_05_30_11_51_12_074_netappfoundation.dat
│
└───root
        configuration
        description.xml
        log_netappfoundation_xx.xx.xx.xx_91_2019_05_30_11_51_12_012_sample.log

Netappstorageperformance:

netappstorageperformance_xx.xx.xx.xx_3_2019_05_30_00_28_01_556
├───1559200078025
│       description.xml
│       log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_08_06_225_iterator_response_times.txt
│       log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_08_06_226_zapi_attributes.txt
│       log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_08_09_441_poll_time.txt
│       log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_08_13_303_poll_time.txt
│       log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_08_15_142_poll_time.txt
│       log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_08_25_691_poll_time.txt
│       netappfoundation_xx.xx.xx.xx_3_2019_05_30_00_06_28_852_copy.zip
│       recording_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_08_06_224_netappstorageperformance.dat
│
├───1559200378012
│       163185_000300_1559199900886_0270000.ccma.gz
│       163185_000300_1559199900886_0270000.ccma.md5
│       163185_000300_1559199900886_0270000.ccma.meta
│       163382_000300_1559199900552_0282000.ccma.gz
│       163382_000300_1559199900552_0282000.ccma.md5
│       163382_000300_1559199900552_0282000.ccma.meta
│       description.xml
│       log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_12_58_014_iterator_response_times.txt
│       log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_12_58_015_zapi_attributes.txt
│       log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_12_59_621_ocie_report_times.txt
│       log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_02_120_poll_time.txt
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_462_WorkloadQueueDBladeCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_495_WorkloadQueueNBladeCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_518_ResourceCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_555_WorkloadDetailCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_651_LifCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_718_NFSv3Counter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_735_WaflCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_763_ProcessorCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_784_DiskConfigCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_796_DiskCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_860_AggregateCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_914_HostadapterCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_920_NicCommonCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_01_983_CifsCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_02_011_NodeCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_02_034_ExtCacheObjCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_02_050_ResourceHeadRoomCPUCounter.csv
│       perfdata_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_13_02_077_ResourceHeadRoomAggrCounter.csv
│       recording_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_12_58_013_netappstorageperformance.dat
│
├─── <truncated 3 numbered folder outputs>
│
└───root
        configuration
        description.xml
        log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_07_58_025_sample.log
        log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_12_58_012_sample.log
        log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_17_58_031_sample.log
        log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_22_58_054_sample.log
        log_netappstorageperformance_xx.xx.xx.xx_91_2019_05_30_00_27_58_014_sample.log

Additional Information

Within the Storage Performance recording are copies of the CCMA files downloaded from the cluster.

• These CCMA files are not human readable and can only be parsed with special software tools.  The CCMA files contain the raw performance data from the cluster.
• The CCMA Files are located within the unix timestamp folder (aka the 'numbered' folder) in the Storage Performance Recording.
• Within each numbered folder, there should be a ccma.gz, a ccma.md5, and a ccma.meta file for each node in the cluster.
  • If any of these files are missing, it could mean that Unified Manager did not receive a CCMA file collection from one of the Cluster nodes.
• To save space in the Support Bundle, each daily Performance Recording file only includes 5 CCMA files out of the numerous Performance recordings that are captured throughout the day.