CIFS audit staging volume got full due to the adtconsolidation process not starting after Takeover Giveback
Applies to
- ONTAP 9
- CIFS auditing
Issue
- Some of the symptoms seen are as follows:
- No audit logs reported after any takeover/giveback operation which was done during the ONTAP upgrade.
- Some of the staging volumes have reached full capacity.
- EMS logs :
[?] Wed Aug 29 07:34:57 BST [cluster-2: wafl_exempt04: wafl.vol.full:notice]: Insufficient space on volumeMDV_aud_fdb78598bd5945ffa6f7bd1197a9f975@vserver:7a9e6a27-aa87-11e6-96cc-00a098a91817 to perform operation. 4.00KB was requested but only 1.00KB was available.
[?] Wed Aug 29 07:34:57 BST [cluster-2: AuditWorkerThread02: adt.stgvol.nospace:EMERGENCY]:Audit subsystem internal error: Staging volume MDV_aud_fdb78598bd5945ffa6f7bd1197a9f975 is full.
[?] Wed Aug 29 07:59:30 BST [cluster-2: AuditWorkerThread00: ems.engine.suppressed:debug]: Event 'adt.stgvol.nospace' suppressed 90 times in last 1397seconds.
[?] Wed Aug 29 07:59:30 BST [cluster-2: AuditWorkerThread00: adt.stgvol.nospace:EMERGENCY]: Audit subsysteminternal error: Staging volume MDV_aud_fdb78598bd5945ffa6f7bd1197a9f975 is full.
::> df MDV*
Filesystem kbytes used avail capacity Mounted on
/vol/MDV_aud_4a9d8065eac9454bbe042ffddd0df645/ 1992296 532 1991764 0% /vol/MDV_aud_4a9d8065eac9454bbe042ffddd0df645/
/vol/MDV_aud_62a9aebc8f3d4fe2990e39bb34c66999/ 1992296 384 1991912 0% /vol/MDV_aud_62a9aebc8f3d4fe2990e39bb34c66999/
/vol/MDV_aud_fdb78598bd5945ffa6f7bd1197a9f975/ 1992296 1992296 0 100% /vol/MDV_aud_fdb78598bd5945ffa6f7bd1197a9f975/ <<<<<< Full
/vol/MDV_aud_d63c9c461ab14609928999846af40f90/ 1992296 192 1992104 0% /vol/MDV_aud_d63c9c461ab14609928999846af40f90/
/vol/MDV_aud_a38d6a453935479d8e0a5a5413f986c3/ 1992296 204 1992092 0% /vol/MDV_aud_a38d6a453935479d8e0a5a5413f986c3/
- Disabling Audit fails:
::*> vserver audit disable -vserver svm01
Error: command failed: Failed to disable auditing for Vserver "svm01". Reason: Audit consolidation job not present for the Vserver and one of the possible reasons might be snapmirror break operation in Vserver DR setup. Retry after some time.
- CIFS share access is affected due to the above behavior.