VSTU alarm is detected for the particular storage volume
Applies to
StorageGRID
Issue
- VSTU alarm is detected for the particular storage volume.
Example:
- GUI does not show any errors for the storage volume.
Example:
- IO error for the storage volume
/var/local/rangedb/X
is detected inbycast.log
. Below sample shows the problem volume is/var/local/rangedb/A.
Example:
Dec 22 10:30:01 StorageNode001 ADE: |12xxxx00 0873608202 SVFY ESCN 2020-12-22T10:30:01.029614| WARNING 0990 SVFY: Failed to verify '/var/local/rangedb/A/p/06/0E/7BCA139787Bxxxxxxx' with error 'Input/output error'. Deferring scan
Dec 22 10:30:01 StorageNode001 ADE: |12xxxx00 0932310809 SWTR #ARS 2020-12-22T10:30:01.030311| WARNING 0303 SWTR: CBID BE750F8EFFFxxxxxx: Operation (write) returned error result 'STIO'/'E005' - Shutting down.
Dec 22 10:30:01 StorageNode001 ADE: |12xxxx00 0932310809 SWTR #ARS 2020-12-22T10:30:01.030400| WARNING 0226 SWTR: CBID BE750F8EFFFxxxxxx: Error 'E005'
Dec 22 10:30:01 StorageNode001 ADE: |12xxxx00 0932310809 SWTR #ARI 2020-12-22T10:30:01.034474| WARNING 0303 SWTR: CBID BE750F8EFFFxxxxxx: Operation (fsync) returned error result 'STIO'/'E005' - Shutting down.
Dec 22 10:30:01 StorageNode001 ADE: |12xxxx00 0932310809 SWTR #ARI 2020-12-22T10:30:01.034555| WARNING 0273 SWTR: CBID BE750F8EFFFxxxxxx: Unable to unlink /var/local/rangedb/A/p/03/0E/00nj-0exxxxxxxxx, err=1160785973
Dec 22 10:30:01 StorageNode001 ADE: |12xxxx00 0932310809 SWTR #ARI 2020-12-22T10:30:01.034587| ERROR 0331 SWTR: CBID BE750F8EFFFxxxxxx: Write failed with I/O error 'E005'
Dec 22 10:30:01 StorageNode001 ADE: |12xxxx00 0873608199 STOR WFIN 2020-12-22T10:30:01.034602| ERROR 2144 STOR: Operation returned non-zero result 'E005' - Going offline.
Dec 22 10:30:01 StorageNode001 ADE: |12xxxx00 0873608199 STOR %DED 2020-12-22T10:30:01.034636| NOTICE 1684 STOR: Storage Writer Module destroyed (PID 932310809) (result 'STIO'/'E005')
Dec 22 10:30:01 StorageNode001 ADE: |12xxxx00 0873608251 RRET EMDT 2020-12-22T10:30:01.034636| WARNING 0762 RRET: Storage queue 0x7f75f9c2edf0 was unexpectedly destroyed
Dec 22 10:30:01 StorageNode001 ADE: |12xxxx00 0873608251 RRET ORTD 2020-12-22T10:30:01.034668| WARNING 0921 RRET: ObjectRetained for CBID BE750F8EFFFxxxxxx TID 3701655513205382255 had error 'E005'
- Filesystem error on dm-X is detected in
kern.log
ordmesg.txt
.
Example:
[Dec22 10:29] WARNING: CPU: 8 PID: 6853 at fs/xfs/libxfs/xfs_bmap.c:719 xfs_bmap_extents_to_btree+0x388/0x5a0 [xfs]
・・・・・・・
[ +0.003751] XFS (dm-8): xfs_do_force_shutdown(0x8) called from line 1042 of file fs/xfs/xfs_trans.c. Return address = 00000000xxxxx
[ +0.013203] XFS (dm-8): Corruption of in-memory data detected. Shutting down filesystem
[ +0.008252] XFS (dm-8): Please umount the filesystem and rectify the problem(s)
[ +0.007506] XFS (dm-8): writeback error on sector 3221228xxxx
[ +0.000391] XFS (dm-8): writeback error on sector 3221228xxxx