StorageGRID rangedb offline due to being 100 percent full and unable to mount
Applies to
- StorageGRID 11.7.0.8 and below
- StorageGRID 11.8.0.4 and below
Issue
- One or more rangedb volume in offline state with 100% Stored value in Support > Grid Topology > Node > LDR > Storage
- StorageGRID alert
Storage volume offline present - miscd.log reports the following error:
miscd/miscd.log:[2024-01-01T01:18:13.011] VolStatusMonitor - ERROR -- Healthcheck on volume /var/local/rangedb/2 failed due to error: [Errno 28] No space left on device: '/var/local/rangedb/2/vma_healthcheck_file'
miscd/miscd.latest.log:[2024-01-01T01:18:13.011] VolStatusMonitor - ERROR -- Healthcheck on volume /var/local/rangedb/2 failed due to error: [Errno 28] No space left on device: '/var/local/rangedb/2/vma_healthcheck_file'
OR
[2025-03-26T08:39:48.427] VolStatusMonitor - DEBUG -- VolStatusMonitor: Executing target function
[2025-03-26T08:39:48.451] VolStatusMonitor - ERROR -- Healthcheck on volume /var/local/rangedb/6 failed due to error: [Errno 5] Input/output error: '/var/local/rangedb/6/vma_healthcheck_file'
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/miscd/threads/volume_status_monitor.py", line 126, in get_current_vol_status
with open(test_file, "w") as f:
OSError: [Errno 5] Input/output error: '/var/local/rangedb/6/vma_healthcheck_file'
[2025-03-26T08:39:48.585] VolStatusMonitor - DEBUG -- Volume /var/local/rangedb/6 is not healthy; unmounting
[2025-03-26T08:39:48.585] VolStatusMonitor - DEBUG -- Running command 'umountof /var/local/rangedb/6'