Failed to start cassandra service when loading metadata from cassandra database tables
Applies to
StorageGRID Webscale 10.3
Issue
The following errors are seen when booting a storage node in /var/local/log/cassandra/system.log:
DEBUG [main] 2018-01-23 02:41:54,011 CommitLogAllocator.java (line 287) Total active commitlog segment space used is 33554432
DEBUG [main] 2018-01-23 02:41:54,012 CommitLog.java (line 340) Active segment is now CommitLogSegment(/var/local/lib/cassandra/commitlog/CommitLog-3-1516675314010.log)
DEBUG [main] 2018-01-23 02:41:54,027 Directories.java (line 530) No level manifest found
...............................
...............................
DEBUG [main] 2018-01-23 02:41:54,031 Directories.java (line 530) No level manifest found
DEBUG [main] 2018-01-23 02:41:54,036 SSTableMetadata.java (line 437) Load metadata for /var/local/cassandra/data/0/ilmqueue/ilm_queue/ilmqueue-ilm_queue-jb-224178
DEBUG [main] 2018-01-23 02:41:54,036 SSTableMetadata.java (line 437) Load metadata for /var/local/cassandra/data/0/ilmqueue/ilm_queue/ilmqueue-ilm_queue-jb-206225
DEBUG [main] 2018-01-23 02:41:54,036 SSTableMetadata.java (line 437) Load metadata for /var/local/cassandra/data/0/ilmqueue/ilm_queue/ilmqueue-ilm_queue-jb-224179
DEBUG [main] 2018-01-23 02:41:54,037 SSTableMetadata.java (line 437) Load metadata for /var/local/cassandra/data/0/ilmqueue/ilm_queue/ilmqueue-ilm_queue-jb-224180
DEBUG [main] 2018-01-23 02:41:54,037 SSTableMetadata.java (line 437) Load metadata for /var/local/cassandra/data/0/ilmqueue/ilm_queue/ilmqueue-ilm_queue-jb-119971
DEBUG [main] 2018-01-23 02:41:54,037 SSTableMetadata.java (line 437) Load metadata for /var/local/cassandra/data/0/ilmqueue/ilm_queue/ilmqueue-ilm_queue-jb-81540
DEBUG [main] 2018-01-23 02:41:54,037 SSTableMetadata.java (line 437) Load metadata for /var/local/cassandra/data/0/ilmqueue/ilm_queue/ilmqueue-ilm_queue-jb-224145
DEBUG [main] 2018-01-23 02:41:54,037 SSTableMetadata.java (line 437) Load metadata for /var/local/cassandra/data/0/ilmqueue/ilm_queue/ilmqueue-ilm_queue-jb-224197
DEBUG [main] 2018-01-23 02:41:54,038 SSTableMetadata.java (line 437) Load metadata for /var/local/cassandra/data/0/ilmqueue/ilm_queue/ilmqueue-ilm_queue-jb-224196
DEBUG [main] 2018-01-23 02:41:54,038 SSTableMetadata.java (line 437) Load metadata for /var/local/cassandra/data/0/ilmqueue/ilm_queue/ilmqueue-ilm_queue-jb-129152
ERROR [main] 2018-01-23 02:41:54,038 CassandraDaemon.java (line 571) Exception encountered during startup
java.lang.NegativeArraySizeException
at org.apache.cassandra.utils.EstimatedHistogram$EstimatedHistogramSerializer.deserialize(EstimatedHistogram.java:335)
at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:463)
at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:448)
at org.apache.cassandra.io.sstable.SSTableMetadata$SSTableMetadataSerializer.deserialize(SSTableMetadata.java:432)
at org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:558)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:344)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:554)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:643)