Rebooting StorageGRID storage node fails after applying hotfix 11.8.0.11
Applies to
- StorageGRID 11.8.0.11
- First reboot after the hotfix 11.8.0.11 was applied
Issue
- The first reboot of storage node after applying hotfix 11.8.0.11 does not complete
- MAINTENANCE > Software update > Apply hotfix indicates applying hotfix was successfully completed
gdu-server.log
on primary admin node indicates that hotfix 11.8.0.11 is successfully installed on the node
INFO -- gdu-server: Successfully installed StorageGRID hotfix 11.8.0.11 on <STORAGE_NODE>
/usr/local/servermanager/reader.rb
indicates adc and dds services areStarting...
Operating System Kernel 5.10.0 Verified
Operating System Environment Debian 11.8 Verified
StorageGRID Release 11.8.0 Verified
Storage Subsystem Verified
Network Monitoring 11.8.0 Stopped
Time Synchronization 1:4.2.8p15+dfsg Stopped
adc 11.8.0 Starting...
dds 11.8.0 Starting...
dmv 11.8.0 Stopped
ldr 11.8.0 Stopped
servermanager.log
indicates dds service is waiting for dynip
dds | restart initiated
dds | waiting for dynip, waiting 2s to try again
dds | waiting for dynip, waiting 4s to try again
dds | waiting for dynip, waiting 6s to try again
dds | waiting for dynip, waiting 8s to try again
dds | waiting for dynip, waiting 10s to try again
bycast.log
indicates that file/var/local/tmp/hotfix-stage/hotfix/dists/storagegrid/main/binary-amd64/Packages
causes failure of hotfix
#######################################
STARTING SERVICES
#######################################
Starting enhanced syslogd: rsyslogd.
Starting SSH
Starting OpenBSD Secure Shell server: sshd.
INFO -- Executing execute-hotfix.py with arguments: -src /var/local/hotfix/current-hotfix -destDir /var/local/tmp/hotfix-stage -cleanup -- --apply --no-start
INFO -- Compressed hotfix file detected.
INFO -- Extracting compressed hotfix file to path "/var/local/tmp/hotfix-stage"
INFO -- Executing command:tar -C /var/local/tmp/hotfix-stage -xf /var/local/hotfix/current-hotfix
INFO -- Executing hotfix file with options: --apply --no-start
INFO -- install: ===== /var/local/tmp/hotfix-stage/install called with arguments ["--apply", "--no-start"]
INFO -- install: Executing command `pkg_version bycast-cmn 2>&1` on localhost
INFO -- install: Executing command `sha256sum -c sha256sum 2>&1` on localhost
INFO -- install: Verified checksum
INFO -- install: Executing command `dpkg -s storage-grid-release 2>&1` on localhost
INFO -- install: Software version verified
INFO -- install: Executing command `/var/local/tmp/hotfix-stage/hotfix/pre-hotfix-install --apply --no-start 2>&1` on localhost
INFO -- install: Stopping services
INFO -- install: Executing command `/var/local/tmp/hotfix-stage/hotfix/sg-service-control --stop --services dynip,ldr,dds,ssm,dmv,mgmt-api,mi,gdu-server,miscd,keepalived,acct,kstn,stat,audit-analysis,chunk,delve,idnt,rsm,nginx-gw,adc,cmn,nms,ams,arc 2>&1` on localhost
INFO -- install: Running :stop on services ["acct", "adc", "chunk", "dds", "dmv", "dynip", "idnt", "kstn", "ldr", "miscd", "rsm", "ssm", "stat"]
INFO -- install: Stopping services
INFO -- install: Need to stop: []
INFO -- install: Success
INFO -- install: Executing command `gunzip /var/local/tmp/hotfix-stage/hotfix/dists/storagegrid/main/binary-amd64/Packages.gz 2>&1` on localhost
INFO -- install: gzip: /var/local/tmp/hotfix-stage/hotfix/dists/storagegrid/main/binary-amd64/Packages already exists;#011not overwritten
ERROR -- install: Error processing hotfix
ERROR -- install: Failed to execute sh command with 'gunzip /var/local/tmp/hotfix-stage/hotfix/dists/storagegrid/main/binary-amd64/Packages.gz 2>&1' on localhost - 2, gzip: /var/local/tmp/hotfix-stage/hotfix/dists/storagegrid/main/binary-amd64/Packages already exists;#011not overwritten
ERROR -- install: /usr/local/lib/site_ruby/bycast/run-cmd.rb:29:in `run_cmd'
ERROR -- install: /var/local/tmp/hotfix-stage/install:35:in `run_cmd'
ERROR -- install: /var/local/tmp/hotfix-stage/install:392:in `run_install'
ERROR -- install: /var/local/tmp/hotfix-stage/install:269:in `block in install'
ERROR -- install: /var/local/tmp/hotfix-stage/install:231:in `inside_tmp_dir'
ERROR -- install: /var/local/tmp/hotfix-stage/install:267:in `install'
ERROR -- install: /var/local/tmp/hotfix-stage/install:598:in `<main>'
Failed to execute sh command with 'gunzip /var/local/tmp/hotfix-stage/hotfix/dists/storagegrid/main/binary-amd64/Packages.gz 2>&1' on localhost - 2, gzip: /var/local/tmp/hotfix-stage/hotfix/dists/storagegrid/main/binary-amd64/Packages already exists;#011not overwritten
ERROR -- Hotfix file execution failed:
Hotfix file execution failed:
ERROR: Failed to install hotfix