Comment 0 for bug 1914258

Revision history for this message
Mihnea Saracin (msaracin) wrote :

Brief Description
-----------------
During restore of system controller the boot ISO is moved to /opt/dc-vault while on the / file system.

Severity
--------
Major

Steps to Reproduce
------------------
Run the restore platform procedure

Expected Behavior
------------------
The restore run is successful

Actual Behavior
----------------
Restore fails due to no space left

Reproducibility
---------------
100% if we have large files int /opt/dc-vault

System Configuration
--------------------
DC system (AIO-DX system controller)

Branch/Pull Time/Commit
-----------------------
stx master build on "2020-01-07"

Last Pass
---------
Didn't test it before

Timestamp/Logs
--------------

# Ansible log of the playbook:

TASK [restore-platform/restore-more-data : Restore dc-vault filesystem] **********************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["tar", "-C", "/", "--overwrite", "-xpf", "/home/sysadmin/cr2_platform_backup_2021_01_07_17_22_03.tgz", "opt/dc-vault"], "delta": "0:00:32.374888", "end": "2021-01-07 20:27:32.168301", "msg": "non-zero return code", "rc": 2, "start": "2021-01-07 20:26:59.793413", "stderr": "tar: opt/dc-vault/loads/20.06/bootimage1.iso: Wrote only 7168 of 10240 bytes\ntar: opt/dc-vault/lost+found: Cannot mkdir: No space left on device\ntar:

# Reason why it failed

Restore on central controller fails due to "No space left on device" /opt/dc-vault pushes / over it's 20G size limit:

/ file system sized to 20G
O/S install consumes 11G of /
Backup file placed in /home/sysadmin consumes 7G of /
Restore initiated
Boot ISO placed in /opt/dc-vault consumes just over 2G of /
Restore fails due to no space left

# df of /opt/dc-vault after failed restore.
localhost:~$ df -BG /opt/dc-vault/
Filesystem 1G-blocks Used Available Use% Mounted on
/dev/sda4 20G 20G 0G 100% /

Test Activity
-------------
Normal use