Comment 16 for bug 1529182

Revision history for this message
Michal Skalski (mskalski) wrote :

In my opinion size of the var partition is just too small. Moreover it is fixed on 10000MB [1] so there is no benefit of having bigger hard disk.

It is not only about snapshots. This partition by default is used for storing bootstrap and os installation images, local mirrors created with fuel-mirror command and installed plugins

There is a high chance that it will be overfilled during usual activities for example:

Disk usage after Fuel Master installation and bootstrap image creation:

[root@fuel ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/os-root 9.5G 2.0G 7.1G 23% /
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 17M 1.9G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/mapper/os-var 9.5G 4.9G 4.2G 54% /var
/dev/mapper/os-varlog 46G 74M 44G 1% /var/log
/dev/vda3 197M 108M 90M 55% /boot
/dev/vda2 200M 0 200M 0% /boot/efi

During environment deployment (os image creation)

[root@fuel nailgun]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/os-root 9.5G 2.0G 7.0G 23% /
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 17M 1.9G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/mapper/os-var 9.5G 6.3G 2.8G 70% /var
/dev/mapper/os-varlog 46G 89M 44G 1% /var/log
/dev/vda3 197M 108M 90M 55% /boot
/dev/vda2 200M 0 200M 0% /boot/efi

After environment deployment, and local mirrors creation with fuel-mirror command:

[root@fuel ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/os-root 9.5G 2.0G 7.0G 23% /
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 17M 1.9G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/mapper/os-var 9.5G 7.7G 1.4G 86% /var
/dev/mapper/os-varlog 46G 129M 44G 1% /var/log
/dev/vda3 197M 108M 90M 55% /boot
/dev/vda2 200M 0 200M 0% /boot/efi

During second environment deployment

[root@fuel ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/os-root 9.5G 2.0G 7.0G 23% /
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 17M 1.9G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/mapper/os-var 9.5G 9.1G 0 100% /var
/dev/mapper/os-varlog 46G 149M 44G 1% /var/log
/dev/vda3 197M 108M 90M 55% /boot
/dev/vda2 200M 0 200M 0% /boot/efi

Second deployment failed because var partition was overfilled and new image creation was not possible.

Fuel agent logs:

2016-01-29 22:26:48.006 29397 ERROR fuel_agent.cmd.agent [-] Unexpected error while running command.
Command: resize2fs -M /var/lib/fuel/ibp/tmpfMEEo_.fuel-agent-image
Exit code: 1
Stdout: 'Resizing the filesystem on /var/lib/fuel/ibp/tmpfMEEo_.fuel-agent-image to 324349 (4k) blocks.\n'
Stderr: "resize2fs 1.42.9 (28-Dec-2013)\nresize2fs: Attempt to write block to filesystem resulted in short write while trying to resize /var/lib/fuel/ibp/tmpfMEEo_.fuel-agent-image\nPlease run 'e2fsck -fy /var/lib/fuel/ibp/tmpfMEEo_.fuel-agent-image' to fix the filesystem\nafter the aborted resize operation.\n"
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent Traceback (most recent call last):
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent File "/usr/lib/python2.7/site-packages/fuel_agent/cmd/agent.py", line 101, in main
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent getattr(mgr, action)()
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent File "/usr/lib/python2.7/site-packages/fuel_agent/manager.py", line 999, in do_build_image
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent bu.shrink_sparse_file(image.img_tmp_file)
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent File "/usr/lib/python2.7/site-packages/fuel_agent/utils/build.py", line 346, in shrink_sparse_file
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent utils.execute('resize2fs', '-M', filename)
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent File "/usr/lib/python2.7/site-packages/fuel_agent/utils/utils.py", line 133, in execute
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent stderr=stderr, cmd=command)
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent ProcessExecutionError: Unexpected error while running command.
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent Command: resize2fs -M /var/lib/fuel/ibp/tmpfMEEo_.fuel-agent-image
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent Exit code: 1
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent Stdout: 'Resizing the filesystem on /var/lib/fuel/ibp/tmpfMEEo_.fuel-agent-image to 324349 (4k) blocks.\n'
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent Stderr: "resize2fs 1.42.9 (28-Dec-2013)\nresize2fs: Attempt to write block to filesystem resulted in short write while trying to resize /var/lib/fuel/ibp/tmpfMEEo_.fuel-agent-image\nPlease run 'e2fsck -fy /var/lib/fuel/ibp/tmpfMEEo_.fuel-agent-image' to fix the filesystem\nafter the aborted resize operation.\n"
2016-01-29 22:26:48.006 29397 TRACE fuel_agent.cmd.agent

[1] https://github.com/openstack/fuel-main/blob/master/iso/ks.template#L297