failed to write (No space left on device) error in image creation via DIB

[Esha Seth]

I have installed DIB in a ppc64le openstack environment which is non- KVM. I have installed multiple dependencies in this env for DIB to work. Finally DIB does workin virtual env but I am running into no disk space issue with minimal or more elements always

DIB installed at:

(env) neo@neo:~/dib/env/bin$ pwd
/home/neo/dib/env/bin

df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 1202368 0 1202368 0% /dev
tmpfs 251328 42176 209152 17% /run
/dev/mapper/pvmvg-root 8726456 2525876 5734252 31% /
tmpfs 1256576 0 1256576 0% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
tmpfs 1256576 0 1256576 0% /sys/fs/cgroup
/dev/md0 475434 69793 376576 16% /boot
/dev/mapper/pvmvg-var 9710112 1950952 7242860 22% /var
/dev/mapper/pvmvg-tmp 967320 1304 899664 1% /tmp
/dev/mapper/pvmvg-home 6025344 756784 4939448 14% /home
tmpfs 251328 0 251328 0% /run/user/1000

Running commands similar to
(env)/dib/env/bin$ sudo ./disk-image-create -a ppc64el -o ubuntu-vm3 -t raw -x vm ubuntu --logfile diblog.txt

Getting error exit 1

2018-04-24 09:26:18.661 | > Unpacking linux-image-4.4.0-121-generic (4.4.0-121.145) ...
2018-04-24 09:26:18.661 | > dpkg: error processing archive /var/cache/apt/archives/linux-image-4.4.0-121-generic_4.4.0-121.145_ppc64el.deb (--unpack):
2018-04-24 09:26:18.661 | > cannot copy extracted data for './lib/modules/4.4.0-121-generic/kernel/drivers/scsi/libiscsi_tcp.ko' to '/lib/modules/4.4.0-121-generic/kernel/drivers/scsi/libiscsi_tcp.ko.dpkg-new': failed to write (No space left on device)
2018-04-24 09:26:18.661 | > No apport report written because the error message indicates a disk full error
2018-04-24 09:26:18.661 | > dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
2018-04-24 09:26:18.661 | > Examining /etc/kernel/postrm.d .
2018-04-24 09:26:18.661 | > run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-121-generic /boot/vmlinux-4.4.0-121-generic
2018-04-24 09:26:18.661 | > Selecting previously unselected package linux-image-extra-4.4.0-121-generic.
2018-04-24 09:26:18.661 | > Preparing to unpack .../linux-image-extra-4.4.0-121-generic_4.4.0-121.145_ppc64el.deb ...
2018-04-24 09:26:18.661 | > Unpacking linux-image-extra-4.4.0-121-generic (4.4.0-121.145) ...
2018-04-24 09:26:18.661 | > dpkg: error processing archive /var/cache/apt/archives/linux-image-extra-4.4.0-121-generic_4.4.0-121.145_ppc64el.deb (--unpack):

Attaching log

[Ian Wienand (iwienand)]

It is probably trying to build in a tmpfs? See https://docs.openstack.org/diskimage-builder/latest/user_guide/building_an_image.html#speedups

I suggest exporting DIB_NO_TMPFS=1 into the environment to stop this

This has been a point of confusion before. I'm wondering if we should turn this off by default

[Esha Seth (eshaseth)]

Used following in dib virtual env

sudo DIB_NO_TMPFS=1 ./disk-image-create -no-tmpfs -a ppc64el -o ubuntu-vm1 -t raw -x vm ubuntu --logfile diblogvm1.txt

Still running into same error of disk space.

Attaching log?field.comment=Used following in dib virtual env

sudo DIB_NO_TMPFS=1 ./disk-image-create -no-tmpfs -a ppc64el -o ubuntu-vm1 -t raw -x vm ubuntu --logfile diblogvm1.txt

Still running into same error of disk space.

Attaching log

[Gregory Haynes (greghaynes)]

At this point in the build its just running in a chroot so either the tmpfs disabling must not be working and you are running out of space there or you are actually running out of disk space on a fs.

Can you try running with check_break=after-error set as well? This should drop you in to a shell at the point of failure. If this works could you check / paste the output of 'df -h'.

[Esha Seth (eshaseth)]

 sudo DIB_NO_TMPFS=1 ./disk-image-create -a ppc64el -o ubuntu-vm1 -t raw -x --no-tmpfs vm ubuntu --logfile diblogvm2.txt

I see --no-tmpfs not to be working
It needs 547MB and 4GB RAM

Filesystem Size Used Avail Use% Mounted on
udev 1.2G 0 1.2G 0% /dev
tmpfs 246M 42M 205M 17% /run
/dev/mapper/pvmvg-root 8.4G 2.5G 5.5G 31% /
tmpfs 1.2G 0 1.2G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 1.2G 0 1.2G 0% /sys/fs/cgroup
/dev/md0 465M 69M 368M 16% /boot
/dev/mapper/pvmvg-var 9.3G 1.9G 6.9G 22% /var
/dev/mapper/pvmvg-tmp 945M 1.3M 879M 1% /tmp
/dev/mapper/pvmvg-home 5.8G 797M 4.7G 15% /home
tmpfs 246M 0 246M 0% /run/user/1000

[Esha Seth (eshaseth)]

in reply to comment 4

(env) neo@p8-gd-115-neo:~/dib/env/bin$ sudo DIB_NO_TMPFS=1 ./disk-image-create -a ppc64el -o ubuntu-vm4 -t raw -x check_break=after-error --no-tmpfs vm ubuntu --logfile diblogvm4.txt

2018-04-28 11:25:13.431 | Traceback (most recent call last):
2018-04-28 11:25:13.431 | File "/home/neo/dib/env/bin/element-info", line 10, in <module>
2018-04-28 11:25:13.431 | sys.exit(main())
2018-04-28 11:25:13.431 | File "/home/neo/dib/diskimage-builder/diskimage_builder/element_dependencies.py", line 342, in main
2018-04-28 11:25:13.431 | elements = _get_elements(args.elements)
2018-04-28 11:25:13.431 | File "/home/neo/dib/diskimage-builder/diskimage_builder/element_dependencies.py", line 253, in _get_elements
2018-04-28 11:25:13.431 | return _expand_element_dependencies(elements, all_elements)
2018-04-28 11:25:13.431 | File "/home/neo/dib/diskimage-builder/diskimage_builder/element_dependencies.py", line 148, in _expand_element_dependencies
2018-04-28 11:25:13.431 | raise MissingElementException("Element '%s' not found" % element)
2018-04-28 11:25:13.432 | diskimage_builder.element_dependencies.MissingElementException: Element 'check_break=after-error' not found

(env) neo@p8-gd-115-neo:~/dib/env/bin$ sudo DIB_NO_TMPFS=1 ./disk-image-create -a ppc64el -o ubuntu-vm4 -t raw -x -check_break=after-error --no-tmpfs vm ubuntu --logfile diblogvm4.txt
disk-image-create: invalid option -- 'e'
disk-image-create: invalid option -- 'k'
disk-image-create: invalid option -- '_'
disk-image-create: invalid option -- 'b'
disk-image-create: invalid option -- 'r'
disk-image-create: invalid option -- 'e'

(env) neo@p8-gd-115-neo:~/dib/env/bin$ sudo DIB_NO_TMPFS=1 ./disk-image-create -a ppc64el -o ubuntu-vm4 -t raw -x --check_break=after-error --no-tmpfs vm ubuntu --logfile diblogvm4.txt
disk-image-create: unrecognized option '--check_break=after-error'

check_break=after_error doesnt seem to work @greghaynes

[Esha Seth (eshaseth)]

--no-tmpfs doesnt work

I pointed my /tmp to a bigger volume space and continued

[Esha Seth (eshaseth)]

2018-04-30 04:53:58.884 | > + apt-get --option Dpkg::Options::=--force-confold --option Dpkg::Options::=--force-confdef --assume-yes install dkms linux-image-generic
2018-04-30 04:53:58.884 | > Reading package lists...
2018-04-30 04:53:58.884 | > Building dependency tree...
2018-04-30 04:53:58.884 | > Reading state information...
2018-04-30 04:53:58.884 | > The following package was automatically installed and is no longer required:
2018-04-30 04:53:58.884 | > libfreetype6
2018-04-30 04:53:58.884 | > Use 'sudo apt autoremove' to remove it.
2018-04-30 04:53:58.884 | > The following additional packages will be installed:
2018-04-30 04:53:58.884 | > binutils cpp cpp-5 crda gcc gcc-5 iw libasan2 libatomic1 libcc1-0
2018-04-30 04:53:58.884 | > libgcc-5-dev libgomp1 libisl15 libitm1 libmpc3 libnl-3-200 libnl-genl-3-200
2018-04-30 04:53:58.884 | > libubsan0 linux-firmware linux-image-4.4.0-122-generic
2018-04-30 04:53:58.884 | > linux-image-extra-4.4.0-122-generic make wireless-regdb
2018-04-30 04:53:58.884 | > Suggested packages:
2018-04-30 04:53:58.884 | > binutils-doc cpp-doc gcc-5-locales gcc-multilib manpages-dev autoconf
2018-04-30 04:53:58.884 | > automake libtool flex bison gdb gcc-doc gcc-5-doc libgcc1-dbg libgomp1-dbg
2018-04-30 04:53:58.884 | > libitm1-dbg libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg
2018-04-30 04:53:58.884 | > libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg fdutils
2018-04-30 04:53:58.884 | > linux-doc-4.4.0 | linux-source-4.4.0 linux-tools
2018-04-30 04:53:58.884 | > linux-headers-4.4.0-122-generic make-doc
2018-04-30 04:53:58.884 | > Recommended packages:
2018-04-30 04:53:58.884 | > fakeroot libc6-dev | libc-dev libc6-dev grub-ieee1275
2018-04-30 04:53:58.884 | > The following NEW packages will be installed:
2018-04-30 04:53:58.885 | > binutils cpp cpp-5 crda dkms gcc gcc-5 iw libasan2 libatomic1 libcc1-0
2018-04-30 04:53:58.885 | > libgcc-5-dev libgomp1 libisl15 libitm1 libmpc3 libnl-3-200 libnl-genl-3-200
2018-04-30 04:53:58.885 | > libubsan0 linux-firmware linux-image-4.4.0-122-generic
2018-04-30 04:53:58.885 | > linux-image-extra-4.4.0-122-generic linux-image-generic make wireless-regdb
2018-04-30 04:53:58.885 | > 0 upgraded, 25 newly installed, 0 to remove and 0 not upgraded.
2018-04-30 04:53:58.885 | > Need to get 0 B/124 MB of archives.
2018-04-30 04:53:58.885 | > After this operation, 547 MB of additional disk space will be used.
2018-04-30 04:53:58.885 | > Selecting previously unselected package libmpc3:ppc64el.
2018-04-30 04:53:58.885 | > (Reading database ...
2018-04-30 04:53:58.885 | > (Reading database ... 5%
2018-04-30 04:53:58.885 | > (Reading database ... 10%
2018-04-30 04:53:58.885 | > (Reading database ... 15%
2018-04-30 04:53:58.885 | > (Reading database ... 20%
2018-04-30 04:53:58.885 | > (Reading database ... 25%
2018-04-30 04:53:58.885 | > (Reading database ... 30%
2018-04-30 04:53:58.885 | > (Reading database ... 35%
2018-04-30 04:53:58.885 | > (Reading database ... 40%
2018-04-30 04:53:58.885 | > (Reading database ... 45%
2018-04-30 04:53:58.885 | > (Reading database ... 50%
2018-04-30 04:53:58.885 | > (Reading database ... 55%
2018-04-30 04:53...