failed to write (No space left on device) error in image creation via DIB

Bug #1766540 reported by Esha Seth
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
diskimage-builder
New
Undecided
Unassigned

Bug Description

I have installed DIB in a ppc64le openstack environment which is non- KVM. I have installed multiple dependencies in this env for DIB to work. Finally DIB does workin virtual env but I am running into no disk space issue with minimal or more elements always

DIB installed at:

(env) neo@neo:~/dib/env/bin$ pwd
/home/neo/dib/env/bin

df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 1202368 0 1202368 0% /dev
tmpfs 251328 42176 209152 17% /run
/dev/mapper/pvmvg-root 8726456 2525876 5734252 31% /
tmpfs 1256576 0 1256576 0% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
tmpfs 1256576 0 1256576 0% /sys/fs/cgroup
/dev/md0 475434 69793 376576 16% /boot
/dev/mapper/pvmvg-var 9710112 1950952 7242860 22% /var
/dev/mapper/pvmvg-tmp 967320 1304 899664 1% /tmp
/dev/mapper/pvmvg-home 6025344 756784 4939448 14% /home
tmpfs 251328 0 251328 0% /run/user/1000

Running commands similar to
(env)/dib/env/bin$ sudo ./disk-image-create -a ppc64el -o ubuntu-vm3 -t raw -x vm ubuntu --logfile diblog.txt

Getting error exit 1

2018-04-24 09:26:18.661 | > Unpacking linux-image-4.4.0-121-generic (4.4.0-121.145) ...
2018-04-24 09:26:18.661 | > dpkg: error processing archive /var/cache/apt/archives/linux-image-4.4.0-121-generic_4.4.0-121.145_ppc64el.deb (--unpack):
2018-04-24 09:26:18.661 | > cannot copy extracted data for './lib/modules/4.4.0-121-generic/kernel/drivers/scsi/libiscsi_tcp.ko' to '/lib/modules/4.4.0-121-generic/kernel/drivers/scsi/libiscsi_tcp.ko.dpkg-new': failed to write (No space left on device)
2018-04-24 09:26:18.661 | > No apport report written because the error message indicates a disk full error
2018-04-24 09:26:18.661 | > dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
2018-04-24 09:26:18.661 | > Examining /etc/kernel/postrm.d .
2018-04-24 09:26:18.661 | > run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.0-121-generic /boot/vmlinux-4.4.0-121-generic
2018-04-24 09:26:18.661 | > Selecting previously unselected package linux-image-extra-4.4.0-121-generic.
2018-04-24 09:26:18.661 | > Preparing to unpack .../linux-image-extra-4.4.0-121-generic_4.4.0-121.145_ppc64el.deb ...
2018-04-24 09:26:18.661 | > Unpacking linux-image-extra-4.4.0-121-generic (4.4.0-121.145) ...
2018-04-24 09:26:18.661 | > dpkg: error processing archive /var/cache/apt/archives/linux-image-extra-4.4.0-121-generic_4.4.0-121.145_ppc64el.deb (--unpack):

Attaching log

Revision history for this message
Esha Seth (eshaseth) wrote :
Revision history for this message
Ian Wienand (iwienand) wrote :

It is probably trying to build in a tmpfs? See https://docs.openstack.org/diskimage-builder/latest/user_guide/building_an_image.html#speedups

I suggest exporting DIB_NO_TMPFS=1 into the environment to stop this

This has been a point of confusion before. I'm wondering if we should turn this off by default

Revision history for this message
Esha Seth (eshaseth) wrote :

Used following in dib virtual env

sudo DIB_NO_TMPFS=1 ./disk-image-create -no-tmpfs -a ppc64el -o ubuntu-vm1 -t raw -x vm ubuntu --logfile diblogvm1.txt

Still running into same error of disk space.

Attaching log?field.comment=Used following in dib virtual env

sudo DIB_NO_TMPFS=1 ./disk-image-create -no-tmpfs -a ppc64el -o ubuntu-vm1 -t raw -x vm ubuntu --logfile diblogvm1.txt

Still running into same error of disk space.

Attaching log

Revision history for this message
Gregory Haynes (greghaynes) wrote :

At this point in the build its just running in a chroot so either the tmpfs disabling must not be working and you are running out of space there or you are actually running out of disk space on a fs.

Can you try running with check_break=after-error set as well? This should drop you in to a shell at the point of failure. If this works could you check / paste the output of 'df -h'.

Revision history for this message
Esha Seth (eshaseth) wrote :

 sudo DIB_NO_TMPFS=1 ./disk-image-create -a ppc64el -o ubuntu-vm1 -t raw -x --no-tmpfs vm ubuntu --logfile diblogvm2.txt

I see --no-tmpfs not to be working
It needs 547MB and 4GB RAM

Filesystem Size Used Avail Use% Mounted on
udev 1.2G 0 1.2G 0% /dev
tmpfs 246M 42M 205M 17% /run
/dev/mapper/pvmvg-root 8.4G 2.5G 5.5G 31% /
tmpfs 1.2G 0 1.2G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 1.2G 0 1.2G 0% /sys/fs/cgroup
/dev/md0 465M 69M 368M 16% /boot
/dev/mapper/pvmvg-var 9.3G 1.9G 6.9G 22% /var
/dev/mapper/pvmvg-tmp 945M 1.3M 879M 1% /tmp
/dev/mapper/pvmvg-home 5.8G 797M 4.7G 15% /home
tmpfs 246M 0 246M 0% /run/user/1000

Revision history for this message
Esha Seth (eshaseth) wrote :

in reply to comment 4

(env) neo@p8-gd-115-neo:~/dib/env/bin$ sudo DIB_NO_TMPFS=1 ./disk-image-create -a ppc64el -o ubuntu-vm4 -t raw -x check_break=after-error --no-tmpfs vm ubuntu --logfile diblogvm4.txt

2018-04-28 11:25:13.431 | Traceback (most recent call last):
2018-04-28 11:25:13.431 | File "/home/neo/dib/env/bin/element-info", line 10, in <module>
2018-04-28 11:25:13.431 | sys.exit(main())
2018-04-28 11:25:13.431 | File "/home/neo/dib/diskimage-builder/diskimage_builder/element_dependencies.py", line 342, in main
2018-04-28 11:25:13.431 | elements = _get_elements(args.elements)
2018-04-28 11:25:13.431 | File "/home/neo/dib/diskimage-builder/diskimage_builder/element_dependencies.py", line 253, in _get_elements
2018-04-28 11:25:13.431 | return _expand_element_dependencies(elements, all_elements)
2018-04-28 11:25:13.431 | File "/home/neo/dib/diskimage-builder/diskimage_builder/element_dependencies.py", line 148, in _expand_element_dependencies
2018-04-28 11:25:13.431 | raise MissingElementException("Element '%s' not found" % element)
2018-04-28 11:25:13.432 | diskimage_builder.element_dependencies.MissingElementException: Element 'check_break=after-error' not found

(env) neo@p8-gd-115-neo:~/dib/env/bin$ sudo DIB_NO_TMPFS=1 ./disk-image-create -a ppc64el -o ubuntu-vm4 -t raw -x -check_break=after-error --no-tmpfs vm ubuntu --logfile diblogvm4.txt
disk-image-create: invalid option -- 'e'
disk-image-create: invalid option -- 'k'
disk-image-create: invalid option -- '_'
disk-image-create: invalid option -- 'b'
disk-image-create: invalid option -- 'r'
disk-image-create: invalid option -- 'e'

(env) neo@p8-gd-115-neo:~/dib/env/bin$ sudo DIB_NO_TMPFS=1 ./disk-image-create -a ppc64el -o ubuntu-vm4 -t raw -x --check_break=after-error --no-tmpfs vm ubuntu --logfile diblogvm4.txt
disk-image-create: unrecognized option '--check_break=after-error'

check_break=after_error doesnt seem to work @greghaynes

Revision history for this message
Esha Seth (eshaseth) wrote :

--no-tmpfs doesnt work

I pointed my /tmp to a bigger volume space and continued

Revision history for this message
Esha Seth (eshaseth) wrote :
Download full text (14.8 KiB)

2018-04-30 04:53:58.884 | > + apt-get --option Dpkg::Options::=--force-confold --option Dpkg::Options::=--force-confdef --assume-yes install dkms linux-image-generic
2018-04-30 04:53:58.884 | > Reading package lists...
2018-04-30 04:53:58.884 | > Building dependency tree...
2018-04-30 04:53:58.884 | > Reading state information...
2018-04-30 04:53:58.884 | > The following package was automatically installed and is no longer required:
2018-04-30 04:53:58.884 | > libfreetype6
2018-04-30 04:53:58.884 | > Use 'sudo apt autoremove' to remove it.
2018-04-30 04:53:58.884 | > The following additional packages will be installed:
2018-04-30 04:53:58.884 | > binutils cpp cpp-5 crda gcc gcc-5 iw libasan2 libatomic1 libcc1-0
2018-04-30 04:53:58.884 | > libgcc-5-dev libgomp1 libisl15 libitm1 libmpc3 libnl-3-200 libnl-genl-3-200
2018-04-30 04:53:58.884 | > libubsan0 linux-firmware linux-image-4.4.0-122-generic
2018-04-30 04:53:58.884 | > linux-image-extra-4.4.0-122-generic make wireless-regdb
2018-04-30 04:53:58.884 | > Suggested packages:
2018-04-30 04:53:58.884 | > binutils-doc cpp-doc gcc-5-locales gcc-multilib manpages-dev autoconf
2018-04-30 04:53:58.884 | > automake libtool flex bison gdb gcc-doc gcc-5-doc libgcc1-dbg libgomp1-dbg
2018-04-30 04:53:58.884 | > libitm1-dbg libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg
2018-04-30 04:53:58.884 | > libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg fdutils
2018-04-30 04:53:58.884 | > linux-doc-4.4.0 | linux-source-4.4.0 linux-tools
2018-04-30 04:53:58.884 | > linux-headers-4.4.0-122-generic make-doc
2018-04-30 04:53:58.884 | > Recommended packages:
2018-04-30 04:53:58.884 | > fakeroot libc6-dev | libc-dev libc6-dev grub-ieee1275
2018-04-30 04:53:58.884 | > The following NEW packages will be installed:
2018-04-30 04:53:58.885 | > binutils cpp cpp-5 crda dkms gcc gcc-5 iw libasan2 libatomic1 libcc1-0
2018-04-30 04:53:58.885 | > libgcc-5-dev libgomp1 libisl15 libitm1 libmpc3 libnl-3-200 libnl-genl-3-200
2018-04-30 04:53:58.885 | > libubsan0 linux-firmware linux-image-4.4.0-122-generic
2018-04-30 04:53:58.885 | > linux-image-extra-4.4.0-122-generic linux-image-generic make wireless-regdb
2018-04-30 04:53:58.885 | > 0 upgraded, 25 newly installed, 0 to remove and 0 not upgraded.
2018-04-30 04:53:58.885 | > Need to get 0 B/124 MB of archives.
2018-04-30 04:53:58.885 | > After this operation, 547 MB of additional disk space will be used.
2018-04-30 04:53:58.885 | > Selecting previously unselected package libmpc3:ppc64el.
2018-04-30 04:53:58.885 | > (Reading database ...
2018-04-30 04:53:58.885 | > (Reading database ... 5%
2018-04-30 04:53:58.885 | > (Reading database ... 10%
2018-04-30 04:53:58.885 | > (Reading database ... 15%
2018-04-30 04:53:58.885 | > (Reading database ... 20%
2018-04-30 04:53:58.885 | > (Reading database ... 25%
2018-04-30 04:53:58.885 | > (Reading database ... 30%
2018-04-30 04:53:58.885 | > (Reading database ... 35%
2018-04-30 04:53:58.885 | > (Reading database ... 40%
2018-04-30 04:53:58.885 | > (Reading database ... 45%
2018-04-30 04:53:58.885 | > (Reading database ... 50%
2018-04-30 04:53:58.885 | > (Reading database ... 55%
2018-04-30 04:53...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.