Building FUEL ISO: make clean && make iso leads to strange errors

[TorstenSchlabach]

Something seems to have changed in the build system recently. I am using fuel-master cloned on March 18th, 2015.

Before March 18th, I was used to the normal procedure of

1. setup my development environment
2. make iso
=> works fine

Do some changes.

make clean && make iso
=> works fine

Test my changes.

Now when I try the same, I end up with

genisoimage: Error: /home/torsten/fuel-master-tascel/fuel-main/build/iso/isoroot-mkisofs/Packages/openstack-neutron-linuxbridge-2014.2.2-fuel6.1.mira17.noarch.rpm.1 and /home/torsten/fuel-master-tascel/fuel-main/build/iso/isoroot-mkisofs/Packages/openstack-neutron-linuxbridge-2014.2.2-fuel6.1.mira17.noarch.rpm have the same Joliet name
genisoimage: Error: /home/torsten/fuel-master-tascel/fuel-main/build/iso/isoroot-mkisofs/Packages/openstack-neutron-oneconvergence-nvsd-2014.2.2-fuel6.1.mira17.noarch.rpm.1 and /home/torsten/fuel-master-tascel/fuel-main/build/iso/isoroot-mkisofs/Packages/openstack-neutron-oneconvergence-nvsd-2014.2.2-fuel6.1.mira17.noarch.rpm have the same Joliet name
genisoimage: Error: /home/torsten/fuel-master-tascel/fuel-main/build/iso/isoroot-mkisofs/Packages/openstack-neutron-openvswitch-2014.2.2-fuel6.1.mira17.noarch.rpm and /home/torsten/fuel-master-tascel/fuel-main/build/iso/isoroot-mkisofs/Packages/openstack-neutron-openvswitch-2014.2.2-fuel6.1.mira17.noarch.rpm.1 have the same Joliet name
Joliet tree sort failed. The -joliet-long switch may help you.
make: *** [/home/torsten/fuel-master-tascel/fuel-main/build/artifacts/fuel-6.1.iso] Error 1

I can do a

make deep_clean && make iso

successfully afterwards, but that means I will re-download all the Ubuntu and CentOS repos every time, which is very cumbersome and a waste of time and bandwith. And I assume this wasn't the idea of make clean at all, was it?

[Nick Sun (sunxin3)]

I also hit this issue. :(

openstack-neutron-linuxbridge-2014.2.2-fuel6.1.mira17.noarch.rpm.1
openstack-neutron-oneconvergence-nvsd-2014.2.2-fuel6.1.mira17.noarch.rpm
openstack-neutron-openvswitch-2014.2.2-fuel6.1.mira17.noarch.rpm.1

The length of 3 above packages'name are more than 64 characters, while the Joliet iso format only support the file which name is less than 64 character, I think we may add the switch : -joliet-long to the command: mkisofs for support so long rpm's file name.

Thanks!
Nick

[TorstenSchlabach (tschlabach)]

I think when it comes to the names ending in rpm.1 they are duplicate downloads which should be avoided in the first place as they will blow up the image size for nothing. They may be a related to bug #1437265; not sure.

When it comes to openstack-neutron-oneconvergence-nvsd-2014.2.2-fuel6.1.mira17.noarch.rpm, as this is something developed as part of the OpenStack project, maybe one could agree on a shorter name.

Otherwise the next problem one may hit is URL lengths for downloading the RPMs from somewhere.

[Alexei Sheplyakov (asheplyakov)]

Please run `make deep_clean' before each ISO build.

> but that means I will re-download all the Ubuntu and CentOS repos every time

Please use a HTTP proxy and set the http_proxy *and* HTTP_PROXY environment variables accordingly.

sudo apt-get install -y polipo
http_proxy=http://localhost:8123 HTTP_PROXY=http://localhost:8123 make -j iso

[Alexei Sheplyakov (asheplyakov)]

Fixing this particular problem (duplicate files downloaded by wget) won't change anything:
if the ISO build fails one needs to run make deep_clean before retrying. It's just that simple.

Many (sub)tasks (creating a chroot/container/etc) have a lot of implicit global state
(loopback devices, tmpfs or procfs mounts, lots of caches - ~/.npm, ~/.rpm,
/var/tmp/yum-*). As a consequence if the subtask X fails there's no way to revert
the global state to the "known good" one (i.e. before running X). The best we can
do is reverting the state to that *before* the build (which is what make deep_clean
is supposed to do). Therefore any ISO built without a proper cleanup should be
considered broken.

[Vladimir Sharshov (vsharshov)]

We have workaround for this problem. Moved it to 8.0

[Roman Vyalov (r0mikiam)]

please use make deep_clean