[horizon] refresh_horizon_django_cache: /usr/share/openstack-dashboard/manage.py compress --force: IOError: [Errno 40] Too many levels of symbolic links

[Vladimir Khlyunev]

ISO 165
Several CI jobs failed with error
(/Stage[main]/Horizon/Exec[refresh_horizon_django_cache]) /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear && /usr/share/openstack-dashboard/manage.py compress --force returned 1 instead of one of [0]
(helpful log - https://product-ci.infra.mirantis.net/job/8.0.system_test.ubuntu.custom_hostname/43/artifact/logs/fail_error_set_custom_hostname-fuel-snapshot-2015-11-16_00-47-33.tar.xz.filtered.log )

One of failed jobs: https://product-ci.infra.mirantis.net/job/8.0.system_test.ubuntu.custom_hostname/43/
Steps:
Scenario:
1. Revert the snapshot
2. Create a cluster
3. Add 3 nodes with controller role
4. Add 1 node with compute role
5. Set custom hostnames for all cluster nodes
6. Deploy the cluster <- failed here

Snapshot
https://product-ci.infra.mirantis.net/job/8.0.system_test.ubuntu.custom_hostname/43/artifact/logs/fail_error_set_custom_hostname-fuel-snapshot-2015-11-16_00-47-33.tar.xz

Critical because it blocks part of tests.

UPD: Error from puppet.log from the controller: http://paste.openstack.org/show/478938/ pay attention on line 24

[Denis Egorenko (degorenko)]

This problem related to broken Horizon package.

For more information see: https://bugs.launchpad.net/mos/+bug/1512665

[Igor Yozhikov (iyozhikov)]

Root cause for this bug is - https://bugs.launchpad.net/mos/+bug/1512665, now we are reverting https://review.fuel-infra.org/#/c/13605/4 and investigating how to make "proper" package which will fit Debian policy .

[Thomas Goirand (thomas-goirand)]

This has been fixed in the latest version of libjs-jquery-ui in Debian. Please sync from it.

[Mikhail Ivanov (mivanov)]

Package was reverted with commit https://review.fuel-infra.org/#/c/13605/, so I think that we should close bug

[Vladimir Khlyunev (vkhlyunev)]

Not reproduced for 4 swarm runs - verified (ISO 195).

[Aleksei Stepanov (penguinolog)]

node-3.test.domain.local 2016-04-20T00:01:09.070600 err: (/Stage[main]/Horizon/Exec[refresh_horizon_django_cache]) Failed to call refresh: /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear && /usr/share/openstack-dashboard/manage.py compress --force returned 1 instead of one of [0]
node-3.test.domain.local 2016-04-20T00:01:09.070776 err: (/Stage[main]/Horizon/Exec[refresh_horizon_django_cache]) Failed to call refresh: /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear && /usr/share/openstack-dashboard/manage.py compress --force returned 1 instead of one of [0]
node-3.test.domain.local 2016-04-20T00:01:09.072493 err: (/Stage[main]/Horizon/Exec[refresh_horizon_django_cache]) /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear && /usr/share/openstack-dashboard/manage.py compress --force returned 1 instead of one of [0]
node-3.test.domain.local 2016-04-20T00:01:09.072786 err: (/Stage[main]/Horizon/Exec[refresh_horizon_django_cache]) /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear && /usr/share/openstack-dashboard/manage.py compress --force returned 1 instead of one of [0]

ISO fuel-9.0-220-2016-04-19_08-00-00.iso

[Aleksei Stepanov (penguinolog)]

Faile CI JOB https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.uca_neutron_ha/8/testReport/%28root%29/uca_neutron_ha/uca_neutron_ha/

[Dmitry Klenov (dklenov)]

Folks, can you please check if master is affected?

[Ivan Udovichenko (iudovichenko)]

I've checked it with our latest packages in 9.0 repository:
# dpkg -l | grep -P '(horizon|dashboard)'
ii horizon-vendor-theme 1.0.0~mos8-2~u14.04+mos2 all SCSS theme package for horizon
ii openstack-dashboard 2:9.0.0-4~u14.04+mos25 all web application to control an OpenStack cloud
ii openstack-dashboard-apache 2:9.0.0-4~u14.04+mos25 all web application to control an OpenStack cloud - Apache support
ii python-django-horizon 2:9.0.0-4~u14.04+mos25 all Django module providing web interaction with OpenStack

And "collectstatic" call works along with "compression".

Please provide more information which packages are used during test. According to the name of Jenkins job 9.0.system_test.ubuntu.uca_neutron_ha it is more likely that Ubuntu Cloud Archive is being used.

[Igor Yozhikov (iyozhikov)]

According to logs from failed tests packages were consumed from Canonical (UCA)
We are not responsible for packages maintained by Ubuntu people.

[Aleksei Stepanov (penguinolog)]

Fuel bug status -> Fix released, UCA: spike found