TASK [os_horizon : Enable project panels] fails

[Oliver Kaufmann]

Environment is CentOS 7.6, OSA version is 18.1.1, failure occurs within setup-openstack.yml. Same error occured with CentOS 7.5 and OSA 18.0.0

[...]
ok: [infra2_horizon_container-bd0de810] => (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/enabled/_1650_project_template_generator_panel.py', u'state': u'file', u'path': u'/usr/share/openstack-dashboard/openstack_dashboard/local/enabled/_1650_project_template_generator_panel.py'})
failed: [infra1_horizon_container-be36b28a] (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/local_settings.d/_1699_orchestration_settings.py', u'state': u'file', u'path': u'/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py'}) => {"changed": false, "item": {"path": "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py", "src": "/usr/share/openstack-dashboard/heat_dashboard/local_settings.d/_1699_orchestration_settings.py", "state": "file"}, "msg": "file (/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py) is absent, cannot continue", "path": "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py", "state": "absent"}
failed: [infra3_horizon_container-f1cf2e35] (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/local_settings.d/_1699_orchestration_settings.py', u'state': u'file', u'path': u'/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py'}) => {"changed": false, "item": {"path": "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py", "src": "/usr/share/openstack-dashboard/heat_dashboard/local_settings.d/_1699_orchestration_settings.py", "state": "file"}, "msg": "file (/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py) is absent, cannot continue", "path": "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py", "state": "absent"}
failed: [infra2_horizon_container-bd0de810] (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/local_settings.d/_1699_orchestration_settings.py', u'state': u'file', u'path': u'/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py'}) => {"changed": false, "item": {"path": "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py", "src": "/usr/share/openstack-dashboard/heat_dashboard/local_settings.d/_1699_orchestration_settings.py", "state": "file"}, "msg": "file (/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py) is absent, cannot continue", "path": "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py", "state": "absent"}
failed: [infra1_horizon_container-be36b28a] (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/conf/heat_policy.json', u'state': u'file', u'path': u'/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json'}) => {"changed": false, "item": {"path": "/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json", "src": "/usr/share/openstack-dashboard/heat_dashboard/conf/heat_policy.json", "state": "file"}, "msg": "file (/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json) is absent, cannot continue", "path": "/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json", "state": "absent"}
failed: [infra3_horizon_container-f1cf2e35] (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/conf/heat_policy.json', u'state': u'file', u'path': u'/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json'}) => {"changed": false, "item": {"path": "/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json", "src": "/usr/share/openstack-dashboard/heat_dashboard/conf/heat_policy.json", "state": "file"}, "msg": "file (/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json) is absent, cannot continue", "path": "/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json", "state": "absent"}
failed: [infra2_horizon_container-bd0de810] (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/conf/heat_policy.json', u'state': u'file', u'path': u'/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json'}) => {"changed": false, "item": {"path": "/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json", "src": "/usr/share/openstack-dashboard/heat_dashboard/conf/heat_policy.json", "state": "file"}, "msg": "file (/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json) is absent, cannot continue", "path": "/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json", "state": "absent"}
changed: [infra1_horizon_container-be36b28a] => (item={u'src': u'/usr/share/openstack-dashboard/designatedashboard/enabled/_1710_project_dns_panel_group.py', u'state': u'absent', u'path': u'/usr/share/openstack-dashboard/openstack_dashboard/local/enabled/_1710_project_dns_panel_group.py'})
[...]

Any help would be very much appreciated. thx in advance!

[Oliver Kaufmann (kaolli)]

sorry, i forgot: install method is 'distro'

[FScorp (fscorp)]

Just to say, seeing the same issue with CentOS Linux release 7.6.1810 (Core) and OSA 18.1.4. Also tried with the latest checkout of "stable/rocky" OSA with same result.

Attempted to build using "distro" install method and RDO packages, as well as "source" install, both with the same result.

Error is as follows:

[...]
failed: [infra1_horizon_container-059e6df0] (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/local_settings.d/_1699_orchestration_settings.py
', u'state': u'file', u'path': u'/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py'}) => {"changed"
: false, "item": {"path": "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py", "src": "/usr/share/o
penstack-dashboard/heat_dashboard/local_settings.d/_1699_orchestration_settings.py", "state": "file"}, "msg": "file (/usr/share/openstack-dashboard/opensta
ck_dashboard/local/local_settings.d/_1699_orchestration_settings.py) is absent, cannot continue", "path": "/usr/share/openstack-dashboard/openstack_dashboa
rd/local/local_settings.d/_1699_orchestration_settings.py", "state": "absent"}
ok: [infra2_horizon_container-386c2dd8] => (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/enabled/_1650_project_template_generator_panel.py
', u'state': u'file', u'path': u'/usr/share/openstack-dashboard/openstack_dashboard/local/enabled/_1650_project_template_generator_panel.py'})
failed: [infra1_horizon_container-059e6df0] (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/conf/heat_policy.json', u'state': u'file', u'pat
h': u'/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json'}) => {"changed": false, "item": {"path": "/usr/share/openstack-dashboard/op
enstack_dashboard/conf/heat_policy.json", "src": "/usr/share/openstack-dashboard/heat_dashboard/conf/heat_policy.json", "state": "file"}, "msg": "file (/us
r/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json) is absent, cannot continue", "path": "/usr/share/openstack-dashboard/openstack_dashboard/conf/heat_policy.json", "state": "absent"}
failed: [infra3_horizon_container-746d175f] (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/local_settings.d/_1699_orchestration_settings.py', u'state': u'file', u'path': u'/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py'}) => {"changed": false, "item": {"path": "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py", "src": "/usr/share/openstack-dashboard/heat_dashboard/local_settings.d/_1699_orchestration_settings.py", "state": "file"}, "msg": "file (/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py) is absent, cannot continue", "path": "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d/_1699_orchestration_settings.py", "state": "absent"}
failed: [infra2_horizon_container-386c2dd8] (item={u'src': u'/usr/share/openstack-dashboard/heat_dashboard/local_settings.d/_1699_orchestration_settings.py', u'state': u'file', u'path': u'/usr/s...

[Shannon Mitchell (shannon-mitchell)]

I just hit the same issue. Here are some notes I was dumping to when looking over the issue:

It looks like horizon is bombing when trying to set up project panels under /usr/share/openstack-dashboard as it doesn't exist. It look like the horizon lib dir was modified to support package installs. Recent python3 changes modify the horizon_lib_dir path without modifying the horizon_dashboard_panel_dir to match on source based installs.

# Example error:

TASK [os_horizon : Enable project panels] *************************************************************************************************************************************************************************
failed: [infra1_horizon_container-63cf055a] (item={'uid': 0, 'woth': False, 'mtime': 1571255171.4404233, 'inode': 1066421, 'isgid': False, 'size': 1070, 'roth': True, 'isuid': False, 'isreg': True, 'pw_name': 'r
oot', 'gid': 0, 'ischr': False, 'wusr': True, 'xoth': False, 'rusr': True, 'nlink': 1, 'issock': False, 'rgrp': True, 'gr_name': 'root', 'path': '/openstack/venvs/horizon-20.0.0.0b2.dev27/lib/python3.6/dist-pack
ages/heat_dashboard/enabled/_1640_project_template_versions_panel.py', 'xusr': False, 'atime': 1571255210.9088569, 'isdir': False, 'ctime': 1571255211.092859, 'isblk': False, 'xgrp': False, 'dev': 64772, 'wgrp':
 False, 'isfifo': False, 'mode': '0644', 'islnk': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"atime": 1571255210.9088569, "ctime": 1571255211.092859, "dev": 64772, "gid": 0, "gr_name": "r
oot", "inode": 1066421, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0644", "mtime": 1571255171.440423
3, "nlink": 1, "path": "/openstack/venvs/horizon-20.0.0.0b2.dev27/lib/python3.6/dist-packages/heat_dashboard/enabled/_1640_project_template_versions_panel.py", "pw_name": "root", "rgrp": true, "roth": true, "rus
r": true, "size": 1070, "uid": 0, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, "msg": "Error while linking: [Errno 2] No such file or directory", "path": "/usr/share/
openstack-dashboard/openstack_dashboard/local/enabled/_1640_project_template_versions_panel.py"}

# Related commit adding package support: https://github.com/openstack/openstack-ansible-os_horizon/commit/2af1b7a6c4644e05b41fcdad5d546dc4662560ff
# Related python3 commit breaking panel tasks: https://github.com/openstack/openstack-ansible-os_horizon/blame/3018868844c6d78a2120f0fba83b41596e6b2dfb/tasks/horizon_install_source.yml#L81

# Found in /etc/ansible/roles/os_horizon/defaults/main.yml

horizon_lib_dir: "{{ _horizon_lib_dir }}"

# Found in /etc/ansible/roles/os_horizon/vars/debian.yml

_horizon_lib_dir: "/usr/share/openstack-dashboard"
horizon_dashboard_panel_dir: "{{ _horizon_lib_dir }}/openstack_dashboard/local/enabled"

# Found in /etc/ansible/roles/os_horizon/tasks/horizon_install_source.yml

    !!!! (This overrides what is set in the defaults, but leaves horizon_dashboard_panel_dir pointing to the old directory.)

- name: Set python lib dir fact
  set_fact:
    horizon_lib_dir: "{{ _horizon_python_venv...

[Shannon Mitchell (shannon-mitchell)]

Actually these look like different issues due to the same set of changes: https://github.com/openstack/openstack-ansible-os_horizon/commit/20e47c9e4b7bcf564905c3b8b090d1b31be8f523

[Shannon Mitchell (shannon-mitchell)]

As far as I'm aware the 'Enable project panels' task only happens on 'source' installs. I created a new ticket just for the source install issues.

https://bugs.launchpad.net/openstack-ansible/+bug/1848561

I'm leaving this alone in case something else is going on with the install method and centos.