os_horizon role fails if horizon_custom_themes is specified

[Dmitriy Rabotyagov]

Description
===========

When horizon_custom_themes is specified in user_variables.yml horizon_post_install.yml fails at step "Collect and compress static files", as even directory for specified theme does not exists during initial configuration.
In the documentation it is stated, that custom theme should be placed on the horizon containers, but in {{ horizon_lib_dir }}/openstack_dashboard/themes, but this directory does not exist on the initial step (before running setup-openstack.yml)

How to reproduce
===========

Before running os_horizon role, specify some custom theme in user_variables.yml and run setup-openstack.yml playbook.

Ways to fix
===========
For instance, this could be fixed by adding additional step in horizon_post_install.yml right after dropping horizon customization module. For example it may be the following tasks:

  - name: Drop horizon custom themes
    synchronize:
      src: "{{ horizon_custom_theme_path }}/{{ item.value.theme_name }}/"
      dest: "{{ horizon_lib_dir }}/openstack_dashboard/{{ item.value.theme_path }}/"
      times: yes
      archive: no
      recursive: yes
      delete: yes
      checksum: yes
    with_dict: "{{ horizon_custom_themes }}"
    notify: Restart apache2

  - name: Setting horizon permissions
    file:
      path: "{{ horizon_lib_dir }}/openstack_dashboard/{{ item.value.theme_path }}"
      state: directory
      owner: "{{ horizon_system_user_name }}"
      group: "{{ horizon_system_group_name }}"
      recurse: yes
    with_dict: "{{ horizon_custom_themes }}"

But it requires an additional variable to be set in user_variables.yml, which is horizon_custom_theme_path. This variable is supposed to provide a local path for the custom themes to be placed in.

[Dmitriy Rabotyagov (noonedeadpunk)]

Sorry, I've forgotten about when directive. So the correct one should be the following:

- name: Drop horizon custom themes
  synchronize:
    src: "{{ horizon_custom_theme_path }}/{{ item.value.theme_name }}"
    dest: "{{ horizon_lib_dir }}/openstack_dashboard/{{ item.value.theme_path }}"
    times: yes
    archive: no
    recursive: yes
    delete: yes
    checksum: yes
  with_dict: "{{ horizon_custom_themes }}"
  notify: Restart apache2
  when: horizon_custom_themes is defined

- name: Setting horizon permissions
  file:
    path: "{{ horizon_lib_dir }}/openstack_dashboard/{{ item.value.theme_path }}"
    state: directory
    owner: "{{ horizon_system_user_name }}"
    group: "{{ horizon_system_group_name }}"
    recurse: yes
  with_dict: "{{ horizon_custom_themes }}"
  when: horizon_custom_themes is defined

[Guilherme Steinmuller Pimentel (guilhermesp)]

Hello! I reproduced the user story that Dmitriy R. stated in this bug. Thanks for that Dmitriy R. anyway!
I did it in one fresh osa-master installation.

Just a few notes:

1. First, I added a new empty variable in /etc/ansible/roles/os_horizon called "horizon_custom_theme_path:".

2. In /etc/openstack_deploy/ I created a directory called "horizon_custom_theme" and inside it I put a copy of the available themes in "{{ horizon_lib_dir }}/openstack_dashboard/themes" (just to see the files being copied i.e /etc/openstack_deploy/horizon_custom_theme/custom/* )

 obs: As stated here https://docs.openstack.org/horizon/latest/configuration/themes.html#inherit-from-an-existing-theme, the right way is to run "python manage.py collectstatic" after create your custom theme directory, but I didn't manage it to do it.

3. In /etc/openstack_deploy/user_variables.yml I added the following variables:

   - horizon_custom_theme_path: "/etc/openstack_deploy/horizon_custom_theme/"

   - horizon_default_theme: "custom"

   -horizon_custom_themes:
      custom_theme:
        theme_name: "custom"
        theme_label: "Custom"
        theme_path: "themes/"

4. In /etc/ansible/roles/os_horizon/tasks/horizon_post_install.yml I added the following task after "Drop horizon customization module" task, like:

- name: Drop horizon custom themes
  synchronize:
    src: "{{ horizon_custom_theme_path }}/{{ item.value.theme_name }}"
    dest: "{{ horizon_lib_dir }}/openstack_dashboard/{{ item.value.theme_path }}"
    times: yes
    archive: no
    recursive: yes
    delete: yes
    checksum: yes
  with_dict: "{{ horizon_custom_themes }}"
  notify: Restart apache2
  when: horizon_custom_themes is defined

  obs: I don't see reasons to add the task "Setting horizon permissions" right after "Drop horizon custom themes" task as the custom themes directories located in "{{ horizon_lib_dir }}/openstack_dashboard/themes" has root:root by default, didn't any effect changing to horizon:horizon, so my suggestion is to remove this task to avoid unnecessary tasks.

5. Then, I executed the setup-openstack.yml (with clean horizon container) and the custom theme were deployed correct in the right path inside horizon venv.

As mentioned in my 2nd step, I got the error bellow I think because I didn't manage to correct create the custom theme directory:

TASK [os_horizon : Ensure static files are collected and compressed] ********************************************************************************************************************
changed: [aio1_horizon_container-4d604984] => (item={'_ansible_no_log': False, 'item': '/openstack/venvs/horizon-18.0.0.0b3/bin/horizon-manage.py collectstatic --noinput', 'started': 1, '_ansible_ignore_errors': None, 'results_file': '/var/lib/horizon/.ansible_async/390504259645.12638', '_ansible_item_result': True, '_ansible_notify': ['Restart apache2'], 'ansible_job_id': '390504259645.12638', 'failed': False, 'changed': False, 'finished': 0, '_ansible_parsed': True})
FAILED - RETRYING: Ensure static files are collected and compressed (300 retries left).
FAILED - RETRYING: Ensure static files are collected and compressed (299 retries left).
FAILED - RE...

[Dmitriy Rabotyagov (noonedeadpunk)]

Hi,

I couldn't reproduce collection fail. In my case theme is placed under /opt/themes/ , like this:
root@eustackdeploy:/opt/openstack-ansible/playbooks# ls -l /opt/themes/
total 4
drwxr-xr-x 3 techs techs 4096 Jul 4 17:47 my_theme
root@eustackdeploy:/opt/openstack-ansible/playbooks# ls -l /opt/themes/my_theme
total 64
drwxr-xr-x 3 techs techs 4096 Jul 4 17:47 horizon
-rw-r--r-- 1 techs techs 2677 Jul 4 17:47 logo_51x35_transp.png
-rw-r--r-- 1 techs techs 6692 Jul 4 17:47 _styles.scss
-rw-r--r-- 1 techs techs 29836 Jul 4 17:47 _variables.scss
-rw-r--r-- 1 techs techs 4336 Jul 4 17:47 login-bg.png
-rw-r--r-- 1 techs techs 5105 Jul 4 17:47 login-logo.png
root@eustackdeploy:/opt/openstack-ansible/playbooks#

And variable set like this:
horizon_custom_theme_path: /opt/themes

But I've found another drowbacks in my approach:
1. host with ansible MUST have connection to the container itself (as synchronize launches rsync from ansible host). For example, we in production had access only to lxc hosts via management network (and surprisingly that worked).
2. If you have non-standart SSH port on hosts (i.e. ansible-port is set), synchronize will fail as well, as containers are created with default ssh port (22). As a workaround dest_port: 22 might be hardcoded in synchronize task (unless lxc containers' SSH configuration won't rely on ansible_port variable)

So, maybe it's worth considering to use unarchive instead, just to be sure, that we have directory for our custom theme, like this (this should be enough for collectstatic task not to fail):
- file:
    path: "{{ horizon_lib_dir }}/openstack_dashboard/{{ item.value.theme_path }}"
    state: directory
    mode: 0755
    owner: "{{ horizon_system_user_name }}"
    group: "{{ horizon_system_group_name }}"
  with_dict: "{{ horizon_custom_themes }}"
  when: horizon_custom_themes is defined

Nevertheless, implementing change just to create a directory for themes will make people with custom themes finding the way to place their themes to containers by themselves, so first approach (with synchronize) seems a bit more universal and useful. Also it makes possible to automate theme delivery in case of it's update.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/586318

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/rocky
Review: https://review.openstack.org/591336

[OpenStack Infra (hudson-openstack)]

Change abandoned by Dmitriy Rabotjagov (noonedeadpunk) (<email address hidden>) on branch: stable/rocky
Review: https://review.openstack.org/591336
Reason: backports before the master patch has merged

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/586318
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-os_horizon/commit/?id=acf7a29609c81529981b9e4a167c63faa78819d8
Submitter: Zuul
Branch: master

commit acf7a29609c81529981b9e4a167c63faa78819d8
Author: Dmitriy R <email address hidden>
Date: Thu Jul 26 21:31:55 2018 +0300

    Implements custom theme distribution

    Adds key ``theme_src_archive`` to ``horizon_custom_themes``,
    which should point to archive with packed theme inside.
    Archive should be placed on the deployment host and it
    may be easily created by git-archive, or ansible git module.
    Structure inside archive should be as a standard theme,
    without any leading folders.

    Unarchive is used instead of synchronize with flat files,
    as synchronize establishes independent SSH connection
    and doesn't use nspawn or any other methods from
    custom ssh plugin and may case connection errors.
    Recursive copy has limitations on number of files inside of the
    directory. This limit is pretty high, but still it's a limit.

    This method differs from uploading custom files, as themes may
    have complex structure and include a lot of files.

    As a result, if ``horizon_custom_themes`` contains
    ``theme_src_archive`` key, theme will be distributed by role.

    Change-Id: Icbfff0793a703de94091cfdcdecf5a2c91bae4be
    Related-Bug: 1778098

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/592635

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/591336
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-os_horizon/commit/?id=41291565f82d804aeebc15d0e1c0d719e00c37e1
Submitter: Zuul
Branch: stable/rocky

commit 41291565f82d804aeebc15d0e1c0d719e00c37e1
Author: Dmitriy R <email address hidden>
Date: Thu Jul 26 21:31:55 2018 +0300

    Implements custom theme distribution

    Adds key ``theme_src_archive`` to ``horizon_custom_themes``,
    which should point to archive with packed theme inside.
    Archive should be placed on the deployment host and it
    may be easily created by git-archive, or ansible git module.
    Structure inside archive should be as a standard theme,
    without any leading folders.

    Unarchive is used instead of synchronize with flat files,
    as synchronize establishes independent SSH connection
    and doesn't use nspawn or any other methods from
    custom ssh plugin and may case connection errors.
    Recursive copy has limitations on number of files inside of the
    directory. This limit is pretty high, but still it's a limit.

    This method differs from uploading custom files, as themes may
    have complex structure and include a lot of files.

    As a result, if ``horizon_custom_themes`` contains
    ``theme_src_archive`` key, theme will be distributed by role.

    Change-Id: Icbfff0793a703de94091cfdcdecf5a2c91bae4be
    Related-Bug: 1778098
    (cherry picked from commit acf7a29609c81529981b9e4a167c63faa78819d8)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/592635
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-os_horizon/commit/?id=2f9f1a2e4b7cf2b57fe8de9a5c62c4c10afcc7a1
Submitter: Zuul
Branch: stable/queens

commit 2f9f1a2e4b7cf2b57fe8de9a5c62c4c10afcc7a1
Author: Dmitriy R <email address hidden>
Date: Thu Jul 26 21:31:55 2018 +0300

    Implements custom theme distribution

    Adds key ``theme_src_archive`` to ``horizon_custom_themes``,
    which should point to archive with packed theme inside.
    Archive should be placed on the deployment host and it
    may be easily created by git-archive, or ansible git module.
    Structure inside archive should be as a standard theme,
    without any leading folders.

    Unarchive is used instead of synchronize with flat files,
    as synchronize establishes independent SSH connection
    and doesn't use nspawn or any other methods from
    custom ssh plugin and may case connection errors.
    Recursive copy has limitations on number of files inside of the
    directory. This limit is pretty high, but still it's a limit.

    This method differs from uploading custom files, as themes may
    have complex structure and include a lot of files.

    As a result, if ``horizon_custom_themes`` contains
    ``theme_src_archive`` key, theme will be distributed by role.

    Change-Id: Icbfff0793a703de94091cfdcdecf5a2c91bae4be
    Related-Bug: 1778098
    (cherry picked from commit acf7a29609c81529981b9e4a167c63faa78819d8)