Asset compression does not happen unless debug mode is enabled

[James Page]

Juno/rc1 of OpenStack on utopic; the dashboard is unthemed and the compressed assets are missing unless DEBUG = True in local settings, at which point things look much better.

ProblemType: Bug
DistroRelease: Ubuntu 14.10
Package: openstack-dashboard 1:2014.2~rc1-0ubuntu3 [modified: usr/share/openstack-dashboard/openstack_dashboard/enabled/_40_router.py]
ProcVersionSignature: User Name 3.16.0-20.27-generic 3.16.3
Uname: Linux 3.16.0-20-generic x86_64
ApportVersion: 2.14.7-0ubuntu5
Architecture: amd64
Date: Fri Oct 10 11:26:21 2014
Ec2AMI: ami-000000af
Ec2AMIManifest: FIXME
Ec2AvailabilityZone: nova
Ec2InstanceType: m1.small
Ec2Kernel: aki-00000002
Ec2Ramdisk: ari-00000002
PackageArchitecture: all
SourcePackage: horizon
UpgradeStatus: No upgrade log present (probably fresh install)
mtime.conffile..etc.apache2.conf.available.openstack.dashboard.conf: 2014-10-10T11:25:49.335633
mtime.conffile..etc.openstack.dashboard.local.settings.py: 2014-10-10T11:25:49.307619

[James Page (james-page)]

I'm assuming that this is something todo with how django-pyscss behaves under DEBUG mode:

https://github.com/fusionbox/django-pyscss

specifically:

In DEBUG mode, DjangoScss will search using all of the finders to find the file. If you are not in DEBUG mode, it assumes you have run collectstatic and will only use staticfiles_storage to find the file.

[James Page (james-page)]

Switching off the ubuntu-theme makes no difference.

[James Page (james-page)]

I'm guessing that this is something todo with how we layout horizon on the filesystem in the packaging; raising an upstream bug task to get some advice on how to debug this (please :-)!)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Kieran Spear (kspear)]

What does the apache log say? Looks like your apache conf isn't allowing access to the dir aliased to /static (and that looks like apache 2.2 rather than 2.4-style access control?).

[James Page (james-page)]

10.5.17.0 - - [13/Oct/2014:09:19:52 +0000] "GET /horizon HTTP/1.1" 200 1561 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0"
10.5.17.0 - - [13/Oct/2014:09:19:53 +0000] "GET /static/dashboard/scss/horizon.scss HTTP/1.1" 200 42258 "http://10.5.17.0/horizon" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0"
10.5.17.0 - - [13/Oct/2014:09:19:53 +0000] "GET /static/horizon/lib/font-awesome/scss/font-awesome.scss HTTP/1.1" 404 544 "http://10.5.17.0/horizon" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0"
10.5.17.0 - - [13/Oct/2014:09:19:53 +0000] "GET /static/dashboard/scss/horizon_workflow.scss HTTP/1.1" 200 2770 "http://10.5.17.0/horizon" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0"
10.5.17.0 - - [13/Oct/2014:09:19:53 +0000] "GET /static/dashboard/scss/horizon_charts.scss HTTP/1.1" 200 3476 "http://10.5.17.0/horizon" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0"
10.5.17.0 - - [13/Oct/2014:09:19:53 +0000] "GET /static/horizon/lib/bootstrap_datepicker/datepicker3.css HTTP/1.1" 404 545 "http://10.5.17.0/horizon" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0"
10.5.17.0 - - [13/Oct/2014:09:19:53 +0000] "GET /static/dashboard/css/rickshaw.css HTTP/1.1" 200 1807 "http://10.5.17.0/horizon" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0"
10.5.17.0 - - [13/Oct/2014:09:19:53 +0000] "GET /static/ubuntu/css/ubuntu.scss HTTP/1.1" 200 8407 "http://10.5.17.0/horizon" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0"
10.5.17.0 - - [13/Oct/2014:09:19:53 +0000] "GET /static/ubuntu/img/favicon-ubuntu.ico HTTP/1.1" 200 922 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0"
10.5.17.0 - - [13/Oct/2014:09:19:53 +0000] "GET /static/dashboard/scss/_variables HTTP/1.1" 404 521 "http://10.5.17.0/static/dashboard/scss/horizon_workflow.scss" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0"

[James Page (james-page)]

I tried compressing the assets manually:

ubuntu@juju-jpos-machine-16:/usr/share/openstack-dashboard$ sudo ./manage.py compress --force
Found 'compress' tags in:
        /usr/lib/python2.7/dist-packages/horizon/templates/horizon/_conf.html
        /usr/share/openstack-dashboard-ubuntu-theme/templates/_stylesheets.html
        /usr/share/openstack-dashboard/openstack_dashboard/templates/_stylesheets.html
        /usr/lib/python2.7/dist-packages/horizon/templates/horizon/_scripts.html
Compressing... CommandError: An error occured during rendering /usr/share/openstack-dashboard-ubuntu-theme/templates/_stylesheets.html: Error evaluating expression:
    $font-size-small

From /usr/share/openstack-dashboard/static/dashboard/scss/_context_selection.scss:20
...imported from <string u'// bootstrap overrides:\n$icon-font-path: "../../bo'...>:141
Traceback:
  File "/usr/lib/python2.7/dist-packages/scss/expression.py", line 130, in evaluate_expression
    return ast.evaluate(self, divide=divide)
  File "/usr/lib/python2.7/dist-packages/scss/expression.py", line 359, in evaluate
    raise SyntaxError("Undefined variable: '%s'." % self.name)
SyntaxError: Undefined variable: '$font-size-small'.

tweaking DEBUG = True and re-running fixes this - so looks like something todo with lookup paths.

[James Page (james-page)]

OK - so the difference in loading with DEBUG = True:

    def get_file_and_storage(self, filename):
        # TODO: the switch probably shouldn't be on DEBUG
        if settings.DEBUG:
            return self.get_file_from_finders(filename)
        else:
            return self.get_file_from_storage(filename)

file_from_finders == good
file_from_storage == bad - and a silent fail right now

forcing and error:

From <string u'// bootstrap overrides:\n$icon-font-path: "../../bo'...>:0
Traceback:
  File "/usr/lib/python2.7/dist-packages/scss/__init__.py", line 498, in manage_children
    self._manage_children_impl(rule, scope)
  File "/usr/lib/python2.7/dist-packages/scss/__init__.py", line 548, in _manage_children_impl
    self._do_import(rule, scope, block)
  File "/usr/lib/python2.7/dist-packages/django_pyscss/scss.py", line 119, in _do_import
    source_file = self._find_source_file(name, relative_to)
  File "/usr/lib/python2.7/dist-packages/django_pyscss/scss.py", line 87, in _find_source_file
    full_filename, storage = self.get_file_and_storage(name)
  File "/usr/lib/python2.7/dist-packages/django_pyscss/scss.py", line 56, in get_file_and_storage
    return self.get_file_from_storage(filename)
  File "/usr/lib/python2.7/dist-packages/django_pyscss/scss.py", line 43, in get_file_from_storage
    raise Exception('%s not found' % filename)
Exception: /usr/share/openstack-dashboard/static/bootstrap/scss/_bootstrap.scss not found

[James Page (james-page)]

Hmm - actually I can reproduce this directly in the upstream codebase by disabling DEBUG and using --runserver.

[James Page (james-page)]

Checking environment.
Environment is up to date.
Starting Django development server...
Validating models...

0 errors found
October 13, 2014 - 10:24:42
Django version 1.6.7, using settings 'openstack_dashboard.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
[13/Oct/2014 10:24:46] "GET / HTTP/1.1" 200 2478
Not Found: /static/dashboard/scss/horizon.scss
Not Found: /static/dashboard/css/rickshaw.css
Not Found: /static/horizon/lib/bootstrap_datepicker/datepicker3.css
Not Found: /static/dashboard/scss/horizon_charts.scss
Not Found: /static/dashboard/scss/horizon_workflow.scss
Not Found: /static/horizon/lib/font-awesome/scss/font-awesome.scss
[13/Oct/2014 10:24:52] "GET /static/horizon/lib/bootstrap_datepicker/datepicker3.css HTTP/1.1" 404 5047
Not Found: /static/dashboard/img/favicon.ico
[13/Oct/2014 10:24:52] "GET /static/dashboard/scss/horizon.scss HTTP/1.1" 404 5047
[13/Oct/2014 10:24:52] "GET /static/dashboard/css/rickshaw.css HTTP/1.1" 404 5047
[13/Oct/2014 10:24:52] "GET /static/horizon/lib/font-awesome/scss/font-awesome.scss HTTP/1.1" 404 5047
[13/Oct/2014 10:24:52] "GET /static/dashboard/scss/horizon_charts.scss HTTP/1.1" 404 5047
[13/Oct/2014 10:24:52] "GET /static/dashboard/scss/horizon_workflow.scss HTTP/1.1" 404 5047
[13/Oct/2014 10:24:52] "GET /static/dashboard/img/favicon.ico HTTP/1.1" 404 5047

[James Page (james-page)]

as devstack uses the same local_settings which has DEBUG = False, I suspect this is generally quite broken:

https://github.com/openstack-dev/devstack/blob/master/lib/horizon

My current thinking is to completely disable the file_from_storage codepath in python-django-scss to workaround this problem.

[James Page (james-page)]

I inadvertently fixed this in the Ubuntu packaging by linking the bootstrap_scss directory into the static root configuration to resolve another issue I was seeing with missing glyph icons; that appears to resolve the issue but I consider that a workaround.

[David Lyle (david-lyle)]

For easy reproduction on the Horizon side:

DJANGO_SETTINGS_MODULE=openstack_dashboard.settings tools/with_venv.sh python manage.py collectstatic --noinput

[David Lyle (david-lyle)]

This bug will block styling on Horizon for DEBUG=False regardless whether you use offline compression or not.

[Eric Peterson (ericpeterson-l)]

it works for me, following these steps:

Change local settings, have the following at the top of the file:
DEBUG = False
TEMPLATE_DEBUG = DEBUG
COMPRESS_OFFLINE = True

Then I run these commands:
tools/with_venv.sh python manage.py collectstatic --noinput
tools/with_venv.sh python manage.py compress --force

The compress step produces this to the console:

Invalid template /home/eric/work/public/horizon/horizon/templates/horizon/common/_formset_table_row.html: Invalid block tag: 'blocktrans', expected 'empty' or 'endfor'
Found 'compress' tags in:
        /home/eric/work/public/horizon/openstack_dashboard/templates/_stylesheets.html
        /home/eric/work/public/horizon/horizon/templates/horizon/_conf.html
        /home/eric/work/public/horizon/horizon/templates/horizon/_scripts.html
Compressing... done
Compressed 4 block(s) from 3 template(s).
eric@eric-VirtualBox:~/work/public/horizon$

While there is an error / warning, it does complete ok and is useable.

[Eric Peterson (ericpeterson-l)]

git describe
2014.2.rc1-145-g123840a

that is my current version where it works

[Eric Peterson (ericpeterson-l)]

additional note: when you do offline compression and stuff get's moved into the static folder, using the --runserver cli to start up django will not serve out the /static url. for this type of use, I believe it's expected that apache would serve out those assets from the ./static folder (which is how I do work, with modwsgi etc)

[Matthias Runge (mrunge)]

odd, I can not reproduce this at all, using Fedora packages e.g. from http://koji.fedoraproject.org/koji/taskinfo?taskID=7862410

[Gloria Gu (gloria-gu)]

I just pulled from master, followed Eric's steps..updated the local_settings...it seems to be ok. Some of the output.

[stack@gloria-stack:/home/stack/horizon]↥ master+ 127 ± git pull
Already up-to-date.
[stack@gloria-stack:/home/stack/horizon]↥ master+ ± tools/with_venv.sh python manage.py collectstatic --noinput
Copying '/home/stack/horizon/.venv/local/lib/python2.7/site-packages/xstatic/pkg/angular/data/angular.js'
Copying '/home/stack/horizon/.venv/local/lib/python2.7/site-packages/xstatic/pkg/angular_cookies/data/angular-cookies
....
Copying '/home/stack/horizon/horizon/static/horizon/js/angular/directives/forms.js'
Copying '/home/stack/horizon/horizon/static/horizon/js/angular/controllers/metadata-widget-controller.js'
Copying '/home/stack/horizon/horizon/static/horizon/js/angular/controllers/dummy.js'

742 static files copied to '/home/stack/horizon/static'.
[stack@gloria-stack:/home/stack/horizon]↥ master+ ± tools/with_venv.sh python manage.py compress --force
Invalid template /home/stack/horizon/horizon/templates/horizon/common/_formset_table_row.html: Invalid block tag: 'blocktrans', expected 'empty' or 'endfor'
Found 'compress' tags in:
        /home/stack/horizon/openstack_dashboard/templates/_stylesheets.html
        /home/stack/horizon/horizon/templates/horizon/_scripts.html
        /home/stack/horizon/horizon/templates/horizon/_conf.html
Compressing... No handlers could be found for logger "scss.expression"
done
Compressed 4 block(s) from 3 template(s).

[James Page (james-page)]

python manage.py collectstatic --noinput is currently missing from the Ubuntu packaging; I'll look see how we can integrate that into the packaging.

[Doug Fish (drfish)]

I've opened https://bugs.launchpad.net/horizon/+bug/1381472 to address the issue Gloria describes in comment 20

[Launchpad Janitor (janitor)]

This bug was fixed in the package horizon - 1:2014.2~rc2-0ubuntu2

---------------
horizon (1:2014.2~rc2-0ubuntu2) utopic; urgency=medium

  * Resolve issues with missing static assets and failing compression
    (LP: #1379761):
    - d/openstack-dashboard*.postinst: Collect and compress static assets
      during installation.
    - d/rules: Drop explicit link to bootstrap scss resources
    - d/p/ubuntu_settings.patch: Switch back to using offline compression.
 -- James Page <email address hidden> Wed, 15 Oct 2014 10:10:08 +0100

[Wu Hong Guang (whg)]

i met this problem recently on Dec 19 when i updated devstack and it's ok aftre manually run sudo ./manage.py compress --force