Combo loader crash when requesting JS assets

[James Tunnicliffe]

If I visit my nodes page directly (http://192.168.1.1/MAAS/#/nodes) when I have one node aquired and one node allocated I don't see any page content (header and footer are there). The Javascript console shows that this path is being referenced, and resulting in an internal server error:

http://192.168.1.1/MAAS/combo/maas/?js/angular/maas.js&js/angular/factories/region.js&js/angular/factories/nodes.js&js/angular/factories/devices.js&js/angular/factories/clusters.js&js/angular/factories/zones.js&js/angular/factories/general.js&js/angular/factories/users.js&js/angular/factories/events.js&js/angular/services/search.js&js/angular/services/manager.js&js/angular/services/managerhelper.js&js/angular/services/error.js&js/angular/services/validation.js&js/angular/directives/call_to_action.js&js/angular/directives/power_parameters.js&js/angular/directives/os_select.js&js/angular/directives/accordion.js&js/angular/directives/dbl_click_overlay.js&js/angular/directives/contenteditable.js&js/angular/filters/nodes.js&js/angular/controllers/error.js&js/angular/controllers/nodes_list.js&js/angular/controllers/add_hardware.js&js/angular/controllers/add_device.js&js/angular/controllers/node_details.js&js/angular/controllers/node_result.js&js/angular/controllers/node_events.js

This seems to happen about 1/3 of the time. The only reference I can find in /var/log/maas is:

./regiond.log:2015-04-22 09:08:01 [HTTPChannel,3,127.0.0.1] 500 Error - /MAAS/combo/angularjs/
./regiond.log:2015-04-22 09:08:01 [-] "127.0.0.1" - - [22/Apr/2015:08:08:01 +0000] "GET /MAAS/combo/angularjs/?angular.min.js&angular-route.min.js&angular-cookies.min.js HTTP/1.1" 500 223 "http://192.168.1.1/MAAS/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36"

Let me know if you want more from /var/log. I can only find reference to angularjs in regiond.log and the apache access log. Two consecutive page views, the second one failing, are shown here: http://pastebin.ubuntu.com/10865075/

dooferlad@homework2:/var/log/maas$ dpkg -l '*maas*'|cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============================-===================================-============-======================================
ii maas 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS server all-in-one metapackage
ii maas-cli 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS command line API tool
ii maas-cluster-controller 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS server cluster controller
ii maas-common 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS server common files
ii maas-dhcp 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS DHCP server
ii maas-dns 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS DNS server
ii maas-proxy 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS Caching Proxy
ii maas-region-controller 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS server complete region controller
ii maas-region-controller-min 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS Server minimum region controller
ii python-django-maas 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS server Django web framework
ii python-maas-client 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS python API client
ii python-maas-provisioningserver 1.8.0~beta3+bzr3825-0ubuntu1~vivid1 all MAAS server provisioning libraries

[Raphaël Badin (rvb)]

Since you got a 500 error, you must have a stacktrace in the logs. Can you please attach the log files (/var/log/maas/*) to this bug?

[Raphaël Badin (rvb)]

In http://pastebin.ubuntu.com/10865075/, I see that the combo loader also returns 500 response when loading other files (YUI files and JQuery files) so it seems the problem is with the combo loader (the component that loads JS files in batches).

[James Tunnicliffe (dooferlad)]

Logs attached.

[Raphaël Badin (rvb)]

Thanks for the logs. It seems the error you're seeing is caused by the following exception: http://paste.ubuntu.com/10865693/

[Ricardo Bánffy (rbanffy)]

Instead of using separate delete/create steps, the register_persistent_error function was changed to use get_or_create, which narrows the time where a race condition can occur and reduces the number of writes to the database.

[Blake Rouse (blake-rouse)]

This can also affect any HTTP request. I hit this issue with the pxeconfig API call from the cluster, which causes nodes to fail to deploy.

Used Ricardo branch, has solved the issue so far for me.

[Raphaël Badin (rvb)]

Seems the race is still here: see bug 1443917.