Webstaff: Replace Grunt with Webpack + Angular 1.6

Bug #1739803 reported by Bill Erickson on 2017-12-22
18
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Evergreen
Medium
Unassigned

Bug Description

Evergreen 3.0 / Wishlist

As mentioned in https://wiki.evergreen-ils.org/doku.php?id=dev:browser_staff:angular5 I suggest we replace Grunt with Webpack for compiling the AngularJS staff client builds. I make this suggestion now for a number of reasons:

1. It can be integrated as a drop-in replacement for Grunt. Application code changes are not required.

2. It (natively) supports named file bundles, e.g. core JS bundle, external dependencies bundle, etc. These bundles reduce the number of files downloaded on initial page load, improving render times.

3. Like Grunt, it has minification plugins.

4. It supports ES6-style 'import' statements, should we decide to integrate those into our web staff code in the future (as prep for Angular5).

5. Webpack is used by default in Angular5 / ng-cli

Patch en route to make these changes and bump our AngularJS version up to 1.6. The following bugs will be superseded by this bug:

https://bugs.launchpad.net/evergreen/+bug/1692097 (minification repairs)
https://bugs.launchpad.net/evergreen/+bug/1680140 (angular 1.6 updates)

New commands:

1. 'grunt test' will be replaced with 'npm run test'.
2. 'grunt build' will be replaced with 'npm run build' and 'npm run build-prod' (for minification).

Bill Erickson (berick) wrote :

Code pushed:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1626157-ang-1.6-plus-webpack

Included in the branch are doc updates, make_release updates, minor cleanup to remove unused files, and commented-out repetitive debug messages that affect unit test running in particular.

==

There is one work flow change in this branch that bears discussion:

I have configured webpack to create a 'core.bundle.js' file from the EG services/*.js files that are loaded on every page, i.e. the list in base_js.tt2. (That part alone reduced the file download count by 20. Note the bundle has a generated inline source map so tracking errors to specific lines of code is still possible). Creating this bundle means webpack has to be run before every code change is deployed on a test server, since base_js.tt2 imports the bundle, not the individual service files.

To make this easier, I have added a 'npm run build-watch' command for auto-compiling builds as changes are applied to affected files. If this process is started and left running in a second terminal, then a typical work flow of "change some js files, copy them into place" will work like before. Note that for now this only affects core service files, not app.js files.

If this is too disruptive, we could avoid creating the core.bundle.js builds when building in developer mode. It would just require webpack communicate to base_js.tt2 that it needs to import individual files (a la EXPAND_WEB_IMPORTS) in dev mode.

tags: added: pullrequest
Changed in evergreen:
assignee: Bill Erickson (berick) → nobody
Bill Erickson (berick) wrote :
summary: - Webstaff: Replace Grunt with Webpack
+ Webstaff: Replace Grunt with Webpack + Angular 1.6
Dan Wells (dbw2) on 2018-02-14
Changed in evergreen:
importance: Undecided → Medium
Jason Stephenson (jstephenson) wrote :

Since I recently had some issues with npm install due to old Node.js, and I am considering opening a bug about the warnings from npm install, I will have a look at this.

I assume Dan is also looking since he altered the but importance. I don't think he'll mind another set of eyes on this.

Changed in evergreen:
assignee: nobody → Jason Stephenson (jstephenson)
Jason Stephenson (jstephenson) wrote :

Bill, thanks for putting this together.

I rebased the branch on master and tested on Ubuntu 16.04. I used the ubuntu-xenial-developer Makefile target to install Node.js. (I also used OpenSRF master branch, fwiw.)

npm install gave me the following warnings:

npm WARN karma-jasmine@1.1.1 requires a peer of jasmine-core@* but none was installed.
npm WARN ajv-keywords@3.1.0 requires a peer of ajv@^6.0.0 but none was installed.
npm WARN evergreen-staff-client@0.0.1 No repository field.
npm WARN evergreen-staff-client@0.0.1 license should be a valid SPDX license expression

npm run test produces the following output:

> evergreen-staff-client@0.0.1 test /home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff
> npm run create-mock-idl ; karma start test/karma.conf --single-run; npm run remove-mock-idl

> evergreen-staff-client@0.0.1 create-mock-idl /home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff
> cd test/data && perl idl2js.pl

module.js:471
    throw err;
    ^

Error: Cannot find module 'jasmine-core'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.resolve (internal/module.js:27:19)
    at initJasmine (/home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff/node_modules/karma-jasmine/lib/index.js:8:42)
    at Array.invoke (/home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff/node_modules/di/lib/injector.js:75:15)
    at Injector.get (/home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff/node_modules/di/lib/injector.js:48:43)
    at /home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff/node_modules/karma/lib/server.js:143:20
    at Array.forEach (native)
    at Server._start (/home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff/node_modules/karma/lib/server.js:142:21)
    at Injector.invoke (/home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff/node_modules/di/lib/injector.js:75:15)
    at Server.start (/home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff/node_modules/karma/lib/server.js:103:18)
    at Object.exports.run (/home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff/node_modules/karma/lib/cli.js:280:26)
    at Object.<anonymous> (/home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff/node_modules/karma/bin/karma:3:23)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)

> evergreen-staff-client@0.0.1 remove-mock-idl /home/opensrf/Evergreen/Open-ILS/web/js/ui/default/staff
> rm test/data/IDL2js.js

npm run build-prod looks OK.

It looks like two dependencies are not being installed. I can't say if that is the fault of this branch or of Node.

Changed in evergreen:
assignee: Jason Stephenson (jstephenson) → nobody
Bill Erickson (berick) wrote :

I was able to fix the issues with:

npm install --save-dev jasmine-core

I'll push a patch to add this to package.json and rebase to master.

Bill Erickson (berick) wrote :

package.json fix and master rebase force pushed to same branch.

Jason Stephenson (jstephenson) wrote :

Thanks, Bill! I am taking another look.

Changed in evergreen:
assignee: nobody → Jason Stephenson (jstephenson)
Changed in evergreen:
assignee: Jason Stephenson (jstephenson) → nobody
Jason Stephenson (jstephenson) wrote :

I have driven this branch for a few miles and I'm satisfied with the performance and comfort. :)

I've pushed a signoff branch to user/dyrcona/lp1739803-ang-1.6-plus-webpack-signoff in the working repo.

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/dyrcona/lp1739803-ang-1.6-plus-webpack-signoff

Since Dan Wells changed the severity of the bug a few days ago, and he's 3.1 RM, I'll leave it up to him to decide if it should go into 3.1 or not, but I say, "Yes!"

Thanks, Bill!

tags: added: signedoff
Galen Charlton (gmc) on 2018-02-19
Changed in evergreen:
assignee: nobody → Galen Charlton (gmc)
Galen Charlton (gmc) wrote :

In process of testing this as well; at first glance, generally looks OK, but offline appears to be broken out of the box. Adding angular-tablesort vendorJsFiles solves one problem, but I'm seeing a fair number of warnings in the console. I will continue to test.

Galen Charlton (gmc) wrote :

Pushed to master for inclusion in 3.1-beta. Thanks, Bill and Jason.

Changed in evergreen:
status: New → Fix Committed
assignee: Galen Charlton (gmc) → nobody
Jason Etheridge (phasefx) wrote :

re: offline, this test is failing: live_t/24-offline-all-assets.t
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests

# Failed test 'No missing assets required by the offline interface'
# at live_t/24-offline-all-assets.t line 7.
# got: '1'
# expected: '0'
# Looks like you failed 1 test of 1.

Jason Etheridge (phasefx) wrote :

I filed a new bug so this doesn't get lost, which seems better than my old practice :)

bug 1751318

Changed in evergreen:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers