Replacement needed for PhantomJS

[Jeff Davis]

We've been using PhantomJS to run unit tests for the web client. Unfortunately that project has been abandoned and is no longer being developed:
https://github.com/ariya/phantomjs/issues/15344

The final release is now several years old and increasingly outdated, so we should be looking for alternatives. Headless Chromium has been suggested as a possibility, although we should avoid bringing in a bunch of dependencies that are only needed to run tests.

There has been periodic discussion of PhantomJS alternatives in IRC, most recently here:
http://irc.evergreen-ils.org/evergreen/2019-09-27#i_421093

[Bill Erickson (berick)]

I hesitate to mention this, since it's far afield of our current testing practices, but it may be worth noting that it's possible to run unit tests with Chrome proper on a Linux desktop (VM or otherwise). It requires Evergreen/OpenSRF checkouts, npm, Perl LibXSLT (for IDL parsing), and a few manual steps to make the test environment happy.

I was able to run unit tests for both Ang and AngJS builds. (Ang image attached to prove it wasn't a hallucination).

Mostly I was just curious if it would still work and was pleased to find it did.

[Jane Sandberg (sandbergja)]

Just a related note that it would be nice to add some e2e tests for Angular, which would rely on something like headless Chrome (or headless Firefox? protractor's documentation seems to say that this would work: http://www.protractortest.org/#/browser-setup)

[Jason Boyer (jboyer)]

Seeing these emails got me interested in chrome/ff testing again today and I did some poking at things. While switching to headless chrome is very easy (4-5 lines) there's no getting around the fact that all of the builds link against 75 shared libraries and none of those are optional. We don't see the window, but the binary linker / loader doesn't know that; it only knows that libXCursor.so.1 and a dozen or so other X11 libraries can't be found and everything breaks.

All of the articles I've seen about how easy it is to switch from PhantomJS to Chrome/FF really lay bare some interesting assumptions about how node projects are developed. (Add or change 3 lines, no need to install anything new!) It looks like no one runs tests during production installs, testing that's not done on dedicated testing servers is all done on dev's desk/laptops. Hopefully before pushing any changes upstream...

So I think we are heading to a situation where web client testing is something that you do outside of a regular install, either on the machine you're developing on like Bill alluded to (or a vm on said machine, etc.), or a dedicated machine with extra packages installed specifically for the purpose of testing. The good news is that if you just add chromium (which will pull in 139 more packages) to the list of packages for the -developer targets it is a very simple set of edits to use it. Even better would be to use both chromium and FF, but I don't know how easy that is to hook up yet.

We'll also need to change the install instructions when this changes.

I'm going to try to get a branch started soon.

[Jason Boyer (jboyer)]

Here is a branch that follows my thoughts from above; if you install the developer tools (<osname>-developer in the INSTALL instructions), Chromium and Firefox are both installed so you can run the Angular(JS) tests in all of our current supported browsers. Because the Karma testing plugin wants to look for Chrome specifically, you have to tell it where Chromium is in an env variable.

The INSTALL doc has been edited to reflect this, assuming that things will have been tested as part of development and you can just build and install the web client components.

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/jboyer/lp1845693-chrome-ang-testing / working/user/jboyer/lp1845693-chrome-ang-testing

[Bill Erickson (berick)]

Well that was easy. Thanks, Jason. Tests run successfully in both browsers and both Ang / AngJS environments.

Sign-off pushed:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1845693-chrome-ang-testing-signoff

[Galen Charlton (gmc)]

Pushed to master and rel_3_4, along with a follow-up to clarify what CHROME_BIN can be set to. Thanks, Jason and Bill!