Cronscript.pm broke unless --sysconfdir specified during configure

[James Fournie]

Clean install from EG master @ 9222fb95a4
OpenSRF @ ff62eb3409f5d36a76a

Ubuntu Lucid/ Postgres 9.1

Built OpenSRF using ./configure --prefix=/srv/openils
Build EG with ./configure --prefix=/srv/openils --with-opensrf-libs=/srv/openils/lib --with-opensrf-headers=/srv/openils/include

When autogen is run:

Updating Evergreen organization tree and IDL

Updating fieldmapper
Could not create file parser context for file "${prefix}/etc/opensrf_core.xml": No such file or directory at /usr/local/share/perl/5.10.1/OpenSRF/Utils/Config.pm line 323.
 -> /srv/openils/var/web/opac/common/js//fmall.js
Updating web_fieldmapper
Could not create file parser context for file "${prefix}/etc/opensrf_core.xml": No such file or directory at /usr/local/share/perl/5.10.1/OpenSRF/Utils/Config.pm line 323.
 -> /srv/openils/var/web/opac/common/js//fmcore.js
Updating OrgTree
Could not create file parser context for file "${prefix}/etc/opensrf_core.xml": No such file or directory at /usr/local/share/perl/5.10.1/OpenSRF/Utils/Config.pm line 323.
Could not create file parser context for file "${prefix}/etc/opensrf_core.xml": No such file or directory at /usr/local/share/perl/5.10.1/OpenSRF/Utils/Config.pm line 323.
No bootstrap config exists. Have you bootstrapped?

It looks like the automake process is not replacing ${prefix} with the actual prefix -- it replaces @@sysconfdir@@ with the literal value of that, but doesn't further translate ${prefix} to that value.

Specifying a sysconfdir of /srv/openils/etc does work, it seems that not specifying a sysconfdir and letting it assume the default is what fails.

Of note, once built, looking in Open-ILS/Makefile

With out specifying sysconfdir in configure: sysconfdir = ${prefix}/etc
With /srv/openils/etc specified: sysconfdir = /srv/openils/etc

[Jason Stephenson (jstephenson)]

As tsbere pointed out in IRC:

Cronscript is being treated like a Makefile, not as a final product, by configure, but then not treated as such by make install.
Configure is expecting it to be parsed such that the prefix value is substituted at runtime.

One solution:

Change Cronscript.pm.in to Cronscript.pm so it is not handled like a Makefile.in by automake and do a sed on the Cronsript.pm like we do on several other files already.

[Jason Stephenson (jstephenson)]

Guess the real bug isn't that automake nor the default sysconfdir are broken.

The real bug is that Cronscript.pm.in is listed as a config file in configure.ac and thus handled much like a Makefile.

Ideally, scripts should not contain hardcoded paths that need to be munged at installation time. Dan Scott suggested in IRC using eg_config --sysconfdir at runtime to get the appropriate directory. I agree that is a better solution than using sed on the script file during installation. If no one objects, I'll work on a branch to do just that.

[Dan Scott (denials)]

I'm in the process of putting together a Goldilocks branch.

As Jason Stephenson noted earlier, configure is the wrong time to set the default variables, as different values can be set at build time via the "make" command (this goes for eg_config as well, unfortunately). Install time is too late, because that breaks testing by leaving us without a built file to actually test in the build directory.

Thus, per http://www.gnu.org/software/automake/manual/html_node/Scripts.html, the approach I'm following is to use "sed" at build time, for *.in in general, not just for Cronscript.pm.

[Dan Scott (denials)]

I've pushed user/dbs/lp960552_autoconf_subvars to working - thoughts?

[Dan Scott (denials)]

Rebased user/dbs/lp960552_autoconf_subvars to current master with an additional commit to trigger a rebuild of generated files at build time if ./configure gets called a subsequent time.

[Jason Stephenson (jstephenson)]

Used the branch and without specifying sysconfdir I get this:

Updating fieldmapper
Could not create file parser context for file "${prefix}/etc/opensrf_core.xml": No such file or directory at /usr/local/share/perl/5.14.2/OpenSRF/Utils/Config.pm line 323.
 -> /openils/var/web/opac/common/js//fmall.js
Updating web_fieldmapper
Could not create file parser context for file "${prefix}/etc/opensrf_core.xml": No such file or directory at /usr/local/share/perl/5.14.2/OpenSRF/Utils/Config.pm line 323.
 -> /openils/var/web/opac/common/js//fmcore.js
Updating OrgTree
Could not create file parser context for file "${prefix}/etc/opensrf_core.xml": No such file or directory at /usr/local/share/perl/5.14.2/OpenSRF/Utils/Config.pm line 323.
Could not create file parser context for file "${prefix}/etc/opensrf_core.xml": No such file or directory at /usr/local/share/perl/5.14.2/OpenSRF/Utils/Config.pm line 323.
No bootstrap config exists. Have you bootstrapped?

I'll have to try it again tomorrow, but doesn't look fixed to me.

Of course, I'm loading tons of other crap on that machine and something might have nullified it.

[Dan Scott (denials)]

Crap! Confirmed on a completely clean checkout (no need to get all the way to "make install", just ./configure and checking Open-ILS/src/perlmods/lib/OpenILS/Util/Cronscript.pm is enough).

I'll remove the pullrequest tag until I clean this up. Sorry for the noise.

[Dan Scott (denials)]

Bah. In my haste I tested master, not my branch. It's worth noting that old files lying around from a previous build have the potential to be a problem, so a clean checkout / autogen.sh will be desired; also, "make" will be necessary with the actual branch. The files won't be generated from a simple "configure".

[Dan Scott (denials)]

All righty. My testing (Fedora 16, autoconf 2.68, automake 1.11.1) of user/dbs/lp960552_autoconf_subvars confirmed my previous success (and watch out for those old files hanging around from previous builds). Steps to reproduce success that I followed:

1. git checkout working/user/dbs/lp960552_autoconf_subvars
2. git clean -f # get rid of all intermediate built / working files that might be left over from previous runs
3. ./autogen.sh
4. ./configure
5. ls Open-ILS/src/perlmods/lib/OpenILS/Utils/ # shows no "Cronscript.pm" exists yet
6. make
7. vim Open-ILS/src/perlmods/lib/OpenILS/Utils/Cronscript.pm

... and that shows:

            'osrf-config=s' => '/openils//etc/opensrf_core.xml',

on line 62. So, that appears to be working as expected to me.

From there, if you run "make install" but haven't copied /openils/etc/opensrf_core.xml.example to /openils/etc/opensrf_core.xml and the other config steps, you'll run into errors in autogen.sh like:

Updating fieldmapper
Could not create file parser context for file "/openils//etc/opensrf_core.xml": No such file or directory at /usr/local/share/perl5/OpenSRF/Utils/Config.pm line 323.

... but that's because the file doesn't exist because the normal config steps haven't been followed (and note that the default directory is a real directory).

[Jason Stephenson (jstephenson)]

Alright, works for me on a fresh install. I must have had some junk hanging around on that other server.

I even tried logging in, retrieving session info, and logging out with this branch applied. Everything looked at /openils/etc as it should.

I've signed off in a collab branch on the working repo:

working/collab/dyrcona/p960552_autoconf_subvars

Thanks, Dan!

P.S. As I mentioned in IRC we should probably do the same for all of the files that we use sed on.....

[Dan Scott (denials)]

Awesome, thanks Jason! Pushed to master.

And 100% agreed on the sed changes for BSD compatibility. Would you be willing to open a separate bug to track that (as I'm not very familiar with BSD) and mark it as "bitesize" for now?