patch 'autogen.sh' to detect automake 1.12 (MacPorts, OS X)

[su_v]

MacPorts recently upgraded automake to 1.12 [1]. Inkscape's 'autogen.sh' script doesn't detect this version. The script instead falls back to the system version (automake 1.10 - if present at all, no longer provided with Xcode >= 4.3.x on Lion and Mountain Lion AFAIK). The system version of automake however lacks several required macros (only present in MacPorts' version), and the script fails:

I am testing that you have the required versions of autoconf,
automake, glib-gettextize and intltoolize. This test is not foolproof and
if anything goes wrong, there may be guidance in the file HACKING.txt

checking for autoconf >= 2.52 ... yes (version 2.69)
checking for automake >= 1.10 ... yes (version 1.10)
checking for glib-gettextize >= 2.0.0 ... yes (version 2.32.1)
checking for intltool >= 0.17 ... yes (version 0.50.2)

WARNING: aclocal's directory is /usr/bin/../share/aclocal, but...
         no file /usr/bin/../share/aclocal/glib-2.0.m4
         You may see fatal macro warnings below.
         If these files are installed in /some/dir, set the ACLOCAL_FLAGS
         environment variable to "-I /some/dir", or install
         /usr/bin/../share/aclocal/glib-2.0.m4.

WARNING: aclocal's directory is /usr/bin/../share/aclocal, but...
         no file /usr/bin/../share/aclocal/glib-gettext.m4
         You may see fatal macro warnings below.
         If these files are installed in /some/dir, set the ACLOCAL_FLAGS
         environment variable to "-I /some/dir", or install
         /usr/bin/../share/aclocal/glib-gettext.m4.

WARNING: aclocal's directory is /usr/bin/../share/aclocal, but...
         no file /usr/bin/../share/aclocal/gtk-2.0.m4
         You may see fatal macro warnings below.
         If these files are installed in /some/dir, set the ACLOCAL_FLAGS
         environment variable to "-I /some/dir", or install
         /usr/bin/../share/aclocal/gtk-2.0.m4.

WARNING: aclocal's directory is /usr/bin/../share/aclocal, but...
         no file /usr/bin/../share/aclocal/intltool.m4
         You may see fatal macro warnings below.
         If these files are installed in /some/dir, set the ACLOCAL_FLAGS
         environment variable to "-I /some/dir", or install
         /usr/bin/../share/aclocal/intltool.m4.

WARNING: aclocal's directory is /usr/bin/../share/aclocal, but...
         no file /usr/bin/../share/aclocal/pkg.m4
         You may see fatal macro warnings below.
         If these files are installed in /some/dir, set the ACLOCAL_FLAGS
         environment variable to "-I /some/dir", or install
         /usr/bin/../share/aclocal/pkg.m4.

Running aclocal-1.10 ...
  configure.ac:181: warning: macro `AM_GLIB_GNU_GETTEXT' not found in library
Running autoheader ...
  aclocal.m4:14: error: this file was generated for autoconf 2.61.
  You have another version of autoconf. If you want to use that,
  you should regenerate the build system entirely.
  aclocal.m4:14: the top level
  autom4te: /opt/local/bin/gm4 failed with exit status: 63
  autoheader: '/opt/local/bin/autom4te' failed with exit status: 63
Please fix the error conditions and try again.

Attached patch works around the issue similar to the earlier fix for automake 1.11 in revision 9109:
<http://bazaar.launchpad.net/~inkscape.dev/inkscape/trunk/revision/9109>

A better solution might be to modify/update 'autgen.sh' as proposed in the merge request to update the autotools (untested):
<https://code.launchpad.net/~jjardon/inkscape/autotools/+merge/77850>
Revision 10658: "autogen.sh: Use autoreconf instead custom script"
<http://bazaar.launchpad.net/~jjardon/inkscape/autotools/revision/10658>

Follow-up report (to support automake 1.13):
Bug #1094576 “Patch for autogen.sh and configure.ac to support automake 1.13”

---
[1] <http://lists.macosforge.org/pipermail/macports-changes/2012-April/076121.html>
[2] See also the ticket in MacPorts for automake 1.11:
    <https://trac.macports.org/ticket/19880>
     and the one filed recently for automake 1.12:
    <https://trac.macports.org/ticket/34380>

[su_v (suv-lp)]

> A better solution might be to modify/update 'autgen.sh' as proposed in the merge
> request to update the autotools (untested):
> <https://code.launchpad.net/~jjardon/inkscape/autotools/+merge/77850>

Initial tests to use the diff from the merge request (after fixing a minor conflict with current trunk 11307) fail to correctly configure inkscape trunk (r11307) on OS X 10.7. 2 (using up-to-date dependencies from MacPorts).

The mentioned merge request needs to be reviewed independently of the reported issue with automake 1.12 when building inkscape trunk on OS X (I don't know whether possibly some linux distros are affected as well).

[Alex Valavanis (valavanisalex)]

In my (very limited!) experience, autoreconf seems a bit temperatmental, and I've generally found it more reliable to use separate calls to autoconf, aclocal etc... That could very well be an effect of my inept usage though!

[su_v (suv-lp)]

Similar patches are used for example in
- GTK+ 2.24:
  <http://git.gnome.org/browse/gtk+/commit/?h=gtk-2-24&id=1d07ef714dfe4ce872a7eaa1e66c65cb31af99b8>
- GIMP (2.8 & master):
  <http://git.gnome.org/browse/gimp/commit/?id=2a982029ad717ccf77b0459d30f464310057fc23>

@ScislaC, @Alex - any objections if I commit this to trunk?

[ScislaC (scislac)]

None here, tested fine.

[su_v (suv-lp)]

Based on this comment in 'autogen.sh' (line 88):
# Prefer earlier versions just so that the earliest supported version gets test coverage by developers.

attaching revised patch 'add-support-for-automake-1.12-v2.diff':
- test first for automake-1.11 as before, and test for automake-1.12 after that.

Summary: Testing for automake-1.10 causes configure failure on OS X if the build environment used for Inkscape does have automake-1.11 or newer installed and the system (<= 10.6.8) or Xcode (4.1, 4.2) provides automake-1.10. The revised patch keeps testing for lowest version which doesn't trigger this failure on OS X first (automake-1.11), and inserts test for automake-1.12 after that (but before the test for automake-1.10).

[Alex Valavanis (valavanisalex)]

@~suv: I was about to commit the changes in the merge request, but then I remembered your comment #2. What was the problem with the configuration when you used the version in the branch? If you could have another go, using the latest trunk and post the output from autogen.sh, we might be able to address the issues.

[su_v (suv-lp)]

On 2013-02-25 24:16 +0100, Alex Valavanis wrote:
> @~suv: I was about to commit the changes in the merge request, but
> then I remembered your comment #2. What was the problem with the
> configuration when you used the version in the branch?

I don't recall the details, but likely it picked up wrong versions of autotools. Running 'configure' had failed with this message (r11307 + diff from branch, on 2012-04-30):

 configure: error: cannot find install-sh, install.sh, or shtool in "." "./.." "./../.."

> If you could have another go, using the latest trunk and post the
> output from autogen.sh, we might be able to address the issues.

The diff from the branch doesn't apply cleanly to current trunk's configure.ac (r12149), and apparently you are suggesting a few changes to the modifications of autogen.sh as well: for easier testing, would you mind attaching a diff against current trunk you want to be tested?

[Alex Valavanis (valavanisalex)]

Here's an updated patch that should apply cleanly against trunk. I've actually come round to the idea of automatically running the configure script at the end of the autogen.sh script. It seems like it's pretty standard behaviour in other projects.

To run autogen without configure:

export NOCONFIGURE=true && ./autogen.sh

Alternatively, it should be possible to set up and configure an out-of-source build by doing something like this from the root directory:

mkdir -p build && cd build
../autogen.sh --enable-dbusapi --enable-<whatever>

The new configuration seems to work on Ubuntu 12.10 amd64.

@suv: Please could you test and report back with a transcript of any errors?

[su_v (suv-lp)]

Same failure as last year with new patch: configure fails to run with

   configure: error: cannot find install-sh, install.sh, or shtool in ".." "../.." "../../.."

Attaching build logs: autogen.sh run first (with 'export NOCONFIGURE=true && ./autogen.sh'), then '../configure' with custom '--prefix' in a second step for an out-of-src build.
The logs are generated by a custom build script which sets up the build environment depending on command line arguments.

The output of autogen.sh isn't giving me any clues at first sight - I can only assume that it is picking up some (unversioned) tools from /usr/bin, instead of the current versions from MacPorts.

[Alex Valavanis (valavanisalex)]

Hmm weird... Did the script actually create an install-sh file in your inkscape root directory?

[Alex Valavanis (valavanisalex)]

OK - I've reproduced the problem... basically, all the configuration stuff is put in ./build-aux. Configure is looking in the wrong place.

I'll look into a fix

[Alex Valavanis (valavanisalex)]

I think I've fixed it. Looks like the AC_CANONICAL_HOST macro was causing problems when it was run before the AC_CONFIG_* macros.

v2 attached. This one configures OK for me.

@suv: Would you mind trying again?

[su_v (suv-lp)]

Second patch seems to work as expected (default, dbus and gtk3 builds with Quartz backend (with automake 1.13) succeeded, now repeating tests with X11-based builds (these use somewhat older versions of the dependencies, and automake 1.11)).

Is there a simpler way to silence 'bzr status' about the macros which now are copied into 'm4/' (m4 itself is under version control), and the auto-generated files in 'po/', other than adding them individually to '.bzrignore'?

[Alex Valavanis (valavanisalex)]

Re comment #15: Not sure about this one. I think, in principle it would be possible to define two separate m4 folders (one for the files we maintain in trunk, and one for generated files). Similarly for the po folder. However, I don't think this is particularly standard practice.

Any thoughts from others? In the meantime, we can add things to .bzrignore as you suggest.

[Alex Valavanis (valavanisalex)]

If no objections, I'll apply the patch in trunk later.

[su_v (suv-lp)]

> If no objections, I'll apply the patch in trunk later.

What are the recommended steps if updating an existing branch with one or more (out-of-src) build directories? (I only tested with new branches, where the current 'autogen.sh' and configure.ac files have never been run).

[su_v (suv-lp)]

TODO: check current osx packaging script (osx-build.sh) for required changes:
autogen.sh and configure are currently separate options:
<http://bazaar.launchpad.net/~inkscape.dev/inkscape/trunk/view/head:/packaging/macosx/osx-build.sh#L128>

(osx packaging in trunk & stable is partially broken, but as long as one doesn't configure to build as osxapp (for bundling), it works ok -> osx-build.sh probably should be kept in sync with autotool build system changes)

[Alex Valavanis (valavanisalex)]

Re comment #18: I think it should just be a case of going into the build directory and running autogen with whichever configure flags you want (any arguments to autogen are just passed to configure).

So, I tested the following (starting from Inkscape root folder) of a pre-configured folder:

cd build
../autogen.sh --enable-dbusapi

autogen and configure both ran OK.

[Alex Valavanis (valavanisalex)]

Alternatively, it can be done in two stages (maybe more efficient if you have lots of build folders) for example...

export NOCONFIGURE=true && ./autogen.sh

cd build
../configure

cd ../build-gtk3
../configure --enable-gtk3-experimental

cd ../build-dbus
../configure --enable-dbusapi

[Alex Valavanis (valavanisalex)]

Should be fixed in lp:inkscape r12150

[su_v (suv-lp)]

> TODO: check current osx packaging script (osx-build.sh) for required changes

Done in r12151: don't autorun configure with new autogen.sh

[su_v (suv-lp)]

On OS X: revision >= 12150 no longer seems to install po files anymore… - is there an additional dependency required with the switch to upstream gettext (gettext 0.18.1.1 is installed)?

[su_v (suv-lp)]

I'm not really familiar with how translation files are generated, but the most obvious difference seems to be that 'ALL_LINGUAS' in the generated Makefile for 'po/' is empty (on OS X as well as on Ubuntu 12.10):

$ grep 'ALL_LINGUAS =' build/po/Makefile
ALL_LINGUAS =

(based on the output of '(cd build/po; make -n install)', it also seems that the destination for the translation files would change from "$prefix/share/locale/$lang/LC_MESSAGES" to "$prefix/lib/locale/$lang/LC_MESSAGES", possibly requiring updates elsewhere, e.g. for packaging scripts?)

[su_v (suv-lp)]

Reopening…

[su_v (suv-lp)]

> rev >= 12150 no longer seems to install po files anymore…

AFAIU this is fixed by adding the file po/LINGUAS (same information formerly stored in the variable "ALL_LINGUAS" in configure.ac).

After adding the file, the translation files (*.mo) are processed and installed, but gettext still gets the destination directory wrong (DATADIRNAME is 'lib' instead of 'share'), at least on OS X (possibly related to bug #398571 (?)).

[su_v (suv-lp)]

On 2013-02-27 05:28 +0100, ~suv wrote:
> After adding the file, the translation files (*.mo) are processed and
> installed, but gettext still gets the destination directory wrong
> (DATADIRNAME is 'lib' instead of 'share'), at least on OS X (possibly
> related to bug #398571 (?)).

Testing on Ubuntu 12.10: after adding the missing po/LINGUAS file, trunk (r12155) can be installed into a custom prefix as expected, including the translation files. The *.mo files install as expected into '$prefix/share/locale/$lang/LC_MESSAGES'.

The updated autotools build system is partially broken though on OS X (and likely on other BSD-based systems (unverified)) wrt to i18n support.

Running configure with '--datadir=$prefix/share' or with '--localedir=$prefix/share/locale" does not help either: the locale files still get installed into '$prefix/lib/locale' instead. Any help fixing this would be appreciated.

(OS X build tested with GTK+/Quartz 2.24.16, autoconf 2.69, automake 1.13.1, gettext 0.18.1.1, intltool 0.50.2).

[su_v (suv-lp)]

New report has been filed about missing po/LINGUAS:
- Bug #1134042 “Missing language list in po/Makefile”
  <https://bugs.launchpad.net/inkscape/+bug/1134042>

[Alex Valavanis (valavanisalex)]

Missing languages list issue is tracked in bug #1134042. Should be fixed in lp:inkscape r12156

[su_v (suv-lp)]

Reopening…

> Missing languages list issue is tracked in bug #1134042.
> Should be fixed in lp:inkscape r12156

That by itself does not fix the broken build system on OS X (and likely other BSD-based systems), see comment #28:
the locale files get installed into an incorrect path in $prefix, and are not found by inkscape anymore.

I have no idea how to fix this.

[Alex Valavanis (valavanisalex)]

Oops... sorry about that! Will read up on the issue and see if I can find anything.

[Alex Valavanis (valavanisalex)]

Useful advice about preparing projects for internationalisation: http://developer.gnome.org/gtkmm-tutorial/3.0/sec-internationalization-intro.html.en

[Alex Valavanis (valavanisalex)]

@suv - Here's a patch based on the advice at the above link. Would you mind seeing if this helps?

[su_v (suv-lp)]

> Here's a patch based on the advice at the above link.
> Would you mind seeing if this helps?

It doesn't address the issue at all as far as I can tell (and possibly breaks i18n-related code elsewhere in Inkscape - at least it likely would require additional changes e.g. for (python-based) extensions in 'share/extensions/inkex.py', and possibly in the packaging scripts too).

1) The problem:
On some platforms, the locale files get installed into the wrong path (based on "wrong" DATADIRNAME from intltool macros (?))

2) revision r12159 (without your patch):
| Chillida:mp-quartz-autotools-2 su_v$ grep locale build/config.h
| #define PACKAGE_LOCALE_DIR "/Volumes/magenta/src/inkscape/inkscape-repo/mp-quartz-autotools-2/inst/share/locale"
and:
| Making install in po
| installing am.gmo as /Volumes/magenta/src/inkscape/inkscape-repo/mp-quartz-autotools-2/inst/lib/locale/am/LC_MESSAGES/inkscape.mo
The paths do not match, and it results in Inkscape not being able to find locale files at runtime.

3) revision r12159 + fix-locale-path.patch:
| Chillida:mp-quartz-autotools-2 su_v$ grep locale build/config.h
| Chillida:mp-quartz-autotools-2 su_v$
and
| Making install in po
| installing am.gmo as /Volumes/magenta/src/inkscape/inkscape-repo/mp-quartz-autotools-2/inst/lib/locale/am/LC_MESSAGES/inkscape.mo
With your patch, the locale dir is no longer defined in config.h, the files are still installed into the wrong path, and Inkscape still cannot find the locale files at runtime (assuming an unchanged shell environment without any variables set specifically for Inkscape trunk).

4) Known workarounds
4a) Manually linking '$prefix/lib/locale' to '$prefix/share/locale' after 'make install' allows Inkscape to find the translation files, but this is a hack, and should not be used as 'fix' for this problem (IMHO).
4b) define $INKSCAPE_LOCALEDIR (set it to the incorrect installation path '$prefix/lib/locale') before launching inkscape from the shell (this should not be required: the binary is supposed to find the locale resources without having to resort to the built-in feature to allow _overriding_ the built-in path at runtime e.g. for full relocatable OS X application bundles)

5) Possibly fixes
5a) ship a patched intltool macro (I would not know how to prevent it from getting replaced when running autogen.sh)
5b) patch 'po/Makefile.in.in' (but isn't that an autogenerated file, too?)
5c) revert to Glib's gettext for now, until this is fixed upstream (?)

AFAICT the problem I'm trying to describe here matches the description in bug #398571 for intltool (Note: I don't know if it's exactly the same underlying issue): it seems to be a bug in intltool's macros (on non-Gnome platforms?), which does not occur when using Glib's gettext version.

[su_v (suv-lp)]

On 2013-02-28 10:09 +0100, ~suv wrote:
> 1) The problem:
> On some platforms, the locale files get installed into the wrong path
> (based on "wrong" DATADIRNAME from intltool macros (?))

This could of course also be due to something borked in my local build setups - OTOH, it didn't reveal major mistakes in the past, and I get the same results whether using my custom build script (on Lion), or using the one included in 'packaging/macosx' (on Leopard, with older versions of most dependencies in a "frozen state" MacPorts tree).

[Alex Valavanis (valavanisalex)]

OK sorry... I got the wrong end of the stick. Another possible fix, which is a variant on 5b:

Add a line to the end of autogen.sh to patch the generated po/Makefile.in.in? Should just be a simple sed line. That way, we wouldn't have to ship the generated file.

[Alex Valavanis (valavanisalex)]

Here's a new patch that corrects the line in the generated po/Makefile.in.in

Fancy trying it out?

[su_v (suv-lp)]

Inkscape 0.48+devel r12161 + fix-locale-path-v2.patch AFAICT works as expected: locale files are now installed into '$prefix/share/locale' again, as it used to be before revision 12150.

Tested so far on OS X 10.7 Lion with these dependency versions:
autoconf 2.69, automake 1.12.2/1.13.1, gettext 0.18.1.1, intltool 0.50.2

Tests on Mac OS X 10.5.8 will follow, with these dependency versions:
autoconf 2.68.2, automake 1.11.1, gettext 1.18.1.1, intltool 0.40.6

I'm a bit worried though that the patch could interfere on other platforms (or e.g. fail with older intltool versions), and am wondering how it works with other projects:
If I check the files installed by dependencies in my local MacPorts tree(s), I currently only find gdl3, gtksourceview3 and libpeas which installed the locale files into MacPorts' "$prefix/lib/locale" (these are local ports, not from upstream MacPorts repo) - all other ports install into MacPorts' "$prefix/share/locale". Either MacPorts does some internal magic (?) during the destroot phase (unlikely), or most projects handle this slightly different than Inkscape trunk >= r12150 (i.e. they didn't make the switch from Glib's gettext to upstream gettext yet, or have a custom fix, or use different macros)?

[su_v (suv-lp)]

> Inkscape 0.48+devel r12161 + fix-locale-path-v2.patch AFAICT works as expected (…)

Tested also ok on
- Mac OS X 10.5.8 (32bit, Intel), with these dependency versions:
autoconf 2.68.2, automake 1.11.1, gettext 1.18.1.1, intltool 0.40.6

- Ubuntu 12.10 (64bit, VM), with these dependency versions:
autoconf 2.69-1ubuntu1, automake 1:1.11.6-1ubuntu1, autopoint 0.18.1.1-9ubuntu1, gettext 0.18.1.1-9ubuntu1, intltool 0.50.2-2

[Alex Valavanis (valavanisalex)]

@suv - It seems like most GNOME projects ship their own po/Makefile.in.in and use GLib Gettext.

Shall we go ahead with the patch or wait for further testing?

[su_v (suv-lp)]

> Shall we go ahead with the patch or wait for further testing?

I have exhausted my available options for testing (patch works as expected) - maybe commit now and watch out for bug reports related to the installation of the locale files (so that any issues could be addressed in time before the next major release)?

[Alex Valavanis (valavanisalex)]

Patch committed in lp:inkscape r12163. Yes, let's keep an eye out for any related bug report.