ubuntu-docs build failure - possible solution

Bug #1869814 reported by Gunnar Hjalmarsson
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
libxml2
New
Unknown
libxml2 (Ubuntu)
Medium
Gunnar Hjalmarsson
ubuntu-docs (Ubuntu)
Undecided
Gunnar Hjalmarsson

Bug Description

ubuntu-docs failed to build during the second Focal Fossa test rebuild:

https://launchpad.net/ubuntu/+archive/test-rebuild-20200327-focal/+build/18982738

It was itstool which failed to generate the localized zh_CN pages. If dropping zh_CN from ubuntu-help/Makefile.am, ubuntu-docs builds. But excluding the Chinese pages is apparently not an acceptable solution.

The itstool package in focal {build-,}depends on python3. I built itstool in a PPA with python2 instead, and in that PPA ubuntu-docs built fine without modifications:

https://launchpad.net/~gunnarhj/+archive/ubuntu/ubuntu-docs/+packages

So the root cause seems to lie neither in ubuntu-docs nor itstool. I would suspect that python3-libxml2 is buggy, but since itstool was built against python2 in eoan, it's hard to tell if it's a regression.

Would building itstool against python2 be an acceptable workaround in focal?

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

@Boyuan: Do you have any thoughts on the above?

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Hmm.. I saw in the buildlog from the successful build that it skipped the Chinese translation of a particular string. The reason proved to be a missing '<' character, i.e. the translated string contains an instance of '/em>' instead of '</em>'. And that's the correct behavior; it should check the Mallard syntax and skip translations with invalid syntax.

Anyway, I added the missing '<', and then ubuntu-docs built with the itstool version in focal.

So the direct reason for the build failure is a syntactically incorrect translation. OTOH, translators make mistakes, and a bad translation *should not* make the whole package build to fail.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

@Shaun: Have you possibly stumbled into anything similar when building e.g. gnome-user-docs?

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Uploaded a new version of ubuntu-docs, which builds on focal despite the issue discussed in this bug report.

Changed in ubuntu-docs (Ubuntu):
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
status: New → Fix Committed
tags: added: focal regression-release
Revision history for this message
Sebastien Bacher (seb128) wrote :

It's segfault in libxml code, their binding doesn't seem to handle invalid files well, somewaht similar to https://gitlab.gnome.org/GNOME/libxml2/-/issues/12

Changed in itstool (Ubuntu):
status: New → Invalid
Revision history for this message
Boyuan Yang (hosiet) wrote :

@Gunnar It seems that the root cause has been found, thanks! It's well known that translation could be fragile when the build system is not well defined; all I can think of is to make the build system robust enough and yell out loudly enough whenever an error (e.g., segfault) happens.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Thanks Sebastien!

Or https://gitlab.gnome.org/GNOME/libxml2/issues/64 (which is not yet closed).

@Boyuan: Well, it does scream loudly in the sense that the build fails.

I wouldn't blame the build system in this case. It has worked in a sensible and robust way for many years. The segfault issue only happens with python3-libxml2, not with python-libxml2. And I have the impression (not yet confirmed) that the segfault only happens for XML syntax errors in Chinese translations; for similar errors in non-Chinese translations it only prints a warning and moves on (which is the desired behavior).

Revision history for this message
Shaun McCance (shaunm-gnome) wrote :

So in general, syntax errors in translations just cause a warning, and the original text is used. itstool only errors on syntax if you pass --strict, and yelp.m4 doesn't even give you a way to pass --strict.

There's a chance this is itstool's fault. You do still have to do memory management with libxml2's Python bindings, so stuff like double frees in application code is possible. But since this crash happens during error reporting for a language with many-byte UTF-8 chars, and since it weirdly only happens with Python 3, I suspect it's this libxml2 bug:

https://bugzilla.gnome.org/show_bug.cgi?id=789714

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Thanks Shaun! It's indeed that bug. Why am I not surprised when seeing that you filed it? :)

I grabbed a Fedora patch and submitted a Debian merge request:

https://salsa.debian.org/xml-sgml-team/libxml2/-/merge_requests/3

It looks pretty spot on, don't you think? At least, with that patch applied I can build ubuntu-docs 19.10.2 successfully again. A syntax error in one of the Chinese translations is reported as warnings as expected.

Changed in libxml2 (Ubuntu):
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
importance: Undecided → Medium
status: New → In Progress
Matthias Klose (doko)
tags: added: ftbfs
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-docs - 20.04.1

---------------
ubuntu-docs (20.04.1) focal; urgency=medium

  * scripts/update-translations.sh:
    - Handle tarballs also from the package
  * Bump version to 20.04 / focal
  * debian/control.in, debian/control:
    - Replace Vcs-Bzr with Vcs-Git
    - Bump Standards-Version to 4.5.0
  * New languages: af, ckb
  * Translations from Launchpad:
    - Syntactically incorrect Chinese translations of two strings were
      corrected before this translation update, which works around the
      focal build failure as described in LP: #1869814.

 -- Gunnar Hjalmarsson <email address hidden> Tue, 31 Mar 2020 14:11:00 +0200

Changed in ubuntu-docs (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Sebastien Bacher (seb128) wrote :

@Gunnar, looks like the patch I backported earlier in the cycle
https://launchpad.net/ubuntu/+source/libxml2/2.9.4+dfsg1-8ubuntu3

But apparently doko synced from Debian overlooking we had a delta...

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

@Sebastien: Indeed, it's the very same thing. Then I spent several hours on an issue that was already spotted and fixed. :(

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

To the sponsor:

libxml2 2.9.10+dfsg-5, including the patch, is now available in Debian unstable. So please sync libxml2.

Mathew Hodson (mhodson)
no longer affects: itstool (Ubuntu)
Changed in libxml2:
status: Unknown → New
Revision history for this message
Graham Inggs (ginggs) wrote :

sync'd. please close this bug once libxml2 has been accepted
https://launchpad.net/ubuntu/focal/+queue?queue_state=1

Matthias Klose (doko)
Changed in libxml2 (Ubuntu):
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libxml2 - 2.9.10+dfsg-5

---------------
libxml2 (2.9.10+dfsg-5) unstable; urgency=medium

  * Team upload.

  [ Mattia Rizzolo ]
  * d/rules:
    + Drop --disable-silent-rules, already passed by dh_auto_configure.
    + Drop --parallel, now default with debhelper compat > 10.
    + Use dh_installdocs and dh_installexamples to install docs and examples.
    + Use dh_missing --fail-missing (and add the relevant d/not-installed).
    + Minimize indep build to build only the docs.
  * d/watch: fix an option to avoid a warning message.
  * d/control:
    + Move most of the build-deps to Build-Depends-Arch.
    + Use ${python:Depends} also for python-libxml2-dbg.
  * Add a lintian override for
    debian-rules-uses-supported-python-versions-without-python-all-build-depends

  [ Gunnar Hjalmarsson ]
  * d/p/python3-unicode-errors.patch:
    Fix segfault issue with itstool and py3. LP: #1869814

 -- Mattia Rizzolo <email address hidden> Fri, 10 Apr 2020 14:53:23 +0200

Changed in libxml2 (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.