getOutputFrom() failure terminates builds

[Jeff Johnson]

Tracker

[FTBFS (ftbfs-redhat-bugs)]

+++ This bug was initially created as a clone of Bug #564757 +++

tex-musixtex-0.114-8.fc13.src.rpm Failed To Build From Source against the rawhide tree. See http://fedoraproject.org/wiki/FTBFS for more information.
If you believe this is actually a bug in another package, do NOT change the component in this bug or close this bug. Instead, add the appropriate bug number from the other package to the "Depends on" line in this bug. If the other package does not yet have a bug created that you think matches, please create one. Doing so helps us properly track bugs and their dependencies, just as we track package dependencies. (If you close this bug, and the other package is not fixed before the next FTBFS run, a new bug will get created. Please follow the above advice to avoid such duplication.)

--- Additional comment from <email address hidden> on 2010-02-13 11:49:36 EST ---

Created an attachment (id=391559)
root.log

root.log for x86_64

--- Additional comment from <email address hidden> on 2010-02-13 11:49:37 EST ---

Created an attachment (id=391560)
build.log

build.log for x86_64

--- Additional comment from <email address hidden> on 2010-02-13 11:49:38 EST ---

Created an attachment (id=391561)
mock.log

mock.log for x86_64

--- Additional comment from <email address hidden> on 2010-02-13 13:10:16 EST ---

This one built fine for me
   http://koji.fedoraproject.org/koji/taskinfo?taskID=1983690

I think there was a problem with the builder when you folks built it because the build.log tails:
   + rm -rf /builddir/build/BUILDROOT/tex-musixtex-0.114-8.fc13.x86_64/usr/share/doc/tex-musixtex-0.114
   + /bin/mkdir -p /builddir/build/BUILDROOT/tex-musixtex-0.114-8.fc13.x86_64/usr/share/doc/tex-musixtex-0.114
   + cp -pr MusiXTeXDistribution/copying changelog.txt /builddir/build/BUILDROOT/tex-musixtex-0.114-8.fc13.x86_64/usr/share/doc/tex-musixtex-0.114
   + exit 0
   getOutputFrom(): Broken pipe
   Child returncode was: 1
   EXCEPTION: Command failed. See logs for output.

Shall we just close the bug?

--- Additional comment from <email address hidden> on 2010-02-13 15:32:55 EST ---

I built everything, and then re-built all the ones that failed. This one failed to build twice for me. I don't then know if it's a mock bug, or something else, but will need to investigate.

Do not close this here; I'll dupe it into a mock bug for now.

[FTBFS (ftbfs-redhat-bugs)]

google points to getOutputFrom() failures dating back to 2002, as a problem in rpmbuild. Moving this there.

[Dan (dan-redhat-bugs)]

I have a package (zabbix) where I see this error quite often.

[Jeff (jeff-redhat-bugs)]

(from ancient memories)

If you are seeing getOutputFrom() EPIPE errors,
you are likely overriding RPM filters that are
exiting before rpm has a chance to write files.

Try reading from stdin in the filter, which will
give rpm to finish writing.

[Orcan (orcan-redhat-bugs)]

Any workarounds for when we are not messing with filters at all?

[Jeff (jeff-redhat-bugs)]

Sure, but the "workaround" is largely equivalent to a "fix".

Since getOutputFrom() is being called, an external script is being invoked.

You need to find which script is being invoked.

The annoyance is that this is a race condition" a script is exiting before
rpmbuild has a chance to get ready to read. Otherwise one could just
add

[Jeff (jeff-redhat-bugs)]

(continued)
...
add
    set -x
or equiv and figger which script was exiting. But that
will change all the timing and so will cause rpmbuild to "work".

(aside)
Adding
   sleep 2
at beginning of all helper scripts might be a workaround.
Otherwise make sure all child helpers read _ALL_ of stdin
before exiting.

The next level of "fix" is to adjust the timers to fdReadable/fdWritable
or increase the size of the output write buffer in getOutputFrom().

IIRC, shortening the fdWritable timer is part of what is needed so that rpmbuild
can write the entire file list before the helper exits.

THe best fix of all is not write/read from parent <-> child in the same
loop in rpmbuild.

[Jeff (jeff-redhat-bugs)]

BTW, not only filters, but also any package that
disables the internal dependency generator, starts
to use getOutputFrom(). So the problem likely
tracks with a disabled internal dependency generator.

The othe usage case is with %doc copying, but there's
no data transferred on the helper pipe.

DIsabling the internal dependency generator and setting
one of the %_find_provides/%_find_requires extractors
to /bin/true in some package with a modestly
large (more than 8Kb perhaps much more) file list should
give you an easier reproducer than random failures in a remote
build system.

And replacing /bin/true with a short script that reads
everything from stdin before exiting will indicate the
nature of the fix.

[Jeff (jeff-redhat-bugs)]

One last note ...

Re-reading the build log in comment #1 ... the last commands displayed are
copying %doc files. Which means that all of what rpmbuild traditionally
foes with helper scripts has already been accomplished.

Note "traditionally".

Hint:
    grep rpmfcExec build/files.c
The checkfiles script is the only helper invocation after %doc is copied.

Try draining stdin always, not just when the $RPM_BUILD_ROOT dir exists:

if [ ! -d "$RPM_BUILD_ROOT" ] ; then
        cat > /dev/null
        exit 1
fi

[Panu (panu-redhat-bugs)]

The problem in case of tex-musixtex is almost certainly due to font provides extraction not eating stdin when /usr/bin/fq-query isn't there (fontconfig is not installed). That case should be fixed now in rawhide, and can be worked around by adding "BuildRequires: fontconfig".

As of zabbix often triggering this, no "obvious" candidates, unless it contains some data that libmagic thinks of as fonts. But by looks of things, there are cases of exiting without consuming all of stdin in corner cases all over the scripts...

Rawhide rpm now actually tells you what the failing script was so finding the guilty bits is easier. If/when you get more of these from rawhide, please drop a note here.

[Jeff (jeff-redhat-bugs)]

Displaying the name of a the script with the failing behavior
-- while helpful -- is hardly a fix for getOutputFrom() failures.

THe best fix of all is not write/read from parent <-> child in the same
loop in rpmbuild.

Have fun!

[Bug (bug-redhat-bugs)]

This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we may not be able to fix it before Fedora 12 is end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora please change the 'version' of this
bug to the applicable version. If you are unable to change the version,
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

[Bug (bug-redhat-bugs)]

Fedora 12 changed to end-of-life (EOL) status on 2010-12-02. Fedora 12 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

[Jeff Johnson (n3npq)]

Move to High to steal the rpm.org implementation