rpmbuild generates perl provides/depends differently in F-10 than any prior leve

[Jeff Johnson]

tracker

[Chris (chris-redhat-bugs)]

Description of problem:

rpm in F10+ processes Perl depends/requires differently than in any prior level of Fedora.

e.g., see koji scratch builds of perl-Data-TreeDumper:

dist-f9: https://koji.fedoraproject.org/koji/taskinfo?taskID=965620
dist-f10: https://koji.fedoraproject.org/koji/taskinfo?taskID=965622
dist-f11: https://koji.fedoraproject.org/koji/taskinfo?taskID=965644

Note dist-f9's list of provides/requires from the build.log:

Provides:
  perl(Data::TreeDumper) = 0.35
Requires:
  perl >= 0:5.006
  perl(:MODULE_COMPAT_5.10.0)
  perl(Carp)
  perl(Check::ISA)
  perl(Class::ISA)
  perl(Exporter)
  perl(Sort::Naturally)
  perl(Text::Wrap)
  perl(strict)
  perl(warnings)

...and dist-f10, f11:

Provides:
  perl(Data::TreeDumper) = 0.35
  perl(Potatoe)
  perl(SuperObject)
  perl(SuperObjectWithAutoload)
  perl(TiedArray)
  perl(TiedHandle)
  perl(TiedHash)
  perl(TiedScalar)
  perl(Vegetable)
Requires:
  perl >= 0:5.006
  perl(AutoLoader)
  perl(Carp)
  perl(Check::ISA)
  perl(Class::ISA)
  perl(Data::TreeDumper)
  perl(Devel::Peek)
  perl(Exporter)
  perl(Sort::Naturally)
  perl(Text::Wrap)
  perl(Tie::Array)
  perl(Tie::Handle)
  perl(Tie::Hash)
  perl(Tie::Scalar)
  perl(strict)
  perl(warnings)

Note the additional "provides" found under f10+ (e.g. perl(Potatoe)) are from scripts stored in %_docdir, without their executable bit set; also note the additional requires (e.g. perl(Tie::Array)) not present in the package built under dist-f9. This is a major change of behaviour from all versions prior to F-10, and may result in many of the nearly 1000 Perl packages in Fedora to start generating unexpected requires/provides when they're rebuilt.

Version-Release number of selected component (if applicable):
rpm-4.6.0-0.rc1.7

[Panu (panu-redhat-bugs)]

Most likely "file" now thinks those files are perl whereas older versions didn't - duplicate or at least closely related to bug 470811 and bug 234691.

[Ralf (ralf-redhat-bugs)]

/me thinks the bug is in rpm.

It should simply ignore anything below /usr/share/doc/ in its dep-tracking.

[Panu (panu-redhat-bugs)]

Yes, rpm bug(s) too. Dependency extraction based on executable/not executable is nice and simple rule except when it breaks, such as in case of perl modules which aren't supposed to be marked as executable but want dependency extraction anyway.

[Ralf (ralf-redhat-bugs)]

This actually isn't the problem, here.

The problem is people are shipping "documentary scripts" as part of a package's documentation. I don't see anything wrong with this, except of rpm bogusly pulling in deps on such files.

[Panu (panu-redhat-bugs)]

It's the same thing really. IF executable / no executable bit were reliable for disabling dependency extraction, this bug wouldn't exist and ditto for the two other bugs mentioned in comment #1. And of course, "executable documentation" is pretty strange idea anyway... :)

In case the above appears unparseable mumble-jumble to you: I do agree, extracting dependencies from documentation makes no sense.

[Ralf (ralf-redhat-bugs)]

(In reply to comment #5)
> #1. And of course, "executable documentation"
> is pretty strange idea anyway... :)
Pardon, but I have to disagree. Having scripts as part of a documentation is a natural thing. Forcing them to be non-executable is a packaging defect.

(cf. the samba case.)

[Panu (panu-redhat-bugs)]

Documentation is for humans to read. If a script is truly just documentation then packaging it as executable is just as much a defect as packaging man pages with executable bits set. But that's well into the land of "spec and packaging hygiene and policies" department which I've zero interest in arguing over, and little to do with the actual issue here.

[Ralf (ralf-redhat-bugs)]

(In reply to comment #7)
> Documentation is for humans to read. If a script is truly just documentation
> then packaging it as executable is just as much a defect as packaging man pages
> with executable bits set.

Documentation containing non-executeable scripts are crippled docs.

> But that's well into the land of "spec and packaging
> hygiene and policies" department which I've zero interest in arguing over, and
> little to do with the actual issue here.

 Note that the patch does not change
> any type definitions, and by making the logic for limit macros match that
> for the corresponding type definitions more closely it makes the header
> much easier to check by inspection than it presently is.
>
You have, because rpm is your job and because it's your rpm which
* is producing broken results.
* forces users to cripple their packages.

[Panu (panu-redhat-bugs)]

In comment #5 I wrote:
> In case the above appears unparseable mumble-jumble to you: I do agree,
> extracting dependencies from documentation makes no sense.

Which part of this was not clear to you?

[Chris (chris-redhat-bugs)]

(In reply to comment #7)
> Documentation is for humans to read. If a script is truly just documentation
> then packaging it as executable is just as much a defect as packaging man pages
> with executable bits set. But that's well into the land of "spec and packaging
> hygiene and policies" department which I've zero interest in arguing over, and
> little to do with the actual issue here.

In this case, the included scripts under %_docdir are not marked as executable; Ralf's comment that bits under %_docdir shouldn't be scanned for dep tracking is with merit, methinks.

The real issue here is that this change in behaviour has potential implications for dep/req generation for all packages in Fedora with anything under %_docdir that was previously not scanned.

I would think a good resolution for this would either be:

1) revert to the prior behaviour; or
2) allow certain directories to be easily excluded from autodep/prov, and default %_docdir to being excluded.

Personally, I'd prefer #2 :)

[Chris (chris-redhat-bugs)]

(In reply to comment #10)

> The real issue here is that this change in behaviour has potential implications
> for dep/req generation for all packages in Fedora with anything under %_docdir
> that was previously not scanned.

I should try to be more clear here. It's not about executable documentation, or what should constitute documentation; it's that this change in behaviour causes previously clean and compliant packages to run afoul of the Fedora packaging guidelines. This change impacts how dep/reqs are generated for every single package.

[Jeff (jeff-redhat-bugs)]

This is the quick fix for not extracting perl(...) dependencies for perl pod's
in /usr/share/doc. This issue was "fixed" like 3+ years ago.

- == rpm.org rpmfc.c
+ == rpm5.org rpmfc.c

@@ -744,361 +748,126 @@
     (void) fclose(fp);

     if (fc->fcolor->vals[fc->ix] & RPMFC_PERL) {
- if (fc->fcolor->vals[fc->ix] & RPMFC_MODULE)
- xx = rpmfcHelper(fc, 'P', "perl");
- if (is_executable || (fc->fcolor->vals[fc->ix] & RPMFC_MODULE))
- xx = rpmfcHelper(fc, 'R', "perl");
- }
+ if (strncmp(fn, "/usr/share/doc/", sizeof("/usr/share/doc/")-1)) {
+ if (fc->fcolor->vals[fc->ix] & RPMFC_MODULE)
+ xx = rpmfcHelper(fc, 'P', "perl");
+ if (is_executable || (fc->fcolor->vals[fc->ix] & RPMFC_MODULE))
+ xx = rpmfcHelper(fc, 'R', "perl");
+ }
+ } else
...

[Panu (panu-redhat-bugs)]

Chris, no need to clarify. Also the rules didn't suddenly change. Perl modules
are a corner case where the normal rules of dependency extraction wrt
executable bits don't apply, never have (perl modules generally aren't supposed
to be executable so that can't be used for detection). The current "file"
utility (whose library rpmbuild internally uses for classifying files) just
seems to be far more eager at claiming files to be "perl modules" than the
older versions, that's where the change in behavior comes from in this and the
other related bugs.

Like I said, extracting dependencies for documentation doesn't make sense to me
either. But the rpmbuild file classifier is built with the assumption that it
can figure out everything from the files on disk, it doesn't "see" what is
specified as %doc etc in the spec. I'm just looking at how to best fix this,
hopefully eliminating bunch of other related hacks in the process instead of
adding new ones. If that starts looking like more of a long-term thing, we can
always just drop in another hardcoded hack such as the "fix" in comment #12 to exclude the docdir where necessary.

[Chris (chris-redhat-bugs)]

Aahhh, ok, that makes much more sense now. Also, the files it's flagging aren't Perl modules per se, but rather scripts that happen to contain embedded package declarations (modules).

[Bug (bug-redhat-bugs)]

This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

[Bug (bug-redhat-bugs)]

This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we may not be able to fix it before Fedora 11 is end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora please change the 'version' of this
bug to the applicable version. If you are unable to change the version,
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

[Bug (bug-redhat-bugs)]

Fedora 11 changed to end-of-life (EOL) status on 2010-06-25. Fedora 11 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.