Bug #336634 “libxxf86vm-dev should depend on libxext-dev” : Bugs : pkg-config

Revision history for this message

In freedesktop.org Bugzilla #4738, Mikhail-zabaluev (mikhail-zabaluev) wrote on 2005-11-21:

#1

I think this needs some more clarification.
This problem arises in distributions, where it's common to separately package
files needed at build time only. Thus, a .pc file for a private dependency may
not be present (as it's in the appropriate -devel or -DEV package), even though
the library files are there.

Revision history for this message

In freedesktop.org Bugzilla #4738, Mikhail-zabaluev (mikhail-zabaluev) wrote on 2005-11-21:

#2

Created an attachment (id=3860)
a quick fix

This patch disables processing of Requires.private in all modes unless --static
option is in effect.
Right now, I'm not so accustomed with the code as to make more discriminate
changes.

P.S. There is an imperfection in the parser code that this patch also amends.

Revision history for this message

In Debian Bug tracker #340904, Loïc Minier (lool) wrote on 2005-11-26: Inclusion of .private's "-I"s

#3

Package: pkg-config
Version: 0.20-1
Severity: normal

Hi,

(we had this discussion on IRC already)

It might be nice to use "-I" of the .private pseudo-headers in .pc
files each time some -I flags are requested, eg. not only for static
builds.

As I understand it, the introduction of .private pseudo-headers was
meant to lower the number of ELF DEPENDS as caused by the use of "-l"
flags when linking objects with the help of pkg-config.
These flags are only needed when linking statically, and pkg-config
offers a "--static" flag to choose the type of linking one wants.

The problems arise for header files of a package distributing a .pc
file when these are including third party headers; for example, the
vte headers include the Atk, Pango, Gtk and other headers.

While there might be some discussion on whether the ELF DEPENDS should
reflect these header inclusions, it seems quite safe to assume that all
-I, from private or non private headers, should be present when
building objets, for static linking or not.

Bye,

--
Loïc Minier <email address hidden>

Revision history for this message

In Debian Bug tracker #340904, Loïc Minier (lool) wrote on 2005-12-14:

#4

Download full text (7.2 KiB)

On sam, nov 26, 2005, Loic Minier wrote:
> (we had this discussion on IRC already)

More bits, the new information here is that people involved in
pkg-config think that using headers of another lib in your headers
means you have a public dep on this other lib, and that will result in
an ELF dep. It also clarifies that the improvement is because of the
removal of the static flags which caused dynamic deps, and won't
improve gtk-ish/gnome-ish libs.

21:55 < lool> vorlon: if you have two minutes, could you please send a word to
              <http://bugzilla.gnome.org/show_bug.cgi?id=322240>?
21:57 < Q_> lool: Still waiting for a fix so I can build gtk-sharp2.
21:58 < lool> Q_: well, I'm still waiting for a consensus to be reached, I
              think the stronger opinion was to add only -Is from private fields
21:58 < lool> but this is all too borderline for me to take the decision, one
              way or the other
22:00 < Q_> Mithrandir: Around?
22:01 < lool> Q_: BTW, I filed #340904 to track the story
22:02 < vorlon> lool: what word are you looking for? :)
22:03 < lool> vorlon: the kind of word carrying final-person opinion (eg where
              I don't relay opinions from third parties :)
22:04 < lool> this is really a pkg-config design question for me, and the
              technical implementation is not the problem, so I'd rather have
              you or Mithrandir or who ever feels he knows what is correct
              express himself there, than me
22:05 < vorlon> lool: well, Mithrandir and I seem to disagree here on what the
                correct design is, and one of is is pkg-config upstream whereas
                the other is not. :)
22:06 < Q_> lool: And nothing happened to it yet.
22:06 < lool> vorlon: exactly, and since I'm not strongly convinced one way or
              the other, I can't represent either party in the discussion O:-)
22:08 < vorlon> lool: well, I don't see much point in trying to argue the
                pkg-config design details in a bug report against *vte*: that's
                not fair to the vte maintainers, and it's not a very effective
                means of getting pkg-config changes made. I don't feel
                strongly enough about this corner case today to argue it at
                *all*, really.
22:10 < lool> vorlon: well, I wanted your take on the situation to vte's
              upstream which is now victim of the change too, I can try to
              explain the situation myself to vte's upstream, but I will be
              short of arguments what ever camp vte's upstream picks
22:10 * lool bravely goes telling vte maintainer that we've pulled them in a
          mid-term problem
22:12 < Mithrandir> Q_: yes
22:13 < vorlon> lool: I don't think vte upstream is going to pick the "I'll let
                my software continue to be broken" camp
22:14 < lool> exactly
22:14 < Q_> Mithrandir: So, did you have some time to think about the problem,
            and the best way to solve it?
22:14 < dilinger> i wish i could admin some AIX machines
22:15 < dilinger> i bet it would cause me to get a lot less frustrated when i'm
                  faced ...

On sam, nov 26, 2005, Loic Minier wrote:
>  (we had this discussion on IRC already)

More bits, the new information here is that people involved in
 pkg-config think that using headers of another lib in your headers
 means you have a public dep on this other lib, and that will result in
 an ELF dep.  It also clarifies that the improvement is because of the
 removal of the static flags which caused dynamic deps, and won't
 improve gtk-ish/gnome-ish libs.

21:55 < lool> vorlon: if you have two minutes, could you please send a word to 
              <http://bugzilla.gnome.org/show_bug.cgi?id=322240>?
21:57 < Q_> lool: Still waiting for a fix so I can build gtk-sharp2.
21:58 < lool> Q_: well, I'm still waiting for a consensus to be reached, I 
              think the stronger opinion was to add only -Is from private fields
21:58 < lool> but this is all too borderline for me to take the decision, one 
              way or the other
22:00 < Q_> Mithrandir: Around?
22:01 < lool> Q_: BTW, I filed #340904 to track the story
22:02 < vorlon> lool: what word are you looking for? :)
22:03 < lool> vorlon: the kind of word carrying final-person opinion (eg where 
              I don't relay opinions from third parties :)
22:04 < lool> this is really a pkg-config design question for me, and the 
              technical implementation is not the problem, so I'd rather have 
              you or Mithrandir or who ever feels he knows what is correct 
              express himself there, than me
22:05 < vorlon> lool: well, Mithrandir and I seem to disagree here on what the 
                correct design is, and one of is is pkg-config upstream whereas 
                the other is not. :)
22:06 < Q_> lool: And nothing happened to it yet.
22:06 < lool> vorlon: exactly, and since I'm not strongly convinced one way or 
              the other, I can't represent either party in the discussion  O:-)
22:08 < vorlon> lool: well, I don't see much point in trying to argue the 
                pkg-config design details in a bug report against *vte*: that's 
                not fair to the vte maintainers, and it's not a very effective 
                means of getting pkg-config changes made.  I don't feel 
                strongly enough about this corner case today to argue it at 
                *all*, really.
22:10 < lool> vorlon: well, I wanted your take on the situation to vte's 
              upstream which is now victim of the change too, I can try to 
              explain the situation myself to vte's upstream, but I will be 
              short of arguments what ever camp vte's upstream picks
22:10  * lool bravely goes telling vte maintainer that we've pulled them in a 
          mid-term problem
22:12 < Mithrandir> Q_: yes
22:13 < vorlon> lool: I don't think vte upstream is going to pick the "I'll let 
                my software continue to be broken" camp
22:14 < lool> exactly
22:14 < Q_> Mithrandir: So, did you have some time to think about the problem, 
            and the best way to solve it?
22:14 < dilinger> i wish i could admin some AIX machines
22:15 < dilinger> i bet it would cause me to get a lot less frustrated when i'm 
                  faced w/ crap like redhat and suse
22:15 < Mithrandir> Q_: oh, the requires.private stuff?
22:16 < Q_> Mithrandir: Yes, the -I
22:17 < Mithrandir> 02:39 < jamesh> Mithrandir: about your pkg-config question: 
                    no.  If a package is a private dependency, then its 
                    interface shouldn't be exposed at all.
22:17 < Mithrandir> 02:39 < jamesh> Mithrandir: so the -I flags shouldn't be 
                    included
22:18 < Q_> You mean we should try to convince jamesh?
22:18 < Mithrandir> I'm somewhat inclined to agree with jamesh, but neuro had 
                    some comments which went in the other direction.  I think 
                    I'll have to sit down and look at what can go wrong in the 
                    different situations.
22:18 < Mithrandir> yes, if you convince jamesh, I'm ok with it.  He knows a 
                    bloody lot about how pkg-config works, is supposed to work 
                    and why it works that way.
22:19  * lool wishes for a clearer definition of the goal of the 
          requires.private header
22:21 < vorlon> lool: well, this opinion from jamesh is consistent with 
                treating glib as a *public* dependency of gtk just because 
                software can cheat and use glib API calls directly when only 
                including gtk. :P
22:23 < Mithrandir> vorlon: yes, but what if glib was a private dependency of 
                    gtk?  Would it be ok to expose, say, gchar*?
22:24 < Mithrandir> (or better, gchar, since a pointer is always ok)
22:25 < vorlon> Mithrandir: if gchar is used in the gtk headers, yes. :)
22:25 < Q_> Mithrandir: A pointer isn't any better.
22:25 < vorlon> Q_: it's legal to have pointers in C that you don't have enough 
                information to be able to dereference.
22:26 < Mithrandir> Q_: sure it is, as long as you don't expose the typedef.
22:26 < Mithrandir> so you can't dereference it, as vorlon says.
22:27 < Q_> vorlon: But you still need to say that it's a type.
22:28 < Mithrandir> Q_: a struct would be a better example and then you can 
                    just say struct foo; and use struct foo* without any 
                    trouble.
22:28 < Q_> so something like "struct foo *bar;" works, "struct foo bar;" 
            doesn't.
22:29 < Q_> But at that point you don't even need the include in the first 
            place.
22:30 < lool> vorlon: if you look at private/public deps in this way, then my 
              feeling is that this clarifies deps for static linking, but won't 
              help reducing dynamic deps
22:31 < vorlon> lool: ?
22:31 < Mithrandir> lool: and we want to reduce dynamic deps.  Possibly not 
                    perfectly (stuff linked to gtk will still link to glib, 
                    since it's an actual public dependency), but better than 
                    today.
22:31 < vorlon> lool: which way?
22:32 < lool> vorlon: in the way of jamesh proposing to treat such interfaces 
              as public deps
22:32 < lool> anyone minds if I copy paste this log to the bug?
22:33 < Mithrandir> lool: feel free
22:33 < lool> Mithrandir: true, it's just sad this helps so little in the case 
              of gtk-ish libs
22:35 < Q_> Anyway, I think the point was if that gchar is part of gtk headers, 
            and gchar changes in such a way that it breaks gtk, and so also 
            things linked to gtk, that both got an api change, and both should 
            do an soname change.
22:35 < vorlon> lool: it's less than ideal, but there are still *some* lib 
                savings.
22:36 < Mithrandir> I would like to accomodate the gnome/gtk model of handling 
                    headers, though.
22:36 < Q_> And that might not be something the gnome people would like to do?
22:37 < lool> Q_: well they included vorlon's patch quite rapidly
22:37 < lool> Q_: I suppose they will use any pkg-config coolness you explain 
              them to use
22:38 < Q_> lool: I think they don't want to and change all their sonames 
            because something in glib changed.
22:38 < Q_> s/to/to go/

-- 
Loïc Minier <lool@dooz.org>

Revision history for this message

In Debian Bug tracker #340904, Loïc Minier (lool) wrote on 2005-12-15:

#5

Download full text (3.7 KiB)

(End of the IRC discussion:)

22:42 < Q_> But it's the right way to do it, just because they can relink
            everything to the new glib, it's not the right way to do it.
22:43 < vorlon> Q_: actually, IME they'll use any excuse to change sonames
                between stable branches
22:44 < Q_> They do?
22:44 < Q_> Then I don't see why they would have a problem with it.
22:46 < Q_> Mithrandir: Do you have any idea around what time jamesh is around
            ussualy?
22:46 < Mithrandir> if you have App A, lib B and lib C. B links to C. A uses
                    B directly, but also dereferences structs which it has
                    gotten from C, through B. C then changes soname (new ABI,
                    same API), but not removed from the system, B is recompiled
                    on the system without any soname changes. Is there any way
                    you can make A continue working?
22:46 < Mithrandir> Q_: he lives in western .au, so he might be around in a
                    couple of hours.
22:47 < lool> Mithrandir: ah you make me think of something really painful,
              there are weird macros accessing structs in headers
22:48 < lool> so including headers of lib C in lib B might be enough for app A
              to get smoe code dependent on the structs in C added to it and
              hence crashing with a newer C
22:49 < Mithrandir> I'm wondering if my case will work correctly if A is linked
                    with both B and C and C has versioned symbols.
22:49 < Q_> Mithrandir: I don't think you can keep A working without also
            relinking it.
22:49 < lool> I'm not sure I have a real life example, but I think the fact
              it's possible might be enough to try to guard against it
22:56 < vorlon> Mithrandir: in that scenario, why is it *not* a bug for B to be
                recompiled without an soname change?
22:57 < Q_> Mithrandir: A will get linked to 2 lib C's in that case.
22:57 < Mithrandir> Q_: that's fine if they have versioned symbols, AIUI?
22:57 < vorlon> Mithrandir: if the headers from C are being included into B, it
                must be for a reason; I can't think of any non-ABI-breaking
                reasons for this other than including trivial enums or defines,
                and that just tells me C's headers should be more granular and
                B shouldn't be linking to C either.
22:58 < vorlon> hmm, ok. possibility: C makes ABI changes to parts that aren't
                referenced by B. Meh.
22:59 < Mithrandir> vorlon: I didn't talk about headers, I talked about
                    linking. A might very well include C's headers itself, to
                    get struct definitions.
22:59 < Mithrandir> I was wondering if you could make it work or not.
22:59 < Q_> Mithrandir: If A include's C's headers, it should link to it itself.
22:59 < vorlon> oh. then I guess you're only talking about the usual
                "versioned symbols" case.
23:00 < Mithrandir> vorlon: yes, I might very well be, and that works, doesn't
                    it?
23:00 < Mithrandir> Q_: yes, I meant to write that, but didn't
23:00 < vorlo...

(End of the IRC discussion:)

22:42 < Q_> But it's the right way to do it, just because they can relink 
            everything to the new glib, it's not the right way to do it.
22:43 < vorlon> Q_: actually, IME they'll use any excuse to change sonames 
                between stable branches
22:44 < Q_> They do?
22:44 < Q_> Then I don't see why they would have a problem with it.
22:46 < Q_> Mithrandir: Do you have any idea around what time jamesh is around 
            ussualy?
22:46 < Mithrandir> if you have App A, lib B and lib C.  B links to C.  A uses 
                    B directly, but also dereferences structs which it has 
                    gotten from C, through B.  C then changes soname (new ABI, 
                    same API), but not removed from the system, B is recompiled 
                    on the system without any soname changes.  Is there any way 
                    you can make A continue working?
22:46 < Mithrandir> Q_: he lives in western .au, so he might be around in a 
                    couple of hours.
22:47 < lool> Mithrandir: ah you make me think of something really painful, 
              there are weird macros accessing structs in headers
22:48 < lool> so including headers of lib C in lib B might be enough for app A 
              to get smoe code dependent on the structs in C added to it and 
              hence crashing with a newer C
22:49 < Mithrandir> I'm wondering if my case will work correctly if A is linked 
                    with both B and C and C has versioned symbols.
22:49 < Q_> Mithrandir: I don't think you can keep A working without also 
            relinking it.
22:49 < lool> I'm not sure I have a real life example, but I think the fact 
              it's possible might be enough to try to guard against it
22:56 < vorlon> Mithrandir: in that scenario, why is it *not* a bug for B to be 
                recompiled without an soname change?
22:57 < Q_> Mithrandir: A will get linked to 2 lib C's in that case.
22:57 < Mithrandir> Q_: that's fine if they have versioned symbols, AIUI?
22:57 < vorlon> Mithrandir: if the headers from C are being included into B, it 
                must be for a reason; I can't think of any non-ABI-breaking 
                reasons for this other than including trivial enums or defines, 
                and that just tells me C's headers should be more granular and 
                B shouldn't be linking to C either.
22:58 < vorlon> hmm, ok.  possibility: C makes ABI changes to parts that aren't 
                referenced by B. Meh.
22:59 < Mithrandir> vorlon: I didn't talk about headers, I talked about 
                    linking.  A might very well include C's headers itself, to 
                    get struct definitions.
22:59 < Mithrandir> I was wondering if you could make it work or not.
22:59 < Q_> Mithrandir: If A include's C's headers, it should link to it itself.
22:59 < vorlon> oh.  then I guess you're only talking about the usual 
                "versioned symbols" case.
23:00 < Mithrandir> vorlon: yes, I might very well be, and that works, doesn't 
                    it?
23:00 < Mithrandir> Q_: yes, I meant to write that, but didn't
23:00 < vorlon> Mithrandir: yes, it does.
23:00 < Q_> Mithrandir: And versioned symbols might make it work, but I don't 
            think in all cases.
23:01 < vorlon> Q_: it works unless B's API is pathological.
23:01 < vorlon> (takes as arguments objects which are defined by C)
23:03 < Mithrandir> I'm thinking that jamesh is correct, but I can't really 
                    tell you why. :-/
23:07 < Mithrandir> well, I gotta sleep.  I have some vacation in a little less 
                    than a week, I'll see if I can get to it then.  Or in the 
                    weekend.

-- 
Loïc Minier <lool@dooz.org>

Revision history for this message

In Debian Bug tracker #340904, Loïc Minier (lool) wrote on 2005-12-17: Re: Bug#343711: gdk-x11-2.0.pc spreads library dependencies needlessly

#6

block #343711 by #340904
thanks

On sam, déc 17, 2005, J.H.M. Dassen (Ray) wrote:
> Yes. I'm going by this guideline from /usr/share/doc/pkg-config/NEWS.gz:
> "Generally, if include another library's headers in your own, it's a public
> dependency and not a private one.".

Yes, that's part of the discussion we had with the pkg-config
maintainer, and that's what he suggests indeed.

However, Gdk includes Pango, which in turn includes Xft, Freetype; Gdk
also includes cairo, which in turn includes Fontconfig and Freetype,
etc. I suppose that would make these libs public dependencies of Gdk
too, following the same logic.

I think I'm going to let this change be blocked by the pkg-config
change I pointed at, until then it wouldn't be wise to introduce build
failures.

The best thing I wish is that someone with experience in ABI / API
handling, pkg-config, and shared libraries in general writes a guide to
handle situations such has: 1/ how to check a .la file is correct 2/
what to put in .pc files.

Bye,
--
Loïc Minier <email address hidden>

Revision history for this message

In Debian Bug tracker #340904, Loïc Minier (lool) wrote on 2006-01-05: block 346039 with 340904

#7

# Automatically generated email from bts, devscripts version 2.9.10
block 346039 with 340904

Revision history for this message

In Debian Bug tracker #340904, Steve Langasek (vorlon) wrote on 2006-01-23: Re: Bug#349318: fixed in xft 2.1.8.2-1

#8

Download full text (3.9 KiB)

reopen 349318
block 349318 by 340904
thanks

> * Add 001_no_export_freetype.diff. This is a modified patch from Steve
> Langasek that causes xft.pc not to export freetype or xrender as part of
> the interface, but hide them in Requires.private instead (closes: #349318)

The patch is:

Index: libXft-X11R7.0-2.1.8.2/xft.pc.in
===================================================================
--- libXft-X11R7.0-2.1.8.2.orig/xft.pc.in 2005-12-30 14:50:49.000000000 -0500
+++ libXft-X11R7.0-2.1.8.2/xft.pc.in 2006-01-22 22:12:27.000000000 -0500
@@ -6,7 +6,7 @@
Name: Xft
Description: X FreeType library
Version: @VERSION@
-Requires: xproto, xrender, fontconfig, freetype2
-Requires.private: xrender, fontconfig, freetype2
+Requires: xproto, fontconfig
+Requires.private: xrender, freetype2
Cflags: -I${includedir}
Libs: -L${libdir} -lXft

This won't work because of the aforementioned bug #340904:

$ grep -h include include/X11/Xft/*.h
#include <X11/Xfuncproto.h>
#include <stdarg.h>
#include <ft2build.h>
#include FT_FREETYPE_H
#include <fontconfig/fontconfig.h>
#include <X11/extensions/Xrender.h>
#include <X11/Xfuncproto.h>
/* #include <X11/Xosdefs.h>*/
#include <X11/Xft/XftCompat.h>
$

Both the xrender and freetype2 headers are included in Xft/Xft.h, and types
from both of these headers are exported in the Xft API -- so any software
using Xft.h needs to be able to find these other headers as well.
Currently, pkg-config does *not* pass cflags from packages listed in
Requires.private when called as pkg-config --cflags <foo>.

So far, the maintainer has resisted changing pkg-config to export cflags
(which should really be called cppflags...) from Requires.private due to a
fallacious argument regarding the nature of "private" dependencies. There
are three real use cases for libraries which depend on other libraries:

- the library intentionally exports the API and ABI of its dependencies when
  linked to, and therefore both the ldflags and the cflags of its
  dependencies should be exported by pkg-config in all cases[1]
- the library intentionally includes headers from dependencies in its own
  headers in order to inherit type definitions, but these definitions are
  not intended for direct consumption by users of this library alone;
  therefore pkg-config must export the cflags from dependencies in all
  cases, but the ldflags only when linking statically
- the library's API includes no headers from its dependencies; pkg-config
  needs to export the ldflags of private dependencies when statically
  linking but not when dynamically linking, and should *never* need to
  export the cflags of these headers.

Please note that today, the handling of Requires.private in pkg-config maps
to *none* of these cases -- I can't think of a single situation in which
cflags of dependencies are needed when statically linking, and not needed
when dynamically linking! Instead, the Requires.private handling is
adequate for the third case; handling of Requires is correct for the first
case; and there is no combination of options that is appropriate for the
second case.

This is unfortunate, because there are a great many packages that are
inheriting de...

reopen 349318
block 349318 by 340904
thanks

>    * Add 001_no_export_freetype.diff. This is a modified patch from Steve
>      Langasek that causes xft.pc not to export freetype or xrender as part of
>      the interface, but hide them in Requires.private instead (closes: #349318)

The patch is:

Index: libXft-X11R7.0-2.1.8.2/xft.pc.in
===================================================================
--- libXft-X11R7.0-2.1.8.2.orig/xft.pc.in      2005-12-30 14:50:49.000000000 -0500
+++ libXft-X11R7.0-2.1.8.2/xft.pc.in   2006-01-22 22:12:27.000000000 -0500
@@ -6,7 +6,7 @@
 Name: Xft
 Description: X FreeType library
 Version: @VERSION@
-Requires: xproto, xrender, fontconfig, freetype2
-Requires.private: xrender, fontconfig, freetype2
+Requires: xproto, fontconfig
+Requires.private: xrender, freetype2
 Cflags: -I${includedir}
 Libs: -L${libdir} -lXft

This won't work because of the aforementioned bug #340904:

$ grep -h include include/X11/Xft/*.h
#include <X11/Xfuncproto.h>
#include <stdarg.h>
#include <ft2build.h>
#include FT_FREETYPE_H
#include <fontconfig/fontconfig.h>
#include <X11/extensions/Xrender.h>
#include <X11/Xfuncproto.h>
/* #include <X11/Xosdefs.h>*/
#include <X11/Xft/XftCompat.h>
$

Both the xrender and freetype2 headers are included in Xft/Xft.h, and types
from both of these headers are exported in the Xft API -- so any software
using Xft.h needs to be able to find these other headers as well.
Currently, pkg-config does *not* pass cflags from packages listed in
Requires.private when called as pkg-config --cflags <foo>.

So far, the maintainer has resisted changing pkg-config to export cflags
(which should really be called cppflags...) from Requires.private due to a
fallacious argument regarding the nature of "private" dependencies.  There
are three real use cases for libraries which depend on other libraries:

- the library intentionally exports the API and ABI of its dependencies when
  linked to, and therefore both the ldflags and the cflags of its
  dependencies should be exported by pkg-config in all cases[1]
- the library intentionally includes headers from dependencies in its own
  headers in order to inherit type definitions, but these definitions are
  not intended for direct consumption by users of this library alone;
  therefore pkg-config must export the cflags from dependencies in all
  cases, but the ldflags only when linking statically
- the library's API includes no headers from its dependencies; pkg-config
  needs to export the ldflags of private dependencies when statically
  linking but not when dynamically linking, and should *never* need to
  export the cflags of these headers.

Please note that today, the handling of Requires.private in pkg-config maps
to *none* of these cases -- I can't think of a single situation in which
cflags of dependencies are needed when statically linking, and not needed
when dynamically linking!  Instead, the Requires.private handling is
adequate for the third case; handling of Requires is correct for the first
case; and there is no combination of options that is appropriate for the
second case.

This is unfortunate, because there are a great many packages that are
inheriting dependencies this way on libraries they don't use.  While it is
true that an ABI change in the dependent library will *sometimes* mean an
ABI change in the depending library, this is not always the case.  As a
result, this behavior of pkg-config causes unnecessary churn for packages
depending on libraries in this scenario.  In the case of libxft2, that's
over 400 packages in Debian that are potentially affected.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
vorlon@debian.org                                   http://www.debian.org/

[1] Personally, I think this is completely specious, but there are GNOME
people who insist that this describes the relationship between gtk+2.0
glib.

Revision history for this message

In Debian Bug tracker #340904, James Henstridge (jamesh) wrote on 2006-02-08: Re: [Steve Langasek] Bug#340904: Bug#349318: fixed in xft 2.1.8.2-1

#9

Download full text (5.4 KiB)

Tollef Fog Heen wrote:

>Hi, any chance you could comment on this? I'm somewhat inclined to
>agree with Steve, but you know this stuff better than I do, so I'd
>value a second opinion.
>
>Also, if there's anything unclear or anything, please do mail Steve
>(and the bug) with me in Cc.
>
>
[snip]

>
>
>Both the xrender and freetype2 headers are included in Xft/Xft.h, and types
>from both of these headers are exported in the Xft API -- so any software
>using Xft.h needs to be able to find these other headers as well.
>Currently, pkg-config does *not* pass cflags from packages listed in
>Requires.private when called as pkg-config --cflags <foo>.
>
>

The fact that the headers are included makes them part of the _public_
ABI of Xft. There are Xft functions that accept or return types defined
in the Xrender and freetype2 headers.

If Xft is updated to a new version of either of those libraries such
that those types are defined differently (altered struct layout,
different type sizes, etc), then the app also needs to be updated to the
new version.

>So far, the maintainer has resisted changing pkg-config to export cflags
>(which should really be called cppflags...) from Requires.private due to a
>fallacious argument regarding the nature of "private" dependencies. There
>are three real use cases for libraries which depend on other libraries:
>
>- the library intentionally exports the API and ABI of its dependencies when
> linked to, and therefore both the ldflags and the cflags of its
> dependencies should be exported by pkg-config in all cases[1]
>
>
Yep, this is the use case addressed by "Requires".

>- the library intentionally includes headers from dependencies in its own
> headers in order to inherit type definitions, but these definitions are
> not intended for direct consumption by users of this library alone;
> therefore pkg-config must export the cflags from dependencies in all
> cases, but the ldflags only when linking statically
>
>
Changes to type definitions _do_ change the ABI. If library A uses
library B's types in its ABI, then it's ABI will break if library B
changes those types. An app using library A should definitely record
the version of library B being used.

>- the library's API includes no headers from its dependencies; pkg-config
> needs to export the ldflags of private dependencies when statically
> linking but not when dynamically linking, and should *never* need to
> export the cflags of these headers.
>
>
This is the use case for "Requires.private". If the dependency doesn't
form part of the library's API, then it obviously doesn't require direct
linkage and is private.

>Please note that today, the handling of Requires.private in pkg-config maps
>to *none* of these cases -- I can't think of a single situation in which
>cflags of dependencies are needed when statically linking, and not needed
>when dynamically linking!
>
The cflags of dependencies listed in "Requires.private" are not included
for either dynamic or static modes of pkg-config, so maps to case 3
quite well.

Consider the following two .pc files. First "foo.pc":
  Name: foo
  Description: foo
  Version: 1.0
  Cflags: foo-cflags
  ...

Tollef Fog Heen wrote:

>Hi, any chance you could comment on this?  I'm somewhat inclined to
>agree with Steve, but you know this stuff better than I do, so I'd
>value a second opinion.
>
>Also, if there's anything unclear or anything, please do mail Steve
>(and the bug) with me in Cc.
>  
>
[snip]

>  
>
>Both the xrender and freetype2 headers are included in Xft/Xft.h, and types
>from both of these headers are exported in the Xft API -- so any software
>using Xft.h needs to be able to find these other headers as well.
>Currently, pkg-config does *not* pass cflags from packages listed in
>Requires.private when called as pkg-config --cflags <foo>.
>  
>

The fact that the headers are included makes them part of the _public_
ABI of Xft.  There are Xft functions that accept or return types defined
in the Xrender and freetype2 headers.

If Xft is updated to a new version of either of those libraries such
that those types are defined differently (altered struct layout,
different type sizes, etc), then the app also needs to be updated to the
new version.

>So far, the maintainer has resisted changing pkg-config to export cflags
>(which should really be called cppflags...) from Requires.private due to a
>fallacious argument regarding the nature of "private" dependencies.  There
>are three real use cases for libraries which depend on other libraries:
>
>- the library intentionally exports the API and ABI of its dependencies when
>  linked to, and therefore both the ldflags and the cflags of its
>  dependencies should be exported by pkg-config in all cases[1]
>  
>
Yep, this is the use case addressed by "Requires".

>- the library intentionally includes headers from dependencies in its own
>  headers in order to inherit type definitions, but these definitions are
>  not intended for direct consumption by users of this library alone;
>  therefore pkg-config must export the cflags from dependencies in all
>  cases, but the ldflags only when linking statically
>  
>
Changes to type definitions _do_ change the ABI.  If library A uses
library B's types in its ABI, then it's ABI will break if library B
changes those types.  An app using library A should definitely record
the version of library B being used.

>- the library's API includes no headers from its dependencies; pkg-config
>  needs to export the ldflags of private dependencies when statically
>  linking but not when dynamically linking, and should *never* need to
>  export the cflags of these headers.
>  
>
This is the use case for "Requires.private".  If the dependency doesn't
form part of the library's API, then it obviously doesn't require direct
linkage and is private.

>Please note that today, the handling of Requires.private in pkg-config maps
>to *none* of these cases -- I can't think of a single situation in which
>cflags of dependencies are needed when statically linking, and not needed
>when dynamically linking!
>
The cflags of dependencies listed in "Requires.private" are not included
for either dynamic or static modes of pkg-config, so maps to case 3
quite well.

Consider the following two .pc files.  First "foo.pc":
  Name: foo
  Description: foo
  Version: 1.0
  Cflags: foo-cflags
  Libs: foo-libs

And "bar.pc":
  Name: bar
  Description: bar
  Version: 1.0
  Cflags: bar-cflags
  Libs: bar-libs
  Requires.private: foo

For the dynamic linking case:
  $ pkg-config --cflags bar
  bar-cflags
  $ pkg-config --libs bar
  bar-libs

For the static linking case:
  $ pkg-config --static --cflags bar
  bar-cflags
  $ pkg-config --static --libs bar
  bar-libs foo-libs

>  Instead, the Requires.private handling is
>adequate for the third case; handling of Requires is correct for the first
>case; and there is no combination of options that is appropriate for the
>second case.
>  
>
I'd argue that the second case is a strawman.  If a particular library
exposes another in its API/ABI, then it is part of case 1.

>This is unfortunate, because there are a great many packages that are
>inheriting dependencies this way on libraries they don't use.  While it is
>true that an ABI change in the dependent library will *sometimes* mean an
>ABI change in the depending library, this is not always the case.  As a
>result, this behavior of pkg-config causes unnecessary churn for packages
>depending on libraries in this scenario.  In the case of libxft2, that's
>over 400 packages in Debian that are potentially affected.
>  
>
Sure, unnecessary churn is bad and should be avoided.  But you do want
to make sure that the churn happens when required.  Relying on indirect
linkage in a lot of cases just results in more fragile applications.

There are cases where it might make sense to move a library to
"Requires.private" even when it is used in some headers, but I don't
think this is one of them.  Some examples where it would be appropriate
include:

* Cairo's dependency on glitz, xlib and freetype (this one has been
      done upstream).  Apps only depend on glitz if they include
      <cairo-glitz.h> and use those APIs.  Apps that don't use those
      APIs shouldn't be linked to glitz so that they'll work on systems
      where Cairo hasn't been built with glitz support.  The Cairo
      developers advise app developers to direct link to both cairo and
      glitz if they are using the glitz features.
    * GTK does not expose the X API through its headers except for
      <gdk/gdkx.h>.  You might argue that it should list the xlib
      dependency as private.

James.

Revision history for this message

In Debian Bug tracker #340904, Steve Langasek (vorlon) wrote on 2006-02-09:

#10

Download full text (6.1 KiB)

Hi James,

Nice to meet you!

On Wed, Feb 08, 2006 at 07:28:39PM +0800, James Henstridge wrote:
> Tollef Fog Heen wrote:

> >Hi, any chance you could comment on this? I'm somewhat inclined to
> >agree with Steve, but you know this stuff better than I do, so I'd
> >value a second opinion.

> >Also, if there's anything unclear or anything, please do mail Steve
> >(and the bug) with me in Cc.

> [snip]

> >Both the xrender and freetype2 headers are included in Xft/Xft.h, and types
> >from both of these headers are exported in the Xft API -- so any software
> >using Xft.h needs to be able to find these other headers as well.
> >Currently, pkg-config does *not* pass cflags from packages listed in
> >Requires.private when called as pkg-config --cflags <foo>.

> The fact that the headers are included makes them part of the _public_
> ABI of Xft. There are Xft functions that accept or return types defined
> in the Xrender and freetype2 headers.

> If Xft is updated to a new version of either of those libraries such
> that those types are defined differently (altered struct layout,
> different type sizes, etc), then the app also needs to be updated to the
> new version.

Ok, here's the problem with this argument.

Yes, if one of the freetype types that's used in Xft/Xft.h changes, that is
an ABI change... *in libXft* -- that's why we care about it! Because it's
an ABI change in libXft, the soname of *libXft* should change. With the
change of libXft's soname, there's no reason an application that is a
consumer of libXft, but *not* a consumer of freetype, should care about
libfreetype *at all*.[1]

On the other hand, there are plenty of cases in which an ABI change in
libfreetype will *not* cause an ABI change in libXft. Addition or removal
of functions, addition of typedefs, or removal or changing of any typedefs
not used in libXft's ABI are all changes that should require an soname
change in libfreetype but not an soname change in libXft. The present ABI
transition in libfreetype is such a case. But because these applications
are being encouraged to link directly to libfreetype, *even though they
don't use it*, they have to care about ABI changes that should not affect
them.

The net result is that pkg-config's handling of Requires/Requires.private is
directly causing churn in response to ABI changes in any indirect library
dependencies, where this should be completely unnecessary on GNU/Linux
platforms. It increases the chances of segfaults or other failures from
loading two different versions of a library into memory, and correspondingly
increases the frequency with which binaries need to be rebuilt in response
to ABI changes that don't actually concern them. And it does this entirely
to support a use case which, as explained above, should not actually exist.

> >- the library intentionally includes headers from dependencies in its own
> > headers in order to inherit type definitions, but these definitions are
> > not intended for direct consumption by users of this library alone;
> > therefore pkg-config must export the cflags from dependencies in all
> > cases, but the ldflags only when linking statically

> Changes to type definition...

Hi James,

Nice to meet you!

On Wed, Feb 08, 2006 at 07:28:39PM +0800, James Henstridge wrote:
> Tollef Fog Heen wrote:

> >Hi, any chance you could comment on this?  I'm somewhat inclined to
> >agree with Steve, but you know this stuff better than I do, so I'd
> >value a second opinion.

> >Also, if there's anything unclear or anything, please do mail Steve
> >(and the bug) with me in Cc.

> [snip]

> >Both the xrender and freetype2 headers are included in Xft/Xft.h, and types
> >from both of these headers are exported in the Xft API -- so any software
> >using Xft.h needs to be able to find these other headers as well.
> >Currently, pkg-config does *not* pass cflags from packages listed in
> >Requires.private when called as pkg-config --cflags <foo>.

> The fact that the headers are included makes them part of the _public_
> ABI of Xft.  There are Xft functions that accept or return types defined
> in the Xrender and freetype2 headers.

> If Xft is updated to a new version of either of those libraries such
> that those types are defined differently (altered struct layout,
> different type sizes, etc), then the app also needs to be updated to the
> new version.

Ok, here's the problem with this argument.

Yes, if one of the freetype types that's used in Xft/Xft.h changes, that is
an ABI change... *in libXft* -- that's why we care about it!  Because it's
an ABI change in libXft, the soname of *libXft* should change.  With the
change of libXft's soname, there's no reason an application that is a
consumer of libXft, but *not* a consumer of freetype, should care about
libfreetype *at all*.[1]

On the other hand, there are plenty of cases in which an ABI change in
libfreetype will *not* cause an ABI change in libXft.  Addition or removal
of functions, addition of typedefs, or removal or changing of any typedefs
not used in libXft's ABI are all changes that should require an soname
change in libfreetype but not an soname change in libXft.  The present ABI
transition in libfreetype is such a case.  But because these applications
are being encouraged to link directly to libfreetype, *even though they
don't use it*, they have to care about ABI changes that should not affect
them.

The net result is that pkg-config's handling of Requires/Requires.private is
directly causing churn in response to ABI changes in any indirect library
dependencies, where this should be completely unnecessary on GNU/Linux
platforms.  It increases the chances of segfaults or other failures from
loading two different versions of a library into memory, and correspondingly
increases the frequency with which binaries need to be rebuilt in response
to ABI changes that don't actually concern them.  And it does this entirely
to support a use case which, as explained above, should not actually exist.

> >- the library intentionally includes headers from dependencies in its own
> >  headers in order to inherit type definitions, but these definitions are
> >  not intended for direct consumption by users of this library alone;
> >  therefore pkg-config must export the cflags from dependencies in all
> >  cases, but the ldflags only when linking statically

> Changes to type definitions _do_ change the ABI.  If library A uses
> library B's types in its ABI, then it's ABI will break if library B
> changes those types.  An app using library A should definitely record
> the version of library B being used.

However:

- If library A's soname is correctly changed in sync with library B's,
  linking application C to library B is redundant.
- If library A's soname is not changed when library B's soname changes,
  pkg-config's behavior does not prevent applications from being broken by
  the ABI change in libA.  At most, it makes it easier to detect such
  breakage due to double-linkage of libB.

> >Please note that today, the handling of Requires.private in pkg-config maps
> >to *none* of these cases -- I can't think of a single situation in which
> >cflags of dependencies are needed when statically linking, and not needed
> >when dynamically linking!

> The cflags of dependencies listed in "Requires.private" are not included
> for either dynamic or static modes of pkg-config, so maps to case 3
> quite well.

Ah, guess I should have looked a bit closer at the behavior in that case.

> >This is unfortunate, because there are a great many packages that are
> >inheriting dependencies this way on libraries they don't use.  While it is
> >true that an ABI change in the dependent library will *sometimes* mean an
> >ABI change in the depending library, this is not always the case.  As a
> >result, this behavior of pkg-config causes unnecessary churn for packages
> >depending on libraries in this scenario.  In the case of libxft2, that's
> >over 400 packages in Debian that are potentially affected.

> Sure, unnecessary churn is bad and should be avoided.  But you do want
> to make sure that the churn happens when required.

Er, the whole reason I'm objecting to the current behavior is that it *is*
unnecessary churn.  If the libfreetype ABI change affected the ABI of
libXft, there would be no sense in trying to avoid the double-linking, and I
wouldn't be bothering! :)

> Relying on indirect linkage in a lot of cases just results in more fragile
> applications.

No, recursive linking of indirect dependencies results in more *brittle*
applications, which break any time anything anywhere in the dependency tree
changes.  I'm not sure why you think that "relying" on libraries to take
care of their own messes without bothering the application is fragile.

Thanks,
-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
vorlon@debian.org                                   http://www.debian.org/

[1] This sets aside the issue that you can't reasonably ensure that the
freetype.h included from Xft/Xft.h matches the ABI used when building
libxft.so except through a package manager; that's simply an argument why
you should properly encapsulate types from other libraries instead of
exposing them in your own API, which doesn't help us in the case when that's
not what people do.  It's also not anything that pkg-config's current
behavior actually helps...

Revision history for this message

In Debian Bug tracker #340904, James Henstridge (jamesh) wrote on 2006-02-09:

#11

Download full text (6.2 KiB)

Steve Langasek wrote:

>>If Xft is updated to a new version of either of those libraries such
>>that those types are defined differently (altered struct layout,
>>different type sizes, etc), then the app also needs to be updated to the
>>new version.
>>
>>
>Ok, here's the problem with this argument.
>
>Yes, if one of the freetype types that's used in Xft/Xft.h changes, that is
>an ABI change... *in libXft* -- that's why we care about it! Because it's
>an ABI change in libXft, the soname of *libXft* should change. With the
>change of libXft's soname, there's no reason an application that is a
>consumer of libXft, but *not* a consumer of freetype, should care about
>libfreetype *at all*.[1]
>
>
The simple matter of fact is that when you ask for the Xft API in your
app (#include <Xft/Xft.h>), you also get the freetype API at the same
time. While some apps don't use it, we can't say that apps don't use
those APIs in the general case. Until we have a widely used way to
express dependencies at that granularity, you need to look at the
library level.

>On the other hand, there are plenty of cases in which an ABI change in
>libfreetype will *not* cause an ABI change in libXft. Addition or removal
>of functions, addition of typedefs, or removal or changing of any typedefs
>not used in libXft's ABI are all changes that should require an soname
>change in libfreetype but not an soname change in libXft. The present ABI
>transition in libfreetype is such a case. But because these applications
>are being encouraged to link directly to libfreetype, *even though they
>don't use it*, they have to care about ABI changes that should not affect
>them.
>
>
They are using it in conjunction with the Xft API. There is no clear
separation between the Xft APIs that use it and those that don't. Of
course, an app that only depends on Xft indirectly and doesn't use Xft's
APIs doesn't need direct linkage to freetype.

>The net result is that pkg-config's handling of Requires/Requires.private is
>directly causing churn in response to ABI changes in any indirect library
>dependencies, where this should be completely unnecessary on GNU/Linux
>platforms. It increases the chances of segfaults or other failures from
>loading two different versions of a library into memory, and correspondingly
>increases the frequency with which binaries need to be rebuilt in response
>to ABI changes that don't actually concern them. And it does this entirely
>to support a use case which, as explained above, should not actually exist.
>
>
The "app links to multiple versions of library A" problem is quite easy
to diagnose, and also makes it clear that an app needs to be rebuilt if
the old library version is removed (i.e. the app doesn't start). The
"app depends on the structure layout of the old version of library A but
doesn't directly link to library A" is more difficult to catch.

It sounds like your argument is that "Xft shouldn't expose the freetype
API". Given that this isn't the case, the direct linkage makes sense.
The freetype change in this case might not break many Xft using apps,
but that won't necessarily be the case next time this type of situation
occurs.

>...

Steve Langasek wrote:

>>If Xft is updated to a new version of either of those libraries such
>>that those types are defined differently (altered struct layout,
>>different type sizes, etc), then the app also needs to be updated to the
>>new version.
>>    
>>
>Ok, here's the problem with this argument.
>
>Yes, if one of the freetype types that's used in Xft/Xft.h changes, that is
>an ABI change... *in libXft* -- that's why we care about it!  Because it's
>an ABI change in libXft, the soname of *libXft* should change.  With the
>change of libXft's soname, there's no reason an application that is a
>consumer of libXft, but *not* a consumer of freetype, should care about
>libfreetype *at all*.[1]
>  
>
The simple matter of fact is that when you ask for the Xft API in your
app (#include <Xft/Xft.h>), you also get the freetype API at the same
time.  While some apps don't use it, we can't say that apps don't use
those APIs in the general case.  Until we have a widely used way to
express dependencies at that granularity, you need to look at the
library level.

>On the other hand, there are plenty of cases in which an ABI change in
>libfreetype will *not* cause an ABI change in libXft.  Addition or removal
>of functions, addition of typedefs, or removal or changing of any typedefs
>not used in libXft's ABI are all changes that should require an soname
>change in libfreetype but not an soname change in libXft.  The present ABI
>transition in libfreetype is such a case.  But because these applications
>are being encouraged to link directly to libfreetype, *even though they
>don't use it*, they have to care about ABI changes that should not affect
>them.
>  
>
They are using it in conjunction with the Xft API.  There is no clear
separation between the Xft APIs that use it and those that don't.  Of
course, an app that only depends on Xft indirectly and doesn't use Xft's
APIs doesn't need direct linkage to freetype.

>The net result is that pkg-config's handling of Requires/Requires.private is
>directly causing churn in response to ABI changes in any indirect library
>dependencies, where this should be completely unnecessary on GNU/Linux
>platforms.  It increases the chances of segfaults or other failures from
>loading two different versions of a library into memory, and correspondingly
>increases the frequency with which binaries need to be rebuilt in response
>to ABI changes that don't actually concern them.  And it does this entirely
>to support a use case which, as explained above, should not actually exist.
>  
>
The "app links to multiple versions of library A" problem is quite easy
to diagnose, and also makes it clear that an app needs to be rebuilt if
the old library version is removed (i.e. the app doesn't start).  The
"app depends on the structure layout of the old version of library A but
doesn't directly link to library A" is more difficult to catch.

It sounds like your argument is that "Xft shouldn't expose the freetype
API".  Given that this isn't the case, the direct linkage makes sense. 
The freetype change in this case might not break many Xft using apps,
but that won't necessarily be the case next time this type of situation
occurs.

>>Changes to type definitions _do_ change the ABI.  If library A uses
>>library B's types in its ABI, then it's ABI will break if library B
>>changes those types.  An app using library A should definitely record
>>the version of library B being used.
>>    
>>
>However:
>
>- If library A's soname is correctly changed in sync with library B's,
>  linking application C to library B is redundant.
>- If library A's soname is not changed when library B's soname changes,
>  pkg-config's behavior does not prevent applications from being broken by
>  the ABI change in libA.  At most, it makes it easier to detect such
>  breakage due to double-linkage of libB.
>  
>
I'd argue that this is a useful feature to have, rather than having apps
break with no indication of the cause.

>>The cflags of dependencies listed in "Requires.private" are not included
>>for either dynamic or static modes of pkg-config, so maps to case 3
>>quite well.
>>    
>>
>Ah, guess I should have looked a bit closer at the behavior in that case.
>  
>
Case 3 was exactly the sort of use case I was trying to model with how
Requires.private functions.

>>>This is unfortunate, because there are a great many packages that are
>>>inheriting dependencies this way on libraries they don't use.  While it is
>>>true that an ABI change in the dependent library will *sometimes* mean an
>>>ABI change in the depending library, this is not always the case.  As a
>>>result, this behavior of pkg-config causes unnecessary churn for packages
>>>depending on libraries in this scenario.  In the case of libxft2, that's
>>>over 400 packages in Debian that are potentially affected.
>>>      
>>>
>>Sure, unnecessary churn is bad and should be avoided.  But you do want
>>to make sure that the churn happens when required.
>>    
>>
>Er, the whole reason I'm objecting to the current behavior is that it *is*
>unnecessary churn.  If the libfreetype ABI change affected the ABI of
>libXft, there would be no sense in trying to avoid the double-linking, and I
>wouldn't be bothering! :)
>  
>
I understand that ABI changes are painful, but this doesn't seem like
the right way to try and combat it.  Other options include:

* don't break the ABI.
    * don't expose unstable dependencies in your own API.
    * increase soname version numbers in lockstep with dependencies.

These all require upstream cooperation though.

>>Relying on indirect linkage in a lot of cases just results in more fragile
>>applications.
>>    
>>
>No, recursive linking of indirect dependencies results in more *brittle*
>applications, which break any time anything anywhere in the dependency tree
>changes.  I'm not sure why you think that "relying" on libraries to take
>care of their own messes without bothering the application is fragile.
>  
>
If the indirect dependency is solely the library's mess, then relying on
indirect linking is the right solution (and is what should happen with
Requires.private).

When details of the indirect dependency leak through the library's API
then the mess belongs to both the app and library, and should be treated
as such.

James.

Revision history for this message

In Debian Bug tracker #340904, Steve Langasek (vorlon) wrote on 2006-02-12:

#12

Download full text (8.1 KiB)

On Thu, Feb 09, 2006 at 06:59:02PM +0800, James Henstridge wrote:
> Steve Langasek wrote:

> >>If Xft is updated to a new version of either of those libraries such
> >>that those types are defined differently (altered struct layout,
> >>different type sizes, etc), then the app also needs to be updated to the
> >>new version.

> >Ok, here's the problem with this argument.

> >Yes, if one of the freetype types that's used in Xft/Xft.h changes, that is
> >an ABI change... *in libXft* -- that's why we care about it! Because it's
> >an ABI change in libXft, the soname of *libXft* should change. With the
> >change of libXft's soname, there's no reason an application that is a
> >consumer of libXft, but *not* a consumer of freetype, should care about
> >libfreetype *at all*.[1]

> The simple matter of fact is that when you ask for the Xft API in your
> app (#include <Xft/Xft.h>), you also get the freetype API at the same
> time. While some apps don't use it, we can't say that apps don't use
> those APIs in the general case.

You're right, we can't. But as library authors/maintainers, we can say
whether we intend to *support* such abuses of the API. You can get a valid
prototype of fprintf() on Linux by doing #include <sql.h> (from UnixODBC)
instead of #include <stdio.h>, too, and we don't know that there isn't
anybody relying on that -- but if they do, and they complain about it
breaking, they are cordially invited to blow it out their ass. ;)

The current policy of pkg-config prevents library authors from making their
own choices about whether to support such API abuses on the part of
applications. I don't think pkg-config should be imposing policies which
prevent library authors from deciding what uses of their headers they intend
to support. The current behavior of pkg-config *ensures* that the API
abuses you describe will work, because use of Requires means they'll get the
ldflags as well as the cflags; that makes pkg-config as it stands pretty
suboptimal for anyone who *doesn't* want to get supporting their lib
dependencies as part of the public API.

> >On the other hand, there are plenty of cases in which an ABI change in
> >libfreetype will *not* cause an ABI change in libXft. Addition or removal
> >of functions, addition of typedefs, or removal or changing of any typedefs
> >not used in libXft's ABI are all changes that should require an soname
> >change in libfreetype but not an soname change in libXft. The present ABI
> >transition in libfreetype is such a case. But because these applications
> >are being encouraged to link directly to libfreetype, *even though they
> >don't use it*, they have to care about ABI changes that should not affect
> >them.

> They are using it in conjunction with the Xft API. There is no clear
> separation between the Xft APIs that use it and those that don't. Of
> course, an app that only depends on Xft indirectly and doesn't use Xft's
> APIs doesn't need direct linkage to freetype.

They are using data types defined by freetype in conjunction with the Xft
API. If they're using function calls from freetype without including
freetype headers, they didn't get that idea from Xft's documentation or by...

On Thu, Feb 09, 2006 at 06:59:02PM +0800, James Henstridge wrote:
> Steve Langasek wrote:

> >>If Xft is updated to a new version of either of those libraries such
> >>that those types are defined differently (altered struct layout,
> >>different type sizes, etc), then the app also needs to be updated to the
> >>new version.

> >Ok, here's the problem with this argument.

> >Yes, if one of the freetype types that's used in Xft/Xft.h changes, that is
> >an ABI change... *in libXft* -- that's why we care about it!  Because it's
> >an ABI change in libXft, the soname of *libXft* should change.  With the
> >change of libXft's soname, there's no reason an application that is a
> >consumer of libXft, but *not* a consumer of freetype, should care about
> >libfreetype *at all*.[1]

> The simple matter of fact is that when you ask for the Xft API in your
> app (#include <Xft/Xft.h>), you also get the freetype API at the same
> time.  While some apps don't use it, we can't say that apps don't use
> those APIs in the general case.

You're right, we can't.  But as library authors/maintainers, we can say
whether we intend to *support* such abuses of the API.  You can get a valid
prototype of fprintf() on Linux by doing #include <sql.h> (from UnixODBC)
instead of #include <stdio.h>, too, and we don't know that there isn't
anybody relying on that -- but if they do, and they complain about it
breaking, they are cordially invited to blow it out their ass. ;)

The current policy of pkg-config prevents library authors from making their
own choices about whether to support such API abuses on the part of
applications.  I don't think pkg-config should be imposing policies which
prevent library authors from deciding what uses of their headers they intend
to support.  The current behavior of pkg-config *ensures* that the API
abuses you describe will work, because use of Requires means they'll get the
ldflags as well as the cflags; that makes pkg-config as it stands pretty
suboptimal for anyone who *doesn't* want to get supporting their lib
dependencies as part of the public API.

> >On the other hand, there are plenty of cases in which an ABI change in
> >libfreetype will *not* cause an ABI change in libXft.  Addition or removal
> >of functions, addition of typedefs, or removal or changing of any typedefs
> >not used in libXft's ABI are all changes that should require an soname
> >change in libfreetype but not an soname change in libXft.  The present ABI
> >transition in libfreetype is such a case.  But because these applications
> >are being encouraged to link directly to libfreetype, *even though they
> >don't use it*, they have to care about ABI changes that should not affect
> >them.

> They are using it in conjunction with the Xft API.  There is no clear
> separation between the Xft APIs that use it and those that don't.  Of
> course, an app that only depends on Xft indirectly and doesn't use Xft's
> APIs doesn't need direct linkage to freetype.

They are using data types defined by freetype in conjunction with the Xft
API.  If they're using function calls from freetype without including
freetype headers, they didn't get that idea from Xft's documentation or by
reading the Xft headers themselves.  Xft's maintainers should not be
constrained by a pkg-config design decision to support people who think
#include <Xft/Xft.h> is an acceptable substitute for #include
<freetype/freetype.h>.

> >The net result is that pkg-config's handling of Requires/Requires.private is
> >directly causing churn in response to ABI changes in any indirect library
> >dependencies, where this should be completely unnecessary on GNU/Linux
> >platforms.  It increases the chances of segfaults or other failures from
> >loading two different versions of a library into memory, and correspondingly
> >increases the frequency with which binaries need to be rebuilt in response
> >to ABI changes that don't actually concern them.  And it does this entirely
> >to support a use case which, as explained above, should not actually exist.

> The "app links to multiple versions of library A" problem is quite easy
> to diagnose, and also makes it clear that an app needs to be rebuilt if
> the old library version is removed (i.e. the app doesn't start).  The
> "app depends on the structure layout of the old version of library A but
> doesn't directly link to library A" is more difficult to catch.

The app doesn't depend on the structure layout of the old version of library
A.  It depends on the structure layout of *library B*, which happens to have
*inherited* that structure from the *headers* of library A.  It's entirely
feasible for a library to do this without actually using library A at
runtime; so then you get library B not linking to library A at all, but
because it uses pkg-config to resolve the header reference, app C *does* get
linked against it for no reason...

> It sounds like your argument is that "Xft shouldn't expose the freetype
> API".  Given that this isn't the case, the direct linkage makes sense. 
> The freetype change in this case might not break many Xft using apps,
> but that won't necessarily be the case next time this type of situation
> occurs.

No, that's not my argument.  It's *true* that libraries should generally
strive not to expose the APIs of other libraries in their own; they also
should use linker scripts to avoid exporting internal symbols, use versioned
symbols to avoid problems with multiple versions of the library,
forward-proof their APIs so structs can be extended without breaking the
ABI, and carefully check their ABIs with every release to make sure they
aren't silently incompatible.  But I still live on Earth, where lots of
libraries are sub-optimally maintained. :)  My argument is, *given* that
such suboptimal libraries exist, pkg-config should accomodate this reality
instead of amplifying the problems.

> >- If library A's soname is correctly changed in sync with library B's,
> >  linking application C to library B is redundant.
> >- If library A's soname is not changed when library B's soname changes,
> >  pkg-config's behavior does not prevent applications from being broken by
> >  the ABI change in libA.  At most, it makes it easier to detect such
> >  breakage due to double-linkage of libB.

> I'd argue that this is a useful feature to have, rather than having apps
> break with no indication of the cause.

Possibly so.  I don't believe it outweighs all the other considerations,
such as the large number of cases in which applications wouldn't break *at
all* except for pkg-config pulling in unnecessary libraries.

It's also not as though such breakage isn't debuggable; you just generally
have to use a debugger, instead of merely looking at the output of ldd...

> >>>This is unfortunate, because there are a great many packages that are
> >>>inheriting dependencies this way on libraries they don't use.  While it is
> >>>true that an ABI change in the dependent library will *sometimes* mean an
> >>>ABI change in the depending library, this is not always the case.  As a
> >>>result, this behavior of pkg-config causes unnecessary churn for packages
> >>>depending on libraries in this scenario.  In the case of libxft2, that's
> >>>over 400 packages in Debian that are potentially affected.

> >>Sure, unnecessary churn is bad and should be avoided.  But you do want
> >>to make sure that the churn happens when required.

> >Er, the whole reason I'm objecting to the current behavior is that it *is*
> >unnecessary churn.  If the libfreetype ABI change affected the ABI of
> >libXft, there would be no sense in trying to avoid the double-linking, and I
> >wouldn't be bothering! :)

> I understand that ABI changes are painful, but this doesn't seem like
> the right way to try and combat it.  Other options include:

>     * don't break the ABI.
>     * don't expose unstable dependencies in your own API.
>     * increase soname version numbers in lockstep with dependencies.

> These all require upstream cooperation though.

Sure.  That makes them substantially less scalable. :)

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
vorlon@debian.org                                   http://www.debian.org/

Revision history for this message

In Debian Bug tracker #340904, Tollef Fog Heen (tfheen) wrote on 2006-08-16: Bug#340904: fixed in pkg-config 0.21-1

#13

Source: pkg-config
Source-Version: 0.21-1

We believe that the bug you reported is fixed in the latest version of
pkg-config, which is due to be installed in the Debian FTP archive:

pkg-config_0.21-1.diff.gz
  to pool/main/p/pkg-config/pkg-config_0.21-1.diff.gz
pkg-config_0.21-1.dsc
  to pool/main/p/pkg-config/pkg-config_0.21-1.dsc
pkg-config_0.21-1_i386.deb
  to pool/main/p/pkg-config/pkg-config_0.21-1_i386.deb
pkg-config_0.21.orig.tar.gz
  to pool/main/p/pkg-config/pkg-config_0.21.orig.tar.gz

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to <email address hidden>,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Tollef Fog Heen <email address hidden> (supplier of updated pkg-config package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing <email address hidden>)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Wed, 16 Aug 2006 21:11:21 +0200
Source: pkg-config
Binary: pkg-config
Architecture: source i386
Version: 0.21-1
Distribution: unstable
Urgency: low
Maintainer: Tollef Fog Heen <email address hidden>
Changed-By: Tollef Fog Heen <email address hidden>
Description:
pkg-config - manage compile and link flags for libraries
Closes: 254289 334896 340904 350176 378570
Changes:
pkg-config (0.21-1) unstable; urgency=low
.
   * New upstream release
     - Adds internal pkg-config package. (closes: #254289)
     - Supports escaping $ rather than %. (closes: #378570)
     - Always adds all cflags. (closes: #340904)
   * Includes full NEWS file. (closes: #334896)
   * Remove libglib2.0-dev build-deps. (closes: #350176)
Files:
e922bea83dadee66fab37c6a83fd0fd4 572 devel optional pkg-config_0.21-1.dsc
476f45fab1504aac6697aa7785f0ab91 998420 devel optional pkg-config_0.21.orig.tar.gz
23c25fd98789a5e18d342a39339346b8 4404 devel optional pkg-config_0.21-1.diff.gz
59f0e1fdc7ba08a4faa1fc4071e05960 67532 devel optional pkg-config_0.21-1_i386.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)

iD8DBQFE43EjQSseMYF6mWoRAmDSAKCluBOZcezXRPykQT3O5PGuhlpCIQCfRJOh
y5+NQC2kJ4aADtbzFvmU9lg=
=OwY/
-----END PGP SIGNATURE-----

Source: pkg-config
Source-Version: 0.21-1

We believe that the bug you reported is fixed in the latest version of
pkg-config, which is due to be installed in the Debian FTP archive:

pkg-config_0.21-1.diff.gz
  to pool/main/p/pkg-config/pkg-config_0.21-1.diff.gz
pkg-config_0.21-1.dsc
  to pool/main/p/pkg-config/pkg-config_0.21-1.dsc
pkg-config_0.21-1_i386.deb
  to pool/main/p/pkg-config/pkg-config_0.21-1_i386.deb
pkg-config_0.21.orig.tar.gz
  to pool/main/p/pkg-config/pkg-config_0.21.orig.tar.gz

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 340904@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Tollef Fog Heen <tfheen@debian.org> (supplier of updated pkg-config package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Wed, 16 Aug 2006 21:11:21 +0200
Source: pkg-config
Binary: pkg-config
Architecture: source i386
Version: 0.21-1
Distribution: unstable
Urgency: low
Maintainer: Tollef Fog Heen <tfheen@debian.org>
Changed-By: Tollef Fog Heen <tfheen@debian.org>
Description: 
 pkg-config - manage compile and link flags for libraries
Closes: 254289 334896 340904 350176 378570
Changes: 
 pkg-config (0.21-1) unstable; urgency=low
 .
   * New upstream release
     - Adds internal pkg-config package.  (closes: #254289)
     - Supports escaping $ rather than %.  (closes: #378570)
     - Always adds all cflags.  (closes: #340904)
   * Includes full NEWS file.  (closes: #334896)
   * Remove libglib2.0-dev build-deps.  (closes: #350176)
Files: 
 e922bea83dadee66fab37c6a83fd0fd4 572 devel optional pkg-config_0.21-1.dsc
 476f45fab1504aac6697aa7785f0ab91 998420 devel optional pkg-config_0.21.orig.tar.gz
 23c25fd98789a5e18d342a39339346b8 4404 devel optional pkg-config_0.21-1.diff.gz
 59f0e1fdc7ba08a4faa1fc4071e05960 67532 devel optional pkg-config_0.21-1_i386.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)

iD8DBQFE43EjQSseMYF6mWoRAmDSAKCluBOZcezXRPykQT3O5PGuhlpCIQCfRJOh
y5+NQC2kJ4aADtbzFvmU9lg=
=OwY/
-----END PGP SIGNATURE-----

Revision history for this message

In freedesktop.org Bugzilla #4738, Rdieter-math (rdieter-math) wrote on 2007-01-24:

#14

Created an attachment (id=8494)
fix up check/check-requires-private too

Revision history for this message

In Red Hat Bugzilla #224148, Rex (rex-redhat-bugs) wrote on 2007-01-24:

#33

Using pkg-config-0.21 given /usr/lib/pkgconfig/foo.pc
containing:

##############
prefix=/usr
exec_prefix=/usr
libdir=/usr/lib
includedir=/usr/include

Name: foo
Description: fof Library
Version: 1.0
Requires.private: bar
Cflags: -I${includedir}
Libs: -L${libdir} -lfoo
##################

$ pkg-config --libs foo
Package bar was not found in the pkg-config search path.
Perhaps you should add the directory containing `bar.pc'
to the PKG_CONFIG_PATH environment variable
Package 'bar', required by 'foo', not found

Libs.private and Requires.private should be ignored and not be used when/if one
doesn't use
$ pkg-config --static

This has been reported upstream:
https://bugs.freedesktop.org/show_bug.cgi?id=4738
with patch supplied (testing that now myself to confirm it works as advertised).

Please consider including this patch, as it fixes a few concrete examples of
brokenness, in particular,
$ yum remove libXext-devel
$ yum install libXmu-devel
$ pkgconfig --libs Xmu
Package xext was not found in the pkg-config search path.
Perhaps you should add the directory containing `xext.pc'
to the PKG_CONFIG_PATH environment variable
Package 'xext', required by 'Xmu', not found
since xmu.pc contains:
Requires.private: ... xext

Revision history for this message

In Red Hat Bugzilla #224148, Rex (rex-redhat-bugs) wrote on 2007-01-24:

#34

oops, the example *should* be:
$ pkg-config --libs xmu

Revision history for this message

In Red Hat Bugzilla #224148, Rex (rex-redhat-bugs) wrote on 2007-01-24:

#35

Created attachment 146406
update to upstream patch to fixup 'make check' behavior as well

Revision history for this message

In Red Hat Bugzilla #224148, Rex (rex-redhat-bugs) wrote on 2007-01-24:

#36

A can confirm patch works as advertised.

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2007-01-30:

#37

I'll wait a bit for Tolleef to make up his mind on the upstream bug. He said
that he would have time for pkg-config bugs in a week.

Revision history for this message

In Red Hat Bugzilla #224148, Ralf (ralf-redhat-bugs) wrote on 2007-02-02:

#38

I think you are vastly underestimating the severity of this bug.

* This bug basically voids Requires.Private.

* it forces developers and packagers to pollute configure scripts, rpm.spec and
build scripts with avoidable (actually bogus) hacks.

In the example above, one will have to add xext to pkg-config checks in
configure-scripts, and will have to add "BuildRequires: libXext-devel" to specs
(otherwise mock will not work).

* it breaks previously (before Requires.Private had been introduced) functional
pkg-config deps and configure scripts.
Before Requires.Private, these deps had been Requires in *.pc, therefore they
had been automatically taken into account.

Therefore, I am making this a review blocker for fc7 also would recommend you to
fix it on older Fedoras, too.

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2007-02-02:

#39

Making this a "review blocker" is not going to force my hand.
What do you want to do next, drop pkgconfig from fedora ?
The last thing I want is a pkgconfig in fedora that diverges from upstream in
how it treats dependencies.

Revision history for this message

In Red Hat Bugzilla #224148, Ralf (ralf-redhat-bugs) wrote on 2007-02-02:

#40

(In reply to comment #6)
> Making this a "review blocker" is not going to force my hand.
Your decision - and fault

> What do you want to do next, drop pkgconfig from fedora ?
You'd better fix the bugs.

> The last thing I want is a pkgconfig in fedora that diverges from upstream in
> how it treats dependencies.
If upstream stuff is nonfunctional, you will not be able to avoid to apply
patches and to diverge from upstream - This is the natural movement of all
packages and should be natural to any maintainer (You already do so with other
patches in pkgconfig)

In this particular case, an alternative would be to replace all
Requires.Private: from all *.pc's - but, you don't really want this, don't you?

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2007-02-02:

#41

> Your decision - and fault

Oh great. So I was wrong to thing that it was Rex fault to tie a package review
to an issue that is totally unrelated to packaging ?

Revision history for this message

In Red Hat Bugzilla #224148, Mamoru (mamoru-redhat-bugs) wrote on 2007-02-02:

#42

*** Bug 215942 has been marked as a duplicate of this bug. ***

Revision history for this message

In freedesktop.org Bugzilla #4738, Matthias Clasen (mclasen) wrote on 2007-02-06:

#15

Tollef, any chance you can look at this patch soon ?
People really want me to fix this in Fedora, but I really don't want to see diverging dependency resolution between upstream and downstream pkg-config,
so I'd be happy if this patch could go in.

Revision history for this message

In freedesktop.org Bugzilla #4738, Freedesktop-tevp (freedesktop-tevp) wrote on 2007-03-29:

#16

If cairo has libpng12 in it's Requires.private, then this should imply that cairo both requires libpng12 header files in it's public headers, and therefore needs it for compiling individual files *and* has a library dependancy on it. If OTOH, cairo only has a library dependancy on libpng12, and not any dependancies in it's public headers on libpng12, then shouldn't that be a Libs.private not Requires.private?

Revision history for this message

In freedesktop.org Bugzilla #4738, Matthias Clasen (mclasen) wrote on 2007-03-29:

#17

Tom, cairo uses Requires.private instead of Libs.private since the private dependency has a pc file.

I have put the patch in the Fedora package now, since it doesn't look as if Tollef will act on them anytime soon.

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2007-03-30:

#43

Fixed in 0.21-5.fc7

Revision history for this message

In freedesktop.org Bugzilla #4738, Dbn-lists (dbn-lists) wrote on 2007-10-19:

#18

This patch unfortunately breaks the behavior added in 0.21 where the Cflags are always listed from Requires.private packages, irrespective of --static.

http://lists.freedesktop.org/archives/xorg/2007-October/029427.html

Maybe another ignore_private_requires boolean can be added and set when --exists is used.

Revision history for this message

In freedesktop.org Bugzilla #4738, Dbn-lists (dbn-lists) wrote on 2007-10-19:

#19

Created an attachment (id=12126)
Simpler fix to just toggle ignore_requires when using --exists

I believe this patch does the right thing. When --exists is in use, it toggles the ignore_requires boolean to TRUE. The parsing will then skip Requires and Requires.private. I checked that the Requires.private behavior for --cflags and --libs worked correctly with and with --static, too.

Revision history for this message

In freedesktop.org Bugzilla #4738, Dbn-lists (dbn-lists) wrote on 2007-12-18:

#20

Created an attachment (id=13197)
Handle Requires and Requires.private separately from main.c

I realized that the previous patch still did not fix the case where a Requires.private was missing and the user requested --libs. This patch adds a new global and helper functions for ignoring Requires.private in addition to the Requires helper functions from the previous patch.

Now, Requires are ignored from main.c unless the user has requested Cflags or Libs. The Requires.private field is ignored unless the user has requested Cflags or static Libs.

Revision history for this message

In freedesktop.org Bugzilla #4738, Dbn-lists (dbn-lists) wrote on 2007-12-18:

#21

Created an attachment (id=13198)
Tests for the case of a missing Requires package

Here are some new tests which check the behavior of pkg-config with various requests when a package from Requires is missing. Along with the previous patch and the earlier patch to check-requires-private, all tests pass.

Revision history for this message

In freedesktop.org Bugzilla #4738, Leio-gentoo (leio-gentoo) wrote on 2008-11-01:

#22

Apparently the link referenced in comment #7 goes to an unrelated post these days, probably due to some mailing list management changes. I'm quite sure the following thread was meant instead, after digging around 2007-October full archive to find it:

http://lists.freedesktop.org/archives/xorg/2007-October/028961.html

The most discussion on the subject is at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=340904 that tries to explain why cflags need to recursively be pulled in for even Requires.private. I'm not particularly agreeing with that, but I see it is necessary as long as there is no way to specify header-only requirements separately, which in turn might be an overkill and as cflags doesn't really imply any extra dynamic dependencies I can live with it.

I found out this bug bites us hard in Gentoo Linux as well, especially in conjunction with xorg proto packages that only provide headers (as for regular packages we don't separate them into regular and -dev packages as is the case in comment #1). Our packages are all built from source even for users, and build time dependencies are handled separately from runtime dependencies, whereas only build time dependencies can be removed after installing (it involves compiling for users), or not installed at all for the case of prebuilt binary packages. So the current state in vanilla pkg-config-0.23 means we'd have to build-time depend on many xorg proto header-only packages for all gtk+ using packages (for example) because pkg-config --exists gtk+-2.0 pulls in for example renderproto via gtk+-2.0 -> cairo -> xrender -> renderproto (latter two being only Requires.private). A fix for this bug would avoid that, and I will need to include such a patch in Gentoo as well, differing from vanilla similar to Fedora.

Please give some attention to this bug, thank you.

Somewhat unrelated to that, but where does pkg-config version control system reside these days...?

Revision history for this message

In Red Hat Bugzilla #224148, Dan (dan-redhat-bugs) wrote on 2008-11-29:

#44

I'm sorry, but this fix is broken. The vanilla pkg-config behavior for Requires.private is correct, and this just makes it act like Libs.private, which is wrong because it breaks Cflags handling.

In the example given above, rpm/yum would not allow you to remove libXext-devel if libXmu-devel requires pkgconfig(xext). It would do that if the pkg-config --print-requires patch also walked Requires.private. I opened bug 426106 that has patches to fix that. After all, you actually need the headers from libXext-devel installed even if you're not linking to libXext.a. So, just making pkg-config skip Requires.private unless --static is used is wrong.

The issue of whether the .pc file should be required to be installed under certain conditions should be handled separately. That's what the freedesktop bug is about. Incidentally, I have more patches to address that issue, but that's orthogonal to this.

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2008-11-29:

#45

It is certainly hard to argue about 'correctness' of totally unspecified and undocumented features like Requires.private.

If upstream comes up with some specification for Requires.private and a matching implementation, that would be great.

Revision history for this message

In Red Hat Bugzilla #224148, Dan (dan-redhat-bugs) wrote on 2008-11-29:

#46

But there is a concrete specification and implementation.

Requires: Recursively fetch Cflags and Libs from listed dependencies. Recursively fetch Libs.private if --static is specified.

Requires.private: Recursively fetch Cflags from listed dependencies. Recursively fetch Libs and Libs.private if --static is specified.

This is what vanilla pkg-config does. What pkg-config doesn't do well is handle the non-existence of .pc files when they aren't needed. In the fdo bug, the reporter is talking about --exists. We can probably agree that pkg-config shouldn't need to recursively search for listed Requires* because we only want to know about the existence of the .pc metadata for the requested package.

In this bug, the problem is that pkg-config keeps searching for .pc files for listed Requires.private even though the user has requested a case where they're not needed: --libs. It would be better if pkg-config didn't search for the listed .pc file in this case, but having it act like Libs.private is wrong because the Cflags are always needed. Another mode is needed than ignore_private_libs to handle that correctly. In the fdo bug, I have a patch which does just that, adding a ignore_requires_private flag:

https://bugs.freedesktop.org/attachment.cgi?id=13197

What I'm proposing is to punt on that issue for now by making rpm add autoreqs for packages listed in Requires and Requires.private. After all, you do need the -devel package for Requires.private packages installed to actually do anything useful because you need the headers and .so links. I.e., libXmu-devel should require libXext-devel since you actually need the headers and libXext.so/.a to compile a libXmu program. While calling `pkg-config --libs xmu' shouldn't strictly need libXext-devel to be installed, it is a lower priority failure than the failure to actually compile and link a libXmu program.

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2008-11-29:

#47

> But there is a concrete specification and implementation.

Where is that specification take from ?

man pgk-config | grep Requires.private comes up empty...

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2008-11-29:

#48

Btw, the specification that I'd really like to see written is not one that talks about the algorithm used for handling various pc file field (though that needs writing too), but one that would be titled "Software development using pkg-config". It would answer questions like

1) My program uses library x. What do I do ?
Answer:
CFLAGS=`pkg-config --cflags x`
LIBS=`pkg-config --libs x`
(followed by a longer explanation of how this looks in autofoo)

2) My library z installs header files which include libx headers. What do I put in my z.pc file ?
...

3) My library z uses libx internally, but does not expose libx datatypes in its public api. What do I put in my z.pc file ?
...

Etc.

Revision history for this message

In Red Hat Bugzilla #224148, Dan (dan-redhat-bugs) wrote on 2008-11-29:

#49

OK, I grant you that the specification is not there, the documentation is poor, and the upstream maintainer is not responsive. Do you agree with my description of the Requires.private implementation, though? And do you agree that this patch breaks handling of --cflags with Requires.private?

I would also be willing to write such a guide if anyone can manage to get Tollef Fog Heen on board.

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2008-12-05:

#50

Ok, I'm going to drop that patch, and you write that guide. Deal ? :-)
Just need to wait for the build system to come back...

Revision history for this message

In Red Hat Bugzilla #224148, Dan (dan-redhat-bugs) wrote on 2008-12-05:

#51

Yay! Thanks, Matthias. I was thinking about writing a pkg-config(7) that explained pkg-config concepts. Maybe an html guide would be better, though.

What about the patch in bug 426106 for also outputting Requires.private with --print-requires? I think that'll be needed to avoid this type of issue from cropping up again.

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2008-12-08:

#52

Fixed in pkgconfig-0.23-6.fc11.

Revision history for this message

In freedesktop.org Bugzilla #4738, Vincent Untz (vuntz) wrote on 2009-01-24:

#23

Was hit by this in openSUSE too :-)

Revision history for this message

In freedesktop.org Bugzilla #4738, Vincent Untz (vuntz) wrote on 2009-01-30:

#24

Some notes from Tollef at http://err.no/personal/blog/tech/2008-03-25-18-07_pkg-config,_sonames_and_Requires.private

I still disagree, though ;-)

Revision history for this message

In Red Hat Bugzilla #224148, Nicolas (nicolas-redhat-bugs) wrote on 2009-02-17:

#53

(In reply to comment #11)
> I'm sorry, but this fix is broken. The vanilla pkg-config behavior for
> Requires.private is correct, and this just makes it act like Libs.private,
I can agree that keeping the original pkg-config behavior should have been made.
But now having the patch reverted in F-10 will broke near everything. Each package maintainer try to fix thing at sight which will sooner or later have a complete rebuild of the whole distro because of bugs in chain.
(example https://bugzilla.redhat.com/show_bug.cgi?id=485667 )

I'm not sure to understand clearly what is the original pkgconfig behavior, my understanding stop at how pkgconfig creates binary linked statically versus shared library using static library as dependency.
It seems to me that requires.private should be used when building a binary and libs.private when building a library.

Anyway... Fedora doesn't use static library that much (it means static libs aren't even build at all in the majority cases). But If the need of a static library appears, then thus library should be moved to a -static subpackage which will requires the -devel one.
That's the developer responsibility to request the -static version at build time when needed. The static version will then requires every dependencies mentioned in the pkgconfig requires.private line, and that could eventually be autorequired by rpm. But the original title of this bugreport remains valid:
The "shared object only -devel sub-package" shouldn't requires.private dependency in any cases.(but this requires.private should belong to the -static one instead). It shouldn't also fail when building shared libs or against shared libs because either no static lib is present or some requires.private .pc are missing.

I'm sorry but this bug is re-opened, we (as packages maintainers) need to know how to handle the pkg-config behavior change. Either to mass-rebuild the whole F-10 distro or to revert to the ancient pkg-config.
See this case study for example: https://bugzilla.redhat.com
/show_bug.cgi?id=485667

(In reply to comment #11)
> I'm sorry, but this fix is broken. The vanilla pkg-config behavior for
> Requires.private is correct, and this just makes it act like Libs.private,
I can agree that keeping the original pkg-config behavior should have been made.
But now having the patch reverted in F-10 will broke near everything. Each package maintainer try to fix thing at sight which will sooner or later have a complete rebuild of the whole distro because of bugs in chain.
(example https://bugzilla.redhat.com/show_bug.cgi?id=485667 )

I'm not sure to understand clearly what is the original pkgconfig behavior, my understanding stop at how pkgconfig creates binary linked statically versus shared library using static library as dependency.
It seems to me that requires.private should be used when building a binary and libs.private when building a library.

Anyway... Fedora doesn't use static library that much (it means static libs aren't even build at all in the majority cases). But If the need of a static library appears, then thus library should be moved to a -static subpackage which will requires the -devel one.
That's the developer responsibility to request the -static version at build time when needed. The static version will then requires every dependencies mentioned in the pkgconfig requires.private line, and that could eventually be autorequired by rpm. But the original title of this bugreport remains valid:
The "shared object only -devel sub-package" shouldn't requires.private dependency in any cases.(but this requires.private should belong to the -static one instead). It shouldn't also fail when building shared libs or against shared libs because either no static lib is present or some requires.private .pc are missing.

I'm sorry but this bug is re-opened, we (as packages maintainers) need to know how to handle the pkg-config behavior change. Either to mass-rebuild the whole F-10 distro or to revert to the ancient pkg-config.
See this case study for example: https://bugzilla.redhat.com
/show_bug.cgi?id=485667

Revision history for this message

In Red Hat Bugzilla #224148, Dan (dan-redhat-bugs) wrote on 2009-02-18:

#54

It's not the static libraries that make the difference. It's that we need the headers and the .so regardless. In the bug you referenced, gconf2-devel doesn't have a BR on dbus-devel, so you can remove dbus-devel without removing gconf2-devel. However, you need the dbus headers (and library) to use gconf2-devel, right? I'd argue that the package was already broken. It's not like you could have compiled and linked a GConf app without dbus-devel installed.

The right way to handle this is to Requires the things that the package needs. It's broken to not require a package where you need the -devel package. I understand that can make life tough for packagers, but the previous behavior was more frustrating for developers. Imagine trying to develop software and having to consider that the tool you're using (pkg-config) works differently on one distro than another. This effectively meant that they could not use the Requires.private feature because it broke for anyone using fedora (quite a few as you can imagine).

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2009-02-18:

#55

> I'm sorry but this bug is re-opened, we (as packages maintainers) need to know
> how to handle the pkg-config behavior change. Either to mass-rebuild the whole
> F-10 distro or to revert to the ancient pkg-config.

I don't see how a mass rebuild of the whole F10 comes even remotely into play here. Yes, it is somewhat unfortunate that pkg-config changed its behaviour in the middle of stable releases, which is why I didn't push this out eagerly, but only after other people complained about behaviour difference between F10 and rawhide.

You may have to adjust BuildRequires slightly as you build updates, that is all.

Revision history for this message

In Red Hat Bugzilla #224148, Nicolas (nicolas-redhat-bugs) wrote on 2009-02-18:

#56

Download full text (3.9 KiB)

(In reply to comment #21)
> It's not the static libraries that make the difference. It's that we need the
> headers and the .so regardless.
No we don't. And this is exactly what the current bug report try to address.(look at the title).
Despite it should have been made in the pkgconfig directly instead.

In this case, using GConf2-devel doesn't requires dbus-devel at all as;
when building an application using GConf2-devel:
- The GConf2-devel headers doesn't use dbus headers explicitely. (it will use glib2-devel and popt-devel as shown by "grep include /usr/include/gconf/2/gconf/* -R" )
- The app using GConf2-devel shouldn't explicitly use dbus-devel as a mandatory requirement in order to build (as it will not use dbus headers directly either).
- As we link to a shared GConf2, we only need to pass -lGConf2 and libs fetched from Requires: so we don't get undefined-non-weak-symbol since thoses are potentially exposed from the API.

- Since We are not using headers from Requires.private in GConf2 and dependent application, the include path and definition that can eventually be present there aren't needed also. It ends that the whole cflags from Requires.private aren't needed at all( even if using pkgconfig -static.)

- It was working fine previously, and rpmlint didn't reported undefined-non-weak-symbol on a GConf2 dependent package AFAIK).

Now I'm all for you to point me a case where the use of Requires.private is correct in a package as starting point; So I can understand how pkgconfig should handle it.

Just as a note. If a given -devel is installed, the .so and headers will be there, but not the static library. If a given -static library is installed, then it will requires the -devel (which will requires the .so and header).

>... In the bug you referenced, gconf2-devel doesn't
> have a BR on dbus-devel, so you can remove dbus-devel without removing
> gconf2-devel. However, you need the dbus headers (and library) to use
> gconf2-devel, right?
Nope, it was working fine previously and the bugreport started with the pkg-config updates.

> like you could have compiled and linked a GConf app without dbus-devel
> installed.
That was how it behaved previously, and it worked like a charm.

Now I want you to answear few questions:

1 - Why pkg-config should look at Requires.private when not using -static ?

2 - Why either Requires.private (and Libs.private after all) aren't emptied when --disable-static is made. (static library is disabled at build time).

3 - When both shared and static version are built and only the shared version is shipped, Why the requires of the -devel package should cover the needs of the static case ? (which will be larger than the shared only accurate requirements).

4 - When should a library use -static with pkgconfig dependencies ?

5 - When should a binary use -static with pkgconfig dependencies ?

6 - What is the correct behaviour between theses two:
Whith the pkg-config GA behaviour:
[root@kwizatz gconf]# pkg-config --cflags gconf-2.0
-I/usr/include/gconf/2 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
With the current pkg-config updates behaviour:
[root@kwizatz gconf]# pkg-config --cflags gconf-2.0
-DORBIT...

(In reply to comment #21)
> It's not the static libraries that make the difference. It's that we need the
> headers and the .so regardless.
No we don't. And this is exactly what the current bug report try to address.(look at the title).
Despite it should have been made in the pkgconfig directly instead.

In this case, using GConf2-devel doesn't requires dbus-devel at all as;
when building an application using GConf2-devel:
- The GConf2-devel headers doesn't use dbus headers explicitely. (it will use glib2-devel and popt-devel as shown by "grep include /usr/include/gconf/2/gconf/*  -R" )
- The app using GConf2-devel shouldn't explicitly use dbus-devel as a mandatory requirement in order to build (as it will not use dbus headers directly either).
- As we link to a shared GConf2, we only need to pass -lGConf2 and libs fetched from Requires: so we don't get undefined-non-weak-symbol since thoses are potentially exposed from the API.

- Since We are not using headers from Requires.private in GConf2 and dependent application, the include path and definition that can eventually be present there aren't needed also. It ends that the whole cflags from Requires.private aren't needed at all( even if using pkgconfig -static.)

- It was working fine previously, and rpmlint didn't reported undefined-non-weak-symbol on a GConf2 dependent package AFAIK).

Now I'm all for you to point me a case where the use of Requires.private is correct in a package as starting point; So I can understand how pkgconfig should handle it.

Just as a note. If a given -devel is installed, the .so and headers will be there, but not the static library. If a given -static library is installed, then it will requires the -devel (which will requires the .so and header).

>... In the bug you referenced, gconf2-devel doesn't
> have a BR on dbus-devel, so you can remove dbus-devel without removing
> gconf2-devel. However, you need the dbus headers (and library) to use
> gconf2-devel, right? 
Nope, it was working fine previously and the bugreport started with the pkg-config updates.

> like you could have compiled and linked a GConf app without dbus-devel
> installed.
That was how it behaved previously, and it worked like a charm.

Now I want you to answear few questions:

1 - Why pkg-config should look at Requires.private when not using -static ?

2 - Why either Requires.private (and Libs.private after all) aren't emptied when --disable-static is made. (static library is disabled at build time).

3 - When both shared and static version are built and only the shared version is shipped, Why the requires of the -devel package should cover the needs of the static case ? (which will be larger than the shared only accurate requirements).

4 - When should a library use -static with pkgconfig dependencies ?

5 - When should a binary use -static with pkgconfig dependencies ?

6 - What is the correct behaviour between theses two:
Whith the pkg-config GA behaviour: 
[root@kwizatz gconf]# pkg-config --cflags gconf-2.0
-I/usr/include/gconf/2 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
With the current pkg-config updates behaviour:
[root@kwizatz gconf]# pkg-config --cflags gconf-2.0
-DORBIT2=1 -pthread -I/usr/include/gconf/2 -I/usr/include/orbit-2.0 -I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include

7 - as you said in c#13:
>Requires: Recursively fetch Cflags and Libs from listed dependencies.
>Recursively fetch Libs.private if --static is specified.
>
>Requires.private: Recursively fetch Cflags from listed dependencies.
>Recursively fetch Libs and Libs.private if --static is specified.
Why Cflags are fetched from Requires.private if not using -static instead of just using Cflags from the given .pc ?

8 - As I'm building a package using GConf2-devel, I have:
Requires.private: ORBit-2.0 dbus-1
How can I do if I want to link to a shared version of dbus , but a static version or ORBit-2.0 ?

Revision history for this message

In Red Hat Bugzilla #224148, Dan (dan-redhat-bugs) wrote on 2009-02-18:

#57

Now that I look at gconf, I see that you're right. This case (library a uses library b internally but does not expose its datatypes) is not handled well by pkg-config. Read this guide if you want to understand pkg-config (skip to the FAQs if you want).

http://people.freedesktop.org/~dbn/pkg-config-guide.html

A workaround would be to add -ldbus-1 to Libs.private and remove dbus-1 from Requires.private in gconf-2.0.pc. Or, since the GConf2 rpm doesn't even install libgconf-2.a, drop all mentions of dbus in the .pc file. A longer term solution would have to be added to pkg-config.

Revision history for this message

In Red Hat Bugzilla #224148, Nicolas (nicolas-redhat-bugs) wrote on 2009-02-19:

#58

Download full text (3.8 KiB)

I've read the guide, but the few words about the Requires/Requires.private doesn't help me to understand the usage. (I've understood the behaviour, but I do not see the matching usage).
So what is following is what i imagine of the differences between Requires and Requires.private.

1-
Actually, if a package use the Cflags from Requires.private for the includepath. It means it will use the headers included from Requires.private. As a consequence, it will <<have to>> "use symbols" of the related libs from Requires.private. Which, if they aren't defined at build time (linking) will need to be resolved at runtime. (rpmlint on installed packages will show undefined-non-weak-symbol).
(If you don't agree with this part you will have to give an example).
I don't see how a package could see some interest in linking to gconf while not linking to dbus directly.(if ever dbus headers was used in GConf2-devel which is not the case).
I could imagine source distribution like gentoo could see some interest in having such behaviour to avoid relinking to the new SOVERSION of a library, but it would fall under the dirty hack.

We (as precompiled distribution as a whole) want this case to fail at install time (instead of opening the possibility of missing symbol at runtime).
It could be possible to have exeption, but it is the common case. And we rely the fact that shared libraries are linked at build time with an appropriate SOVERSION.

2-
Then I've wondered if it has something to do with building software from a tree.
The main feature between tree vs installed software are that "private headers" start to be accessible.
So it would have been unappropriate to use Requires.private from gconf-2.0.pc instead of gconf-2.0-uninstalled.pc.
But even in this case, there is a need to link to libs (more than ever actually). And fetching both Cflags and libs is exactly the same behaviour as Requires:). So maybe the Requires line of a given *-(installed).pc can be different than *-(uninstalled).pc, but it still doesn't explain the different usage of Requires.private versus Requires

3-
Now what is misleading (see http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-November/056017.html ) is that as Libs.private is known to do have something to do with static library, some might expect the same with Requires.private. But it is not. (that was the previously patched version of this bug, more or less).

The previous case was more or less rare cases, whereas this is a way more common. When a binary is aimed to be portable on any given GNU/Linux distro it may links against a static version of a library. But such library will only contain symbol of itself, hence there is a need for the static linker to search for its dependency also. Here is what would Requires.private aimed for.
It helps the linker to find symbols needed by a given static library. If only a shared library exist, it stop to the Libs. If it is a static library, then it pick Requires.private in the related .pc file and so on.

Then pkgconfig could look at some environment value to say when both static version and shared version exist which ones to pick. and eventually, which versioned symbol (like libresolv.so.2(...

I've read the guide, but the few words about the Requires/Requires.private doesn't help me to understand the usage. (I've understood the behaviour, but I do not see the matching usage).
So what is following is what i imagine of the differences between Requires and Requires.private.

1- 
Actually, if a package use the Cflags from Requires.private for the includepath. It means it will use the headers included from Requires.private. As a consequence, it will <<have to>> "use symbols" of the related libs from Requires.private. Which, if they aren't defined at build time (linking) will need to be resolved at runtime. (rpmlint on installed packages will show undefined-non-weak-symbol).
(If you don't agree with this part you will have to give an example).
I don't see how a package could see some interest in linking to gconf while not linking to dbus directly.(if ever dbus headers was used in GConf2-devel which is not the case).
I could imagine source distribution like gentoo could see some interest in having such behaviour to avoid relinking to the new SOVERSION of a library, but it would fall under the dirty hack.

We (as precompiled distribution as a whole) want this case to fail at install time (instead of opening the possibility of missing symbol at runtime).
It could be possible to have exeption, but it is the common case. And we rely the fact that shared libraries are linked at build time with an appropriate SOVERSION.

2-
Then I've wondered if it has something to do with building software from a tree.
The main feature between tree vs installed software are that "private headers" start to be accessible. 
So it would have been unappropriate to use Requires.private from gconf-2.0.pc instead of gconf-2.0-uninstalled.pc.
But even in this case, there is a need to link to libs (more than ever actually). And fetching both Cflags and libs is exactly the same behaviour as Requires:). So maybe the Requires line of a given *-(installed).pc can be different than *-(uninstalled).pc, but it still doesn't explain the different usage of Requires.private versus Requires

3-
Now what is misleading (see http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-November/056017.html ) is that as Libs.private is known to do have something to do with static library, some might expect the same with Requires.private. But it is not. (that was the previously patched version of this bug, more or less).

The previous case was more or less rare cases, whereas this is a way more common. When a binary is aimed to be portable on any given GNU/Linux distro it may links against a static version of a library. But such library will only contain symbol of itself, hence there is a need for the static linker to search for its dependency also. Here is what would Requires.private aimed for.
It helps the linker to find symbols needed by a given static library. If only a shared library exist, it stop to the Libs. If it is a static library, then it pick Requires.private in the related .pc file and so on.

Then pkgconfig could look at some environment value to say when both static version and shared version exist which ones to pick. and eventually, which versioned symbol (like libresolv.so.2(GLIBC_2.9) in glibc) need to be picked from the static version, and other (ex: -lGL) from the shared version.

But it would mean pkgconfig --libs package --static would fetch the Requires.privates libs and not the cflags, which is the opposite of what the vannilla pkgconfig do.

@Matthias
As you've said previously, you were reluctant to do the update. 
Whatever your current personal thought are; I do think we need a "common decision" to be made, so we can advertise packagers about how to eventually fix things in stable release accurately. (and send the patches upstream as appropriate).

So I'm requesting a topic about "pkgconfig how to deal with Requires.private" to be discussed at friday 20th FESCo.

Revision history for this message

In Red Hat Bugzilla #224148, Dan (dan-redhat-bugs) wrote on 2009-02-19:

#59

The common case where one library uses datatypes from another:

$ cat /usr/lib/pkgconfig/cairo.pc
prefix=/usr
exec_prefix=/usr
libdir=/usr/lib
includedir=/usr/include

Name: cairo
Description: Multi-platform 2D graphics library
Version: 1.6.4

Requires.private: freetype2 >= 9.7.3 fontconfig libpng12 xrender >= 0.6 x11 pixman-1 >= 0.10.0
Libs: -L${libdir} -lcairo
Libs.private: -lz -lz -lm
Cflags: -I${includedir}/cairo

I'm writing a cairo app, so I want to link it to libcairo. However, in order to compile my app, it also needs to be get the Xrender headers and link to libXrender. Now, I could directly link my app to libXrender, but since I don't use any Xrender symbols, it's extra baggage. The linker will still resolve all the symbols at build time because libXrender is marked with a a DT_NEEDED tag in libcairo.

$ readelf -d /usr/lib/libcairo.so | grep NEEDED
0x00000001 (NEEDED) Shared library: [libz.so.1]
0x00000001 (NEEDED) Shared library: [libfreetype.so.6]
0x00000001 (NEEDED) Shared library: [libfontconfig.so.1]
0x00000001 (NEEDED) Shared library: [libpng12.so.0]
0x00000001 (NEEDED) Shared library: [libXrender.so.1]
0x00000001 (NEEDED) Shared library: [libX11.so.6]
0x00000001 (NEEDED) Shared library: [libpixman-1.so.0]
0x00000001 (NEEDED) Shared library: [libm.so.6]
0x00000001 (NEEDED) Shared library: [libc.so.6]

So, there's no runtime lookup of undefined symbols, I just want link my app with "gcc -lcairo -o myapp myapp.c" instead of "gcc -lcairo -lXrender -lX11 ...". On the other hand, static linking myapp would require that I pull in all the libraries to resolve all symbols.

$ pkg-config --libs cairo
-lcairo
$ pkg-config --libs --static cairo
-lcairo -lfreetype -lfontconfig -lexpat -lpng12 -lz -lm -lXrender -lpixman-1 -lX11 -lpthread -lxcb-xlib -lxcb -lXau -lXdmcp

Regardless of how I want to link cairo, I need to be able to find the headers for fontconfig, Xrender and X11 since the datatypes are exposed by cairo.

That's the extremely common case where one library uses another library's datatypes and symbols. It does have something to do with static linking, but it's all part of not acquiring more DT_NEEDED libs then necessary.

http://wiki.mandriva.com/en/Overlinking

As I said before, the case you're describing with gconf and dbus is different and is not handled really well by pkg-config since it expects that you will need to access the dbus headers when compiling. Maybe there should be another pkg-config field like Requires.privatelib that does not expose the CFLAGS.

The common case where one library uses datatypes from another:

$ cat /usr/lib/pkgconfig/cairo.pc 
prefix=/usr
exec_prefix=/usr
libdir=/usr/lib
includedir=/usr/include

Name: cairo
Description: Multi-platform 2D graphics library
Version: 1.6.4

Requires.private: freetype2 >= 9.7.3 fontconfig libpng12 xrender >= 0.6 x11 pixman-1 >= 0.10.0 
Libs: -L${libdir} -lcairo
Libs.private: -lz -lz -lm
Cflags: -I${includedir}/cairo

I'm writing a cairo app, so I want to link it to libcairo. However, in order to compile my app, it also needs to be get the Xrender headers and link to libXrender. Now, I could directly link my app to libXrender, but since I don't use any Xrender symbols, it's extra baggage. The linker will still resolve all the symbols at build time because libXrender is marked with a a DT_NEEDED tag in libcairo.

$ readelf -d /usr/lib/libcairo.so | grep NEEDED
 0x00000001 (NEEDED)                     Shared library: [libz.so.1]
 0x00000001 (NEEDED)                     Shared library: [libfreetype.so.6]
 0x00000001 (NEEDED)                     Shared library: [libfontconfig.so.1]
 0x00000001 (NEEDED)                     Shared library: [libpng12.so.0]
 0x00000001 (NEEDED)                     Shared library: [libXrender.so.1]
 0x00000001 (NEEDED)                     Shared library: [libX11.so.6]
 0x00000001 (NEEDED)                     Shared library: [libpixman-1.so.0]
 0x00000001 (NEEDED)                     Shared library: [libm.so.6]
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]

So, there's no runtime lookup of undefined symbols, I just want link my app with "gcc -lcairo -o myapp myapp.c" instead of "gcc -lcairo -lXrender -lX11 ...". On the other hand, static linking myapp would require that I pull in all the libraries to resolve all symbols.

$ pkg-config --libs cairo
-lcairo
$ pkg-config --libs --static cairo
-lcairo -lfreetype -lfontconfig -lexpat -lpng12 -lz -lm -lXrender -lpixman-1 -lX11 -lpthread -lxcb-xlib -lxcb -lXau -lXdmcp

Regardless of how I want to link cairo, I need to be able to find the headers for fontconfig, Xrender and X11 since the datatypes are exposed by cairo.

That's the extremely common case where one library uses another library's datatypes and symbols. It does have something to do with static linking, but it's all part of not acquiring more DT_NEEDED libs then necessary.

http://wiki.mandriva.com/en/Overlinking

As I said before, the case you're describing with gconf and dbus is different and is not handled really well by pkg-config since it expects that you will need to access the dbus headers when compiling. Maybe there should be another pkg-config field like Requires.privatelib that does not expose the CFLAGS.

Revision history for this message

In Red Hat Bugzilla #224148, Nicolas (nicolas-redhat-bugs) wrote on 2009-02-19:

#60

Download full text (3.4 KiB)

(In reply to comment #26)
...
> I'm writing a cairo app, so I want to link it to libcairo. However, in order to
> compile my app, it also needs to be get the Xrender headers and link to
> libXrender. Now, I could directly link my app to libXrender, but since I don't
> use any Xrender symbols, it's extra baggage. The linker will still resolve all
> the symbols at build time because libXrender is marked with a a DT_NEEDED tag
> in libcairo.
I Don't use cairo, so forgive me if I say weird things.
If you really do not use libXrender symbol while using cairo, why just not include cairo-xlib-xrender.h which is the only one to declare X11/extensions/Xrender.h. In the majority of the cases, cairo.pc should have Requires: xrender >= 0.6 IMO.
Now if linking with xrender is only optional when using cairo, then it should be handled in another .pc or if cairo apps aren't expecting to directly links xrender at all, then it should be best to have X11/extensions/Xrender.h not included at all from the cairo-devel (so it will not be exposed from the cairo API).

> $ readelf -d /usr/lib/libcairo.so | grep NEEDED
The internal dependency of cairo itself is not in question.
...
> So, there's no runtime lookup of undefined symbols, I just want link my app
As you have defined your fictional app not to use symbol from xrender, it is necessary true. But doesn't help with some more info either.

> with "gcc -lcairo -o myapp myapp.c" instead of "gcc -lcairo -lXrender -lX11
> ...". On the other hand, static linking myapp would require that I pull in all
> the libraries to resolve all symbols.
What if an application (compiled gcc -lcairo -o myapp myapp.c ) using cairo and xrender symbol is provided on a system where cario have been compiled without xrender and the xrender library isn't available ?

> Regardless of how I want to link cairo, I need to be able to find the headers
> for fontconfig, Xrender and X11 since the datatypes are exposed by cairo.
So why they aren't in Requires: instead of Requires.private ?
OK , there is the Overlink problem , so another workaround would be to have @LIBX11_CFLAGS@ @LIBX11_CFLAGS@ @LIBX11_CFLAGS@ added in cairo.pc.in at the Clags libs.Then the cairo maintainer would have to requires the related -devel package manually.

But in the majority of the cases, Cflags and Libs from Requires.private doesn't matches. When building your application against cario shared/static, you don't need the -I/usr/include/libpng12 for example. while you will need the -lpng while your binary will be linked statically.
This is usually the case since whenever it is possible, internal dependencies should be hidden from the API.

> That's the extremely common case where one library uses another library's
> datatypes and symbols. It does have something to do with static linking, but
> it's all part of not acquiring more DT_NEEDED libs then necessary.
A agree that there is a need to reduce linking whenever it is possible.
> http://wiki.mandriva.com/en/Overlinking

> As I said before, the case you're describing with gconf and dbus is different
> and is not handled really well by pkg-config since it expects that you will
> need to access the dbus headers when compi...

(In reply to comment #26)
...
> I'm writing a cairo app, so I want to link it to libcairo. However, in order to
> compile my app, it also needs to be get the Xrender headers and link to
> libXrender. Now, I could directly link my app to libXrender, but since I don't
> use any Xrender symbols, it's extra baggage. The linker will still resolve all
> the symbols at build time because libXrender is marked with a a DT_NEEDED tag
> in libcairo.
I Don't use cairo, so forgive me if I say weird things.
If you really do not use libXrender symbol while using cairo, why just not include cairo-xlib-xrender.h which is the only one to declare X11/extensions/Xrender.h. In the majority of the cases, cairo.pc should have Requires: xrender >= 0.6 IMO.
Now if linking with xrender is only optional when using cairo, then it should be handled in another .pc or if cairo apps aren't expecting to directly links xrender at all, then it should be best to have X11/extensions/Xrender.h not included at all from the cairo-devel (so it will not be exposed from the cairo API).

> $ readelf -d /usr/lib/libcairo.so | grep NEEDED
The internal dependency of cairo itself is not in question.
...
> So, there's no runtime lookup of undefined symbols, I just want link my app
As you have defined your fictional app not to use symbol from xrender, it is necessary true. But doesn't help with some more info either.

> with "gcc -lcairo -o myapp myapp.c" instead of "gcc -lcairo -lXrender -lX11
> ...". On the other hand, static linking myapp would require that I pull in all
> the libraries to resolve all symbols.
What if an application (compiled gcc -lcairo -o myapp myapp.c ) using cairo and xrender symbol is provided on a system where cario have been compiled without xrender and the xrender library isn't available ?

> Regardless of how I want to link cairo, I need to be able to find the headers
> for fontconfig, Xrender and X11 since the datatypes are exposed by cairo.
So why they aren't in Requires: instead of Requires.private ?
OK , there is the Overlink problem , so another workaround would be to have @LIBX11_CFLAGS@ @LIBX11_CFLAGS@ @LIBX11_CFLAGS@ added in cairo.pc.in at the Clags libs.Then the cairo maintainer would have to requires the related -devel package manually.

But in the majority of the cases, Cflags and Libs from Requires.private doesn't matches. When building your application against cario shared/static, you don't need the -I/usr/include/libpng12 for example. while you will need the -lpng while your binary will be linked statically.
This is usually the case since whenever it is possible, internal dependencies should be hidden from the API.

> That's the extremely common case where one library uses another library's
> datatypes and symbols. It does have something to do with static linking, but
> it's all part of not acquiring more DT_NEEDED libs then necessary.
A agree that there is a need to reduce linking whenever it is possible.
> http://wiki.mandriva.com/en/Overlinking

> As I said before, the case you're describing with gconf and dbus is different
> and is not handled really well by pkg-config since it expects that you will
> need to access the dbus headers when compiling. Maybe there should be another
> pkg-config field like Requires.privatelib that does not expose the CFLAGS.
To sum up, Requires.private seems to be used when: 
- compiling staticaly for the libs
- compiling shared/static for the include pathes.
This sound like a miss design for me.

Revision history for this message

In Red Hat Bugzilla #224148, Dan (dan-redhat-bugs) wrote on 2009-02-19:

#61

If you think pkg-config is not designed correctly, please bring the discussion to the pkg-config list. Or look in the pkg-config archives since there have been many discussions on Requires.private.

http://lists.freedesktop.org/mailman/listinfo/pkg-config

The fact of the matter is that fedora can't just change the behavior of the tool because it's inconvenient. Although there is no formal specification, what I've described is the expected and stated behavior of Requires.private. It can't mean one thing on debian and another thing on fedora. That makes the tool useless.

Revision history for this message

In Red Hat Bugzilla #224148, Mamoru (mamoru-redhat-bugs) wrote on 2009-02-19:

#62

(In reply to comment #28)
> The fact of the matter is that fedora can't just change the behavior of the
> tool because it's inconvenient. Although there is no formal specification, what
> I've described is the expected and stated behavior of Requires.private. It
> can't mean one thing on debian and another thing on fedora. That makes the tool
> useless.

This is not correct. In many applications Fedora already adds
its own implementation.

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2009-02-20:

#63

You can continue discussing this here. But if you want to see any behaviour changes, the discussion needs to happen on the pkg-config list upstream.
Fedora is not going to deviate from upstream behaviour for basic development infrastructure.

Revision history for this message

In Red Hat Bugzilla #224148, Nicolas (nicolas-redhat-bugs) wrote on 2009-02-20:

#64

This seems to be the nearest upstream bug-report.
http://bugs.freedesktop.org/show_bug.cgi?id=4738
I'm still searching for historic and previous reports/talks.

I wonder if an acceptable workaround wouldn't be to have pkg-config --cflags foo to remain quiet if there is missing .pc from Requires.private.

The more correct Fix would be to sort out
Requires.Cflags from Requires.private, because the same info is taken from Requires.private, but for different usage.
I understand the Overlinking concern here, But while trying to fix it, the problem has just moved elsewhere. (with Over requiring dependencies somehow).

Revision history for this message

In Red Hat Bugzilla #224148, Nicolas (nicolas-redhat-bugs) wrote on 2009-02-20:

#65

This bug is re-open (agreed with mclasen) with the aim for the problem to be solved with upstream.

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2009-02-20:

#66

Fwiw, I did no such thing (agree to reopen this). I just tired to fight the close/reopen game with you.

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2009-02-21:

#67

How very unfriendly of you to take this to FESCo without even informing me.

Revision history for this message

Jan Niklas Hasse (jhasse) wrote on 2009-03-02:

#25

Binary package hint: libxxf86vm-dev

When you install libxxf86vm-dev and don't have libxext-dev installed you'll get the following error message when trying to use the .pc file:

jhasse@hardy:~$ pkg-config xxf86vm --cflags --libs
Package xext was not found in the pkg-config search path.
Perhaps you should add the directory containing `xext.pc'
to the PKG_CONFIG_PATH environment variable
Package 'xext', required by 'Xxf86vm', not found
jhasse@hardy:~$

Revision history for this message

Bryce Harrington (bryce) wrote on 2009-03-05:

#26

Hi jhasse,

Please attach the output of `lspci -vvnn`, and attach your /var/log/Xorg.0.log (and maybe Xorg.0.log.old) file from after reproducing this issue. If you've made any customizations to your /etc/X11/xorg.conf please attach that as well.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

Changed in libxxf86vm:
status:	New → Incomplete

Revision history for this message

Jan Niklas Hasse (jhasse) wrote on 2009-03-06:

#27

I don't think these log files are needed, are they?

Revision history for this message

In Red Hat Bugzilla #224148, Christian (christian-redhat-bugs) wrote on 2009-03-06:

#68

I've just stumbled over the problem during re-compiling the current compiz package (F10).

I've just fixed it for myself by adding libXres-devel as BR to the spec file.
However, I'm not sure whether that's the correct way dealing with the problem.

Is there already an agreement what would be the best way to fix build problems in existing packages?

Revision history for this message

In Red Hat Bugzilla #224148, Bug (bug-redhat-bugs) wrote on 2009-06-09:

#69

This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Revision history for this message

Jan Niklas Hasse (jhasse) wrote on 2009-08-30:

#28

This bug isn't fixed yet. Can someone please reopen it?

Bryce Harrington (bryce) on 2009-09-02

tags:

added: hardy

Revision history for this message

Jan Niklas Hasse (jhasse) wrote on 2009-10-17:

#29

This still hasen't been fixed in karmic. Please reopen.

Martin Mai (mrkanister-deactivatedaccount-deactivatedaccount) on 2009-11-18

affects:	libxxf86vm (Ubuntu) → pkg-config (Ubuntu)
Changed in pkg-config (Ubuntu):
status:	Incomplete → Triaged
importance:	Undecided → Low

Revision history for this message

Jan Niklas Hasse (jhasse) wrote on 2009-11-18:

#30

I still think that libxext-dev should be added as a dependency to the libxxf86vm-dev package until pkg-config fixes the issue (which will probably take years).

tags:

added: karmic
removed: needs-lspci-vvnn needs-xorglog

Bug Watch Updater (bug-watch-updater) on 2009-11-18

Changed in pkg-config (Fedora):
status:	Unknown → In Progress

Bug Watch Updater (bug-watch-updater) on 2009-11-19

Changed in pkg-config:
status:	Unknown → Confirmed

Bug Watch Updater (bug-watch-updater) on 2009-11-19

Changed in pkg-config (Debian):
status:	Unknown → Fix Released

Revision history for this message

In Red Hat Bugzilla #224148, Rex (rex-redhat-bugs) wrote on 2009-12-14:

#70

I agree with comment #30 whole-heartedly , and since fedora is shipping upstream setting here and there is an upstream list and bug to track this now, the issue is resolved->upstream (as far as I'm concerned, the original reporter).

Revision history for this message

In Red Hat Bugzilla #224148, Matthias (matthias-redhat-bugs) wrote on 2009-12-14:

#71

A new pkgconfig release is imminent, which should finally put this issue to rest.

Revision history for this message

In Red Hat Bugzilla #224148, Rex (rex-redhat-bugs) wrote on 2009-12-14:

#72

amen! :)

Bug Watch Updater (bug-watch-updater) on 2009-12-15

Changed in pkg-config (Fedora):
status:	In Progress → Fix Released

Bug Watch Updater (bug-watch-updater) on 2010-09-13

Changed in pkg-config:
importance:	Unknown → Medium

Bug Watch Updater (bug-watch-updater) on 2011-01-25

Changed in pkg-config:
importance:	Medium → Unknown

Bug Watch Updater (bug-watch-updater) on 2011-02-04

Changed in pkg-config:
importance:	Unknown → Medium

Revision history for this message

In freedesktop.org Bugzilla #4738, Jan Niklas Hasse (jhasse) wrote on 2013-11-26:

#31

Any news on this?

Revision history for this message

In freedesktop.org Bugzilla #4738, Dbn-lists (dbn-lists) wrote on 2013-12-14:

#32

The way I see it is that this can't be fixed in the current fields without breaking the expected behavior. So, my plan is to get a version 1 spec out that describes the exact semantics pkg-config has now. Then, I plan on introducing variants for Requires - module(Cflags) and module(Libs). This would allow you to narrow what you want to gather on module collection. To fix this bug, by specifying "Requires.private: foo(Libs)", pkg-config would be allowed to completely drop the need for foo.pc in the typical shared case because you've informed it that you do not need the Cflags.

Unfortunately, it's been a busy year for me and I haven't had a ton of time to work on pkg-config. I did prototype the above and it worked well.

Bug Watch Updater (bug-watch-updater) on 2017-10-27

Changed in pkg-config (Fedora):
importance:	Unknown → High
status:	Fix Released → Won't Fix

Revision history for this message

In freedesktop.org Bugzilla #4738, Simon McVittie (smcv) wrote on 2018-03-18:

#73

pkg-config

libxxf86vm-dev should depend on libxext-dev

Bug Description

Duplicates of this bug

Other bug subscribers

Remote bug watches

	Status	Importance	Assigned to
pkg-config	Unknown	Medium	freedesktop-bugs #4738
pkg-config (Debian)	Fix Released	Unknown	debbugs #340904
pkg-config (Fedora)	Won't Fix	High	redhat-bugs #224148
pkg-config (Ubuntu)	Triaged	Low	Unassigned