packetary

Incorrect rpm packages dependency resolution

Bug #1578224 reported by Sergey Kulanov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
packetary
Fix Released
Critical
Sergey Kulanov
packetary 0.2.0

Bug Description

Incorrect package resolution:

packetary --threads-num 10 clone \
  -t rpm \
  -r /home/skulanov/projects/fuel-main/build/centos_mirror.yaml \
  -R /home/skulanov/projects/fuel-main/yaml/packages.yaml \
  -d /home/skulanov/projects/fuel-main/build/iso/isoroot
detected 158 mandatory packages.
loaded: 1209 packages from http://localhost:8000/centos//x86_64/.
detected 158 mandatory packages.
loaded: 1209 packages from http://localhost:8000/centos//x86_64/.
loaded: 1676 packages from http://osci-mirror-kha.kha.mirantis.net/pkgs/centos-2016-05-22-170053//updates/x86_64/.
detected 170 mandatory packages.
loaded: 9007 packages from http://osci-mirror-kha.kha.mirantis.net/pkgs/centos-2016-05-22-170053//os/x86_64/.
loaded: 301 packages from http://osci-mirror-kha.kha.mirantis.net/pkgs/centos-2016-05-22-170053//extras/x86_64/.

Unresolved relation: python-positional (>= 0:1.0.1) from python-keystoneauth1
Unresolved relation: python-debtcollector (>= 0:1.2.0) from python-oslo-log
Unresolved relation: python-hacking (>= 0:0.10.2) from python-oslo-log
Unresolved relation: python-debtcollector (>= 0:1.2.0) from python-oslo-utils
Unresolved relation: python-hacking (>= 0:0.10.2) from python-oslo-messaging
Unresolved relation: python-debtcollector (>= 0:1.2.0) from python-oslo-messaging
Unresolved relation: python-hacking (>= 0:0.10.2) from python-oslo-i18n
Unresolved relation: python-debtcollector (>= 0:1.2.0) from python2-oslo-config
Unresolved relation: python-openstacksdk (>= 0:0.7.4) from python-openstackclient
Unresolved relation: python-yaql (>= 0:1.1.0) from python-muranoclient
Unresolved relation: python-debtcollector (>= 0:1.2.0) from python-oslo-cache
Unresolved relation: python-hacking (>= 0:0.10.2) from python-oslo-cache
cloning repository 'http://osci-mirror-kha.kha.mirantis.net/pkgs/centos-2016-05-22-170053//updates/x86_64/' to '/home/skulanov/projects/fuel-main/build/iso/isoroot'

centos_mirror.yaml
- name: "mos-repos"
  uri: "http://localhost:8000/centos//x86_64"
  priority: 10
  path: "/home/skulanov/projects/fuel-main/build/iso/isoroot/mos-centos"

- name: "upstream-os"
  uri: "http://osci-mirror-kha.kha.mirantis.net/pkgs/centos-2016-05-22-170053//os/x86_64"
  priority: 90
  path: "/home/skulanov/projects/fuel-main/build/iso/isoroot"
- name: "upstream-updates"
  uri: "http://osci-mirror-kha.kha.mirantis.net/pkgs/centos-2016-05-22-170053//updates/x86_64"
  priority: 10
  path: "/home/skulanov/projects/fuel-main/build/iso/isoroot"
- name: "upstream-extras"
  uri: "http://osci-mirror-kha.kha.mirantis.net/pkgs/centos-2016-05-22-170053//extras/x86_64"
  priority: 90
  path: "/home/skulanov/projects/fuel-main/build/iso/isoroot"
- name: "mos-repos"
  uri: "http://localhost:8000/centos//x86_64"
  priority: 1
  path: "/home/skulanov/projects/fuel-main/build/iso/isoroot/mos-centos"

1)
yum deplist python-keystoneauth1
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
 * base: mirror.logol.ru
 * extras: mirror.logol.ru
 * updates: mirror.logol.ru
package: python-keystoneauth1.noarch 2.2.0-1.el7~mos13
  dependency: python(abi) = 2.7
   provider: python.x86_64 2.7.5-34.el7
  dependency: python-argparse
   provider: python.x86_64 2.7.5-34.el7
  dependency: python-iso8601 >= 0.1.9
   provider: python-iso8601.noarch 0.1.10-1.el7~mos1
  dependency: python-oslo-config
   provider: python2-oslo-config.noarch 2:3.9.0-2.el7~mos1
  dependency: python-positional >= 1.0.1
   provider: python2-positional.noarch 1.0.1-3.el7~mos1
  dependency: python-requests >= 2.8.1
   provider: python-requests.noarch 2.8.1-1.el7
  dependency: python-six >= 1.9.0
   provider: python-six.noarch 1.10.0-2.el7~mos1
  dependency: python-stevedore >= 1.5.0
   provider: python-stevedore.noarch 1.9.0-1.el7~mos1

2)
[root@4b4adbdfb436 /]# yum deplist python-oslo-log
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
 * base: mirror.logol.ru
 * extras: mirror.logol.ru
 * updates: mirror.logol.ru
package: python-oslo-log.noarch 3.2.0-1.el7~mos11
  dependency: python(abi) = 2.7
   provider: python.x86_64 2.7.5-34.el7
  dependency: python-babel >= 1.3
   provider: python-babel.noarch 1.3-6.el7~mos1
  dependency: python-dateutil >= 2.4.2
   provider: python-dateutil.noarch 1:2.4.2-3.el7~mos1
  dependency: python-debtcollector >= 1.2.0
   provider: python2-debtcollector.noarch 1.2.0-1.el7~mos1
  dependency: python-hacking >= 0.10.2
   provider: python2-hacking.noarch 0.10.2-1.el7~mos1
  dependency: python-iso8601 >= 0.1.9
   provider: python-iso8601.noarch 0.1.10-1.el7~mos1
  dependency: python-oslo-config >= 3.7.0
   provider: python2-oslo-config.noarch 2:3.9.0-2.el7~mos1
  dependency: python-oslo-context >= 0.2.0
   provider: python-oslo-context.noarch 2.2.0-1.el7~mos0
  dependency: python-oslo-i18n >= 2.1.0
   provider: python-oslo-i18n.noarch 3.4.0-2.el7~mos2
  dependency: python-oslo-serialization >= 1.10.0
   provider: python-oslo-serialization.noarch 2.4.0-1.el7~mos2
  dependency: python-oslo-utils >= 3.5.0
   provider: python-oslo-utils.noarch 3.7.0-1.el7~mos6
  dependency: python-pbr >= 1.6
   provider: python-pbr.noarch 1.8.0-1.el7~mos4
  dependency: python-setuptools
   provider: python-setuptools.noarch 17.1.1-3.el7~mos1
  dependency: python-six >= 1.9.0
   provider: python-six.noarch 1.10.0-2.el7~mos1

See original description
Sergey Kulanov (skulanov)
Changed in packetary:
importance: Undecided → Critical
milestone: none → 0.2.0
Revision history for this message
Sergey Kulanov (skulanov) wrote :
description: updated
Revision history for this message
Sergey Kulanov (skulanov) wrote :

Just to simplify the debugging let's take one package:

packetary --debug --threads-num 10 clone \
   -t rpm \
   -r centos_mirror.yaml \
   -R packages.yaml \
   -d /tmp/mirrors/

centos_mirror.yaml:

- name: "mos-repos"
  uri: "http://localhost:8000/centos//x86_64"
  priority: 1
  path: "/tmp/mirrors/mos-centos"

- name: "upstream-os"
  uri: "http://osci-mirror-kha.kha.mirantis.net/pkgs/centos-2016-05-22-170053//os/x86_64"
  priority: 90
  path: "/tmp/mirrors/centos"
- name: "upstream-updates"
  uri: "http://osci-mirror-kha.kha.mirantis.net/pkgs/centos-2016-05-22-170053//updates/x86_64"
  priority: 10
  path: "/tmp/mirrors/centos"
- name: "upstream-extras"
  uri: "http://osci-mirror-kha.kha.mirantis.net/pkgs/centos-2016-05-22-170053//extras/x86_64"
  priority: 90
  path: "/tmp/mirrors/centos"
- name: "mos-repos"
  uri: "http://localhost:8000/centos//x86_64"
  priority: 1
  path: "/tmp/mirrors/mos-centos"

packages.yaml:

packages:
  - name: python2-oslo-config

# repositories:
# - name: mos-repos
# excludes:
# - name: "/^.*debuginfo.*/"

Result:

Unresolved relation: python-debtcollector (>= 0:1.2.0) from python2-oslo-config

So it seems we have some issues in logic:

Information from spec:
Provides: python-debtcollector = %{version}-%{release}
Obsoletes: python-debtcollector < 0.7.0-3

From package meta information:
$] cat OBSOLETES
python-debtcollector < 1.2.0-1.el7~mos1
python-debtcollector < 0.7.0-3

$] cat PROVIDES
python-debtcollector = 1.2.0-1.el7~mos1
python2-debtcollector = 1.2.0-1.el7~mos1

So rpm package can either provide or obsolete packages, and this this logic [1] doesn't take it into account

[1]. https://github.com/openstack/packetary/blob/master/packetary/objects/packages_tree.py#L63-L80

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to packetary (master)

Fix proposed to branch: master
Review: https://review.openstack.org/321004

Changed in packetary:
assignee: nobody → Sergey Kulanov (skulanov)
status: New → In Progress
Revision history for this message
Sergey Kulanov (skulanov) wrote :

This comes from %python_provide macros, eg:
# rpm --eval '%python_provide python2-foo'
Provides: python-foo(x86-64) = %{version}-%{release}
Provides: python-foo = %{version}-%{release}
Obsoletes: python-foo(x86-64) < %{version}-%{release}
Obsoletes: python-foo < %{version}-%{release}

# rpm --eval '%python_provide python-foo'
Provides: python2-foo = %{version}-%{release}

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to packetary (master)

Reviewed: https://review.openstack.org/321004
Committed: https://git.openstack.org/cgit/openstack/packetary/commit/?id=ff85847eca9a689e7925cc72786b944b74effe53
Submitter: Jenkins
Branch: master

commit ff85847eca9a689e7925cc72786b944b74effe53
Author: Sergey Kulanov <email address hidden>
Date: Wed May 25 16:28:45 2016 +0300

    Fix rpm package provides/obsoletes relations

    Rpm package can provides and obsoletes the same package name
    but with different versions, for eg:
      OBSOLETES:
         python-debtcollector < 1.2.0-1.el7~mos1
         python-debtcollector < 0.7.0-3

      PROVIDES:
         python-debtcollector = 1.2.0-1.el7~mos1
         python2-debtcollector = 1.2.0-1.el7~mos1

    This comes from %python_provide macros, eg:
        # rpm --eval '%python_provide python2-foo'
           Provides: python-foo(x86-64) = %{version}-%{release}
           Provides: python-foo = %{version}-%{release}
           Obsoletes: python-foo(x86-64) < %{version}-%{release}
           Obsoletes: python-foo < %{version}-%{release}

        # rpm --eval '%python_provide python-foo'
           Provides: python2-foo = %{version}-%{release}

    So we need to align flow

    Change-Id: I07b3d8b9a25c5c98e97e697c26e6ae9d1f4960ff
    Closes-bug: #1578224

Changed in packetary:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.