No way to pull requirements from source in devstack jobs

[Boden R]

Problem:
There's no proper way to install "siblings" [1] in devstack based zuul
v3 jobs for projects that also require the siblings via requirements.txt.

Background:
Following the zuul v3 migration guide for "sibling requirements" [1]
works fine for non-devstack based jobs. However, jobs that use devstack
must take other measures to install those siblings in their playbooks.

Based on what I see projects like oslo.messaging doing for cross
testing, they are using the PROJECTS env var to specify the siblings in
their playbook (example [2]). This approach may work if those siblings
are not in requirements.txt, but for projects that also require the
siblings at runtime (in requirements.txt) it appears the version from
the requirements.txt is used rather than the sibling's source.

For example the changes in [3][4].

Also see [5][6].

[1]
https://docs.openstack.org/infra/manual/zuulv3.html#installation-of-sibling-requirements
[2]
https://github.com/openstack/oslo.messaging/blob/master/playbooks/oslo.messaging-telemetry-dsvm-integration-amqp1/run.yaml#L37
[3] https://review.openstack.org/#/c/638099
[4]
http://logs.openstack.org/99/638099/6/check/tricircle-functional/0b34687/logs/devstacklog.txt.gz#_2019-02-21_14_57_44_553
[5] http://lists.openstack.org/pipermail/openstack-discuss/2019-February/003010.html
[6] https://bugs.launchpad.net/tricircle/+bug/1816644

[Ghanshyam Mann (ghanshyammann)]

I do not think LIBS_FROM_GIT and requirement.txt are two conflict entity. repo
mentioned in LIBS_FROM_GIT will be checked against the requirement.txt version and
they should satisfy with the latest master version of repo mentioned in LIBS_FROM_GIT.

For example, in your case neutron is mentioned in LIBS_FROM_GIT so devstack will
pickup the neutron master version which should be compatible with the requirement.txt (>=neutron-released-version)

I saw in your patch - https://review.openstack.org/#/c/638099/ (taking the example of PS7), neutron is installed form master[1] and it did satisfy the
requirement.txt version [5]. So the final installed version of neutron is 14.0.0.0b2.dev243 which is the latest master.

similar case with required_proj2ct which end up appending in LIBS_FROM_GIT by devstack so all repo
mentioned in required_projects are installed from source until installing project has explicitly constrained
them by upper_constarinted etc.

The problem I see in your patch is networking-sfc latest version is not picked up even that is installed from source.
i found networking-sfc-7.0.0 has neutron.db.api imported which has been changed to neutron_lib.db.api in networking-sfc-8.0.0
 In the failure, networking-sfc-7.0.0 is being picked up[3] instead of networking-sfc-8.0.0 which fail with the latest neutron 14.0.0.0b2.dev243.

All other PS8-11 [4], I cannot find the networking-sfc installed in that job and so does no error. I am not sure how that is passing without networking-sfc.

But in term of installation, devstack pickup the source version and then apply constraint according to what installation repo has.

[1] http://logs.openstack.org/99/638099/7/check/tricircle-functional/5b20269/logs/devstacklog.txt.gz#_2019-02-21_19_23_35_571
[2] http://logs.openstack.org/99/638099/7/check/tricircle-functional/5b20269/logs/devstacklog.txt.gz#_2019-02-21_19_23_31_906
[3] http://logs.openstack.org/99/638099/7/check/tricircle-functional/5b20269/logs/devstacklog.txt.gz#_2019-02-21_19_23_39_418
[4] http://logs.openstack.org/99/638099/8/check/tricircle-functional/92841bb/logs/devstacklog.txt.gz

[Ghanshyam Mann (ghanshyammann)]

Even I checked the master gate job on networking-sfc side and the latest networking-sfc version installed there is 7.1.0.dev45.
which confirm that networking-sfc and neutron version in tricircle job is from source. There is some issue on networking-sfc side.

- http://logs.openstack.org/52/637852/1/check/networking-sfc-tempest-dsvm/2aa75a5/job-output.txt.gz#_2019-02-19_14_43_25_098614

[Boden R (boden)]

In regards to networking-sfc...
Based on [1] it looks like networking-sfc 8.0.0.0b1 was just released a few days ago.
The job results in comment #2 are a bit old; before 8.0.0.0b1 was released. At that time I believe 7.1.0.dev45 was the latest.

In recent SFC jobs it looks like 8.0.0.0b1 is used [2]. That said I'm not sure there's an issue here is there?

In regards to the devstack behavior...
In tricircle PS7 [3] you referenced, the version of sfc in tricircle's requirements is networking-sfc>=6.0.0 and sfc is in the LIBS_FROM_GIT. If I understand, this means sfc should be installed from source, and that shouldn't conflict with requirements. However in this case a 'dev' (source) version of sfc is not installed [4] and instead networking-sfc-7.0.0 is used.
This seems in contradiction as to what was said in comment #1; it doesn't seem like sfc is in fact being pulled from source. But maybe I'm not understanding this completely yet.

[1] https://github.com/openstack/releases/blob/master/deliverables/stein/networking-sfc.yaml
[2] http://logs.openstack.org/55/638255/1/gate/networking-sfc-tempest-dsvm/1972af2/job-output.txt.gz#_2019-02-26_22_12_56_627059
[3] https://review.openstack.org/#/c/638099/7
[4] http://logs.openstack.org/99/638099/7/check/tricircle-functional/5b20269/logs/devstacklog.txt.gz#_2019-02-21_19_23_39_418

[Boden R (boden)]

Here is another current example that shows devstack uses the requirements and not LIBS_FROM_GIT (the master source).

If we look at PS1 of [1] we can see the requirements specify networking-sfc>=8.0.0.0b1 [2] and we also add it to LIBS_FROM_GIT in the playbook [3].

Based on my understanding of comment #1 above, devstack should be using git master for networking-sfc since its constraint of >=8.0.0.0b1 doesn't conflict with the git/master version of sfc (I think that would be some "dev" version of networking-sfc).
Instead, networking-sfc version 8.0.0.0b1 is used [4] which is a tagged/released beta version of sfc [5] and not the git/master source code.

What this bug is about is trying to use the git/master source for required projects within devstack so I'm still missing how this can be done??

[1] https://review.openstack.org/#/c/639662/
[2] https://review.openstack.org/#/c/639662/1/requirements.txt
[3] https://github.com/openstack/tricircle/blob/master/playbooks/tricircle-dsvm-functional/run.yaml#L46
[4] http://logs.openstack.org/62/639662/1/check/tricircle-functional/0e306b2/logs/devstacklog.txt.gz#_2019-02-27_14_07_13_879
[5] https://github.com/openstack/networking-sfc/tree/8.0.0.0b1

[Ghanshyam Mann (ghanshyammann)]

I agree with networking-sfc case that its dev version is not used. I am not sure why that happened.

But if you see the neutron case, that is taken from source and its dev version is installed. I suspect issue you are seeing is only with networking-sfc.

Devstack installs and honours the required_projects or LIBS_FROM_GIT the same way for neutron and networking-sfc. Till it finds the constraint are all ok then it will install all repo mentioned in LIBS_FROM_GIT from their source.

[Dr. Jens Harbott (j-harbott)]

Sadly all the logs have expired, please attach fresh logs if this is still an issue.

[Launchpad Janitor (janitor)]

[Expired for devstack because there has been no activity for 60 days.]