fuel-package-updates works incorrectly with non-default set of repositories

[Dennis Dmitriev]

If the default repository list was changed in Nailgun, 'fuel-package-updates' changes link for only the first (upstream) Ubuntu repository. All additional upstream and mos repositories are left unchanged (see attached screenshot).

Example:

[root@nailgun]# fuel-package-updates -d ubuntu-baseos -r 2014.2.2-6.1 --no-download --make-default --apply -s 10.109.30.2 -p admin -b http://10.109.30.2:8080/ubuntu-part -e 2 --clear-upstream-repos
2015-05-22 15:18:22,361 - INFO - Skipping repository download...
2015-05-22 15:18:22,363 - INFO - Updating environment repositories...
2015-05-22 15:18:22,363 - INFO - Updating default repositories...
2015-05-22 15:18:22,369 - WARNING - Authorization failure: Authentication required
[root@nailgun]#

Result is on the screenshot.

[Alex Schultz (alex-schultz)]

So in looking into how this could be accomplished, I don't think it's correct to remove customized ubuntu repos. It currently works correctly if you do not add additional repositories for the ubuntu repos. Did the ubuntu-{0,1,2,3} repos get added via a native fuel process meaning we should handle for the name matches for ubuntu-{0,1,2,3}?

[Alex Schultz (alex-schultz)]

Also by using '-d ubuntu-baseos', it will only update the ubuntu repo and does not touch the mos urls. So you would need to run that command with '-d ubuntu' and then again with '-d ubutu-baseos' to have both the os and mos repos updated.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/185689

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/185689
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=b8ce1824aecaa60be186e26e0c61e4cdb97edbde
Submitter: Jenkins
Branch: master

commit b8ce1824aecaa60be186e26e0c61e4cdb97edbde
Author: Alex Schultz <email address hidden>
Date: Tue May 26 13:16:19 2015 -0500

    Add warning about customized ubuntu mirrors

    If a user adds custom ubuntu upstream mirrors to their environment, the
    --clear-upstream-repos option does not automatically remove them from
    the environment. This commit adds a warning to indicate that user
    intervention may be required.

    Change-Id: Iafc745cc53768f5de1765a07e8c5d333f4849c51
    Partial-Bug: 1458597

[Dennis Dmitriev (ddmitriev)]

If we prepare another set of repositories, why don't we just remove all repositories from default and make an another default list of repositories from scratch? Anyway, the new list won't match the default count of repositories.

In this case, it is not required to find some special keywords in the repository properties, so it should be quite trivial.

[Dennis Dmitriev (ddmitriev)]

As there can be some misunderstanding, customer should be noted in the Fuel UI the following issues:

1) The tool 'fuel-createmirror' will add the local repository to a newly created cluster only if the list of Ubuntu repositories in this cluster was unchanged by the customer and was contained only default entries. If customer changed the list of repositories and then desided to run 'fuel-createmirror', it is better to press "Load defaults" and then "Save settings" before starting the 'fuel-createmirror'.

2) When 'fuel-createmirror' finished, customer should reload "Settings" tab, and may modify the list of repositories by his own.

[Alex Schultz (alex-schultz)]

The problem is that the repository list is all of them (ubuntu/mos/aux or centos/mos/aux) but you only operate on one of these sets via the tool as it is currently implemented. --clear-upstream-repos only modifies the ubuntu mirrors so we can't just clear them all at this time. The only affects people who have added/modified the repositories and so removing all their repositories would be bad. For example if I added my own software repo for my company's product and then I ran this but you removed my repository, i would have to re-add it. The warning indicates that if the user has added a repository that it was untouched and so they may need to intervene. The script works as expected for the default set of repositories.

[Dennis Dmitriev (ddmitriev)]

This note should be on Fuel UI to clarify to customer what will happen to the list of repositories, what will be done automatically and what the customer should perform manually.

[Alex Schultz (alex-schultz)]

Sounds like something that should probably be wrapped into the documentation. The review is still open for the docs linked via the UI, https://review.openstack.org/#/c/182226/

[Vitaly Kramskikh (vkramskikh)]

The correct links are already in the UI, so we wait for merge of the docs

[Matthew Mosesohn (raytrac3r)]

This was a known task to fix before I went on holiday, but I will address this in fuel-package-updates. The tasks necessary are:
update repo_merge to add capability such that if b contains a key called "delete", it should not be removed form the resulting list of repos
update get_ubuntu_baseos_repos to generate a set of repos containing the following:
[{name: 'ubuntu', delete: 'true'},
{name: 'ubuntu-security', delete: 'true'},
{name: 'ubuntu-updates', delete: 'true'},
{name: 'trusty': type: 'deb', uri: repourl, suite, repo, section: "main", priority: None}

[Matthew Mosesohn (raytrac3r)]

Correction for #12. It looks like this was merged already https://review.openstack.org/#/c/182691/

I don't think we need any more tasks except updates of docs.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/187244

[Matthew Mosesohn (raytrac3r)]

oops I didn't mean to put this patch to this bug. Moving back.