[fuel-qa] building of bootstrap image is failed in 9.x

[Nikita Karpin]

Detailed bug description:

When using repositories generated by https://github.com/openstack/fuel-qa/blob/stable/mitaka/utils/jenkins/conv_snapshot_file.py bootstrap image is failed to be built, because of incorrect URL for mos-proposed repo for Ubuntu.

Steps to reproduce:

export ENABLE_MOS_UBUNTU_PROPOSED=true
export ENABLE_MOS_UBUNTU_UPDATES=true
export ENABLE_MOS_UBUNTU_SECURITY=true
export ENABLE_MOS_UBUNTU_HOLDBACK=true
export ENABLE_MOS_CENTOS_OS=true
export ENABLE_MOS_CENTOS_PROPOSED=true
export ENABLE_MOS_CENTOS_UPDATES=true
export ENABLE_MOS_CENTOS_SECURITY=true
export ENABLE_MOS_CENTOS_HOLDBACK=true

wget https://product-ci.infra.mirantis.net/job/9.x.snapshot/lastSuccessfulBuild/artifact/snapshots.params

./utils/jenkins/conv_snapshot_file.py

source extra_repos.sh

export EXTRA_DEB_REPOS
export EXTRA_RPM_REPOS
export UPDATE_FUEL_MIRROR
export UPDATE_MASTER

And run the system test

Expected results:

tests pass

Actual result:

Boostrap image is not built because of these errors in the log:

Starting new HTTP connection (1): mirror.seed-cz1.fuel-infra.org
"GET /mos-repos/ubuntu/snapshots/9.0-2016-08-12-160322/x86_64/dists/mos9.0-proposed/Release HTTP/1.1" 404 None
Got non-critical error when accessing to http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-08-12-160322/x86_64/dists/mos9.0-proposed/Release on 29 attempt: 404 Client Error: Not Found for url: http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-08-12-160322/x86_64/dists/mos9.0-proposed/Release
Starting new HTTP connection (1): mirror.seed-cz1.fuel-infra.org
"GET /mos-repos/ubuntu/snapshots/9.0-2016-08-12-160322/x86_64/dists/mos9.0-proposed/Release HTTP/1.1" 404 None
Got non-critical error when accessing to http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-08-12-160322/x86_64/dists/mos9.0-proposed/Release on 30 attempt: 404 Client Error: Not Found for url: http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-08-12-160322/x86_64/dists/mos9.0-proposed/Release
Starting new HTTP connection (1): mirror.seed-cz1.fuel-infra.org
"GET /mos-repos/ubuntu/snapshots/9.0-2016-08-12-160322/x86_64/dists/mos9.0-proposed/Release HTTP/1.1" 404 None
Got non-critical error when accessing to http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-08-12-160322/x86_64/dists/mos9.0-proposed/Release on 31 attempt: 404 Client Error: Not Found for url: http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-08-12-160322/x86_64/dists/mos9.0-proposed/Release
Failed to build bootstrap image: Exceeded maximum http request retries for %s

got an error: Exceeded maximum http request retries for %s
Traceback (most recent call last):
  File "/bin/fuel-bootstrap", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/fuel_bootstrap/main.py", line 54, in main
    return fuel_bootstrap_app.run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 255, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 374, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 54, in run
    self.take_action(parsed_args)
  File "/usr/lib/python2.7/site-packages/fuel_bootstrap/commands/build.py", line 177, in take_action
    image_uuid, path = bs_image.make_bootstrap(vars(parsed_args))
  File "/usr/lib/python2.7/site-packages/fuel_bootstrap/utils/bootstrap_image.py", line 174, in make_bootstrap
    mngr.do_mkbootstrap()
  File "/usr/lib/python2.7/site-packages/fuel_agent/manager.py", line 1020, in do_mkbootstrap
    direct_repo_addrs=driver_os.proxies.direct_repo_addr_list)
  File "/usr/lib/python2.7/site-packages/fuel_agent/manager.py", line 616, in _set_apt_repos
    proxies=proxies, direct_repo_addrs=direct_repo_addrs)
  File "/usr/lib/python2.7/site-packages/fuel_agent/utils/build.py", line 469, in add_apt_preference
    direct_repo_addrs=direct_repo_addrs))
  File "/usr/lib/python2.7/site-packages/fuel_agent/utils/build.py", line 394, in get_release_file
    noproxy_addrs=direct_repo_addrs).text
  File "/usr/lib/python2.7/site-packages/fuel_agent/utils/utils.py", line 266, in init_http_request
    "Exceeded maximum http request retries for %s".format(url))
fuel_agent.errors.HttpUrlConnectionError: Exceeded maximum http request retries for %s

In browser i also get 404 when trying to access this URL - http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-08-12-160322/x86_64/dists/mos9.0-proposed/Release
However this one exists - http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-08-12-160322/dists/mos9.0-proposed/Release

I checked and found that this URL is hardcoded here - https://github.com/openstack/fuel-qa/blob/stable/mitaka/utils/jenkins/conv_snapshot_file.py#L124-L125

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/355395

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/355396

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/355395
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=6207b38182ee3ec47d32ed42e9f19ce955aedadd
Submitter: Jenkins
Branch: master

commit 6207b38182ee3ec47d32ed42e9f19ce955aedadd
Author: Ivan Remizov <email address hidden>
Date: Mon Aug 15 12:46:02 2016 +0300

    Fix incorrect url for deb repo

    This patch fixes incorrect suffix for deb repo url which
    was introduced by copy paste.

    Change-Id: Icb41c5d9e00c588e9535f4fe96c0690bb2413f3c
    Closes-Bug: #1613210

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/355396
Committed: https://git.openstack.org/cgit/openstack/fuel-qa/commit/?id=388fbb5a8f84294496e1a98aedf3b9f148703917
Submitter: Jenkins
Branch: stable/mitaka

commit 388fbb5a8f84294496e1a98aedf3b9f148703917
Author: Ivan Remizov <email address hidden>
Date: Mon Aug 15 12:46:02 2016 +0300

    Fix incorrect url for deb repo

    This patch fixes incorrect suffix for deb repo url which
    was introduced by copy paste.

    Change-Id: Icb41c5d9e00c588e9535f4fe96c0690bb2413f3c
    Closes-Bug: #1613210

[TatyanaGladysheva (tgladysheva)]

Verified on 9.1 snapshot #234.