[9.x][BVT] Building of a bootstrap image fails

[Roman Podoliaka]

Community BVT jobs have been failing since September the 16th on creation of a bootstrap image:

Starting new HTTP connection (1): 127.0.0.1
"GET /ubuntu/x86_64/dists/mos9.1/Release HTTP/1.1" 404 0
Got non-critical error when accessing to http://127.0.0.1:8080/ubuntu/x86_64/dists/mos9.1/Release on 1 attempt: 404 Client Error: Not Found for url: http://127.0.0.1:8080/ub
untu/x86_64/dists/mos9.1/Release
Starting new HTTP connection (1): 127.0.0.1
"GET /ubuntu/x86_64/dists/mos9.1/Release HTTP/1.1" 404 0
Got non-critical error when accessing to http://127.0.0.1:8080/ubuntu/x86_64/dists/mos9.1/Release on 2 attempt: 404 Client Error: Not Found for url: http://127.0.0.1:8080/ub
untu/x86_64/dists/mos9.1/Release
Starting new HTTP connection (1): 127.0.0.1
"GET /ubuntu/x86_64/dists/mos9.1/Release HTTP/1.1" 404 0
Got non-critical error when accessing to http://127.0.0.1:8080/ubuntu/x86_64/dists/mos9.1/Release on 3 attempt: 404 Client Error: Not Found for url: http://127.0.0.1:8080/ub
untu/x86_64/dists/mos9.1/Release

...

got an error: Exceeded maximum http request retries for http://127.0.0.1:8080/ubuntu/x86_64/dists/mos9.1/Release
Traceback (most recent call last):
  File "/bin/fuel-bootstrap", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/fuel_bootstrap/main.py", line 54, in main
    return fuel_bootstrap_app.run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 255, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 374, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 54, in run
    self.take_action(parsed_args)
  File "/usr/lib/python2.7/site-packages/fuel_bootstrap/commands/build.py", line 177, in take_action
    image_uuid, path = bs_image.make_bootstrap(vars(parsed_args))
  File "/usr/lib/python2.7/site-packages/fuel_bootstrap/utils/bootstrap_image.py", line 174, in make_bootstrap
    mngr.do_mkbootstrap()
  File "/usr/lib/python2.7/site-packages/fuel_agent/manager.py", line 1020, in do_mkbootstrap
    direct_repo_addrs=driver_os.proxies.direct_repo_addr_list)
  File "/usr/lib/python2.7/site-packages/fuel_agent/manager.py", line 616, in _set_apt_repos
    proxies=proxies, direct_repo_addrs=direct_repo_addrs)
  File "/usr/lib/python2.7/site-packages/fuel_agent/utils/build.py", line 458, in add_apt_preference
    direct_repo_addrs=direct_repo_addrs))
  File "/usr/lib/python2.7/site-packages/fuel_agent/utils/build.py", line 383, in get_release_file
    noproxy_addrs=direct_repo_addrs).text
  File "/usr/lib/python2.7/site-packages/fuel_agent/utils/utils.py", line 267, in init_http_request
    url=url))
fuel_agent.errors.HttpUrlConnectionError: Exceeded maximum http request retries for http://127.0.0.1:8080/ubuntu/x86_64/dists/mos9.1/Release

Latest fail:

https://ci.fuel-infra.org/job/9.0-community.main.ubuntu.bvt_2/497/

[Maksim Malchuk (mmalchuk)]

Looks like a regression added by: https://review.openstack.org/#/q/I0b04200dc9414fa5f453d62f0f80c7f124f45d8a

[Vladimir Khlyunev (vkhlyunev)]

what? did you opened my patch?

real issue: http://127.0.0.1:8080/ubuntu/x86_64/dists/mos9.1/Release
mos9.1
It is substitution issue because in release we have {cluster.release.version} in several repos in openstack.yaml

[0] https://github.com/openstack/fuel-web/blob/stable/mitaka/nailgun/nailgun/fixtures/openstack.yaml#L1992

[Vladimir Kozhukalov (kozhukalov)]

This bug is not related to nailgun openstack.yaml because it is about building bootstrap image. The rpm repos for bootstrap are defined here https://github.com/openstack/fuel-menu/blob/master/fuelmenu/settings.yaml#L16-L57 where mos_version variable is equal to the content of the /etc/fuel_release file.

This BVT test is run with MOS proposed repository where the package fuel-release contains 9.1 (it must be 9.0). We need to fix the downstream code where 9.1 is introduced and rebuild fuel-release package.

Compare upstream [1] and downstream [2]

[1] https://github.com/openstack/fuel-main/blob/stable/mitaka/specs/fuel-main.spec#L43
[2] https://review.fuel-infra.org/gitweb?p=openstack/fuel-main.git;a=blob;f=specs/fuel-main.spec;h=709ed873384246f8407a6f375c59e7ffc02aef26;hb=refs/heads/mos/mitaka#l44

[Vladimir Kozhukalov (kozhukalov)]

https://review.fuel-infra.org/#/c/26783/

[Dmitry Belyaninov (dbelyaninov)]

Runs #569-#589 are green.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Andreas Jaeger (<email address hidden>) on branch: stable/mitaka
Review: https://review.opendev.org/374871
Reason: This repo is retired now, no further work will get merged.