registry.local hostname not stored in docker proxy configuration

Bug #1831507 reported by Erich Cordoba
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Critical
Yan Chen

Bug Description

Brief Description
-----------------
The registry.local hostname recently introduced is not stored in the docker proxy configuration. This cause a failure in the download of images.

Severity
--------
Major

Steps to Reproduce
------------------
Run system application-apply platform-integ-apps in a proxy environment. The download will fail as the http://registry.local cannot be found.

If the docker proxy configuration is changed to :
/etc/systemd/system/docker.service.d/http-proxy.conf

[Service]
Environment="HTTP_PROXY=http://proxy-chain.intel.com:911"
Environment="HTTPS_PROXY=http://proxy-chain.intel.com:912"
Environment="NO_PROXY=registry.local,localhost,127.0.0.1,192.168.204.2,192.168.204.3,10.10.10.3,10.10.10.4"

Then the download succeeds.

Expected Behavior
------------------
The application-apply command should succeed.

Actual Behavior
----------------
The following errors appears in /var/log/sysinv.log

Image registry.local:9001/quay.io/external_storage/rbd-provisioner:v2.1.1-k8
s1.11 download failed from local registry: 500 Server Error: Internal Server Error ("Get https://registry.local:9001/v2/: read tcp 10.10.10.3:5
8248->10.19.8.224:912: read: connection reset by peer")

Reproducibility
---------------
100% in a proxy environment.

System Configuration
--------------------
All configurations

Branch/Pull Time/Commit
-----------------------
BUILD_ID="20190602T233000Z"

Timestamp/Logs
--------------
2019-06-03 22:11:05.148 101263 ERROR sysinv.conductor.kube_app [-] Image registry.local:9001/quay.io/external_storage/rbd-provisioner:v2.1.1-k8
s1.11 download failed from local registry: 500 Server Error: Internal Server Error ("Get https://registry.local:9001/v2/: read tcp 10.10.10.3:5
8248->10.19.8.224:912: read: connection reset by peer")
2019-06-03 22:11:05.148 101263 ERROR sysinv.conductor.kube_app [-] Deployment of application platform-integ-apps (1.0-6) failed: failed to down
load one or more image(s).
2019-06-03 22:11:05.148 101263 TRACE sysinv.conductor.kube_app Traceback (most recent call last):
2019-06-03 22:11:05.148 101263 TRACE sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 1
212, in perform_app_apply
2019-06-03 22:11:05.148 101263 TRACE sysinv.conductor.kube_app self._download_images(app)
2019-06-03 22:11:05.148 101263 TRACE sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 5
47, in _download_images
2019-06-03 22:11:05.148 101263 TRACE sysinv.conductor.kube_app reason="failed to download one or more image(s).")
2019-06-03 22:11:05.148 101263 TRACE sysinv.conductor.kube_app KubeAppApplyFailure: Deployment of application platform-integ-apps (1.0-6) faile
d: failed to download one or more image(s).
2019-06-03 22:11:05.148 101263 TRACE sysinv.conductor.kube_app

Test Activity
-------------
Sanity

Bruce Jones (brucej)
Changed in starlingx:
importance: Undecided → Critical
assignee: nobody → yong hu (yhu6)
Revision history for this message
Ghada Khalil (gkhalil) wrote :

Marking as release gating; high priority as this is resulting in a red sanity for environments using proxy. Likely a side effect of changes made in https://review.opendev.org/#/c/661849/

tags: added: stx.2.0 stx.containers stx.sanity
Revision history for this message
Matt Peters (mpeters-wrs) wrote :

A workaround to this issue is to specify registry.local as part of the Ansible overrides during deployment.

docker_no_proxy:
- registry.local
- <other ip>
- <other ip>

Revision history for this message
yong hu (yhu6) wrote :

@Yan, as discussed, please make a patch for this critical ASAP.

Changed in starlingx:
assignee: yong hu (yhu6) → nobody
assignee: nobody → Yan Chen (ychen2u)
Revision history for this message
Yan Chen (ychen2u) wrote :
Changed in starlingx:
status: New → Incomplete
status: Incomplete → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to config (master)

Reviewed: https://review.opendev.org/662927
Committed: https://git.openstack.org/cgit/starlingx/config/commit/?id=e2bcaab2b5bed6493fe0694525c13423073f28e2
Submitter: Zuul
Branch: master

commit e2bcaab2b5bed6493fe0694525c13423073f28e2
Author: chenyan <email address hidden>
Date: Tue Jun 4 10:55:16 2019 +0800

    add registry.local as a default no_proxy url.

    Closes-Bug: 1831507

    Change-Id: Ida605ed04412b1b272db825e069c7bab65dd3a12
    Signed-off-by: chenyan <email address hidden>

Changed in starlingx:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.