Timeouts while connecting to mirror.fuel-infra.org and mirror.seed-cz1.fuel-infra.org

Bug #1609770 reported by Timur Sufiev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Pawel Brzozowski

Bug Description

According to excerpts from https://packaging-ci.infra.mirantis.net/job/9.0-pkg-systest-ubuntu/1418/artifact/fuel-qa/logs/fail_error_ceph_rados_gw-fuel-snapshot-2016-08-04_07-29-28.tar.gz.filtered.log there were connectivity issues with http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/:

node-1.test.domain.local 2016-08-04T06:34:08.706883 err: ERROR: Unable to fetch url 'http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/', error 'execution expired'. Please verify node connectivity to this URL, or remove it from the settings page if it is invalid. on node node-1.test.domain.local
node-1.test.domain.local 2016-08-04T06:34:08.707801 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:95:in `rescue in fetch'
node-1.test.domain.local 2016-08-04T06:34:08.707801 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:83:in `fetch'
node-1.test.domain.local 2016-08-04T06:34:08.707801 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:106:in `block (4 levels) in <top (required)>'
node-1.test.domain.local 2016-08-04T06:34:08.710003 err: ERROR: Unable to fetch url 'http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/', error 'execution expired'. Please verify node connectivity to this URL, or remove it from the settings page if it is invalid. on node node-1.test.domain.local
node-1.test.domain.local 2016-08-04T06:34:08.710003 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:95:in `rescue in fetch'
node-1.test.domain.local 2016-08-04T06:34:08.710003 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:83:in `fetch'
node-1.test.domain.local 2016-08-04T06:34:08.710003 err: /etc/puppet/modules/osnailyfacter/lib/puppet/parser/functions/url_available.rb:106:in `block (4 levels) in <top (required)>'

node-3.test.domain.local 2016-08-04T07:14:28.353731 err: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install cinder-scheduler' returned 100: Reading package lists...
node-3.test.domain.local 2016-08-04T07:14:28.353731 err: Building dependency tree...
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: Reading state information...
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: The following packages were automatically installed and are no longer required:
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: cloud-guest-utils eatmydata python-cheetah python-configobj python-oauth
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: python-serial python3-pycurl python3-software-properties
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: software-properties-common unattended-upgrades
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: Use 'apt-get autoremove' to remove them.
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: The following NEW packages will be installed:
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: cinder-scheduler
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: Need to get 11.7 kB of archives.
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: After this operation, 77.8 kB of additional disk space will be used.
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: WARNING: The following packages cannot be authenticated!
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: cinder-scheduler
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: Authentication warning overridden.
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: Err http://mirror.seed-cz1.fuel-infra.org/mos-repos/ubuntu/snapshots/9.0-2016-08-03-144320/ mos9.0-proposed/main cinder-scheduler all 2:8.0.0-5~u14.04+mos72
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: Could not connect to mirror.seed-cz1.fuel-infra.org:80 (5.43.231.47), connection timed out
node-3.test.domain.local 2016-08-04T07:14:28.354232 err: E: Failed to fetch http://mirror.seed-cz1.fuel-infra.org/mos-

Revision history for this message
Timur Sufiev (tsufiev-x) wrote :
summary: - Timeouts while connecting to mirror.fuel-infra.org
+ Timeouts while connecting to mirror.fuel-infra.org and mirror.seed-
+ cz1.fuel-infra.org
Changed in fuel:
assignee: nobody → Fuel Sustaining (fuel-sustaining-team)
milestone: none → 9.1
Timur Sufiev (tsufiev-x)
Changed in fuel:
importance: Undecided → High
tags: added: gating-failure
Dmitry Pyzhov (dpyzhov)
Changed in fuel:
assignee: Fuel Sustaining (fuel-sustaining-team) → Fuel CI (fuel-ci)
Changed in fuel:
status: New → Confirmed
Revision history for this message
Pawel Brzozowski (pbrzozowski) wrote :

Thanks for reporting this Timur!

This morning we have noticed problems with connections between DC's and it was reported to IT:

https://mirantis.jira.com/browse/IT-30404

Resolved with:

--- cut ---
Vladimir Efimov added a comment - 5 hours ago

looks like that the global provider Cogent has a problem with a balancing in Europe network. Now DC turned off this uplink and all works fine over RETN.

— registry03-scc.fuel-infra.org ping statistics —
151 packets transmitted, 150 received, 0% packet loss, time 149962ms

We will try to turn on and check Cogent's uplink next morning.
--- cut ---

Please reopen if it happens again.

Changed in fuel:
status: Confirmed → Fix Released
assignee: Fuel CI (fuel-ci) → Pawel Brzozowski (pbrzozowski)
Revision history for this message
Boris Bobrov (bbobrov) wrote :
Changed in fuel:
status: Fix Released → Confirmed
Revision history for this message
Boris Bobrov (bbobrov) wrote :

Oops, my bad, it has not happened again.

Changed in fuel:
status: Confirmed → Fix Released
Roman Vyalov (r0mikiam)
Changed in fuel:
status: Fix Released → Fix Committed
Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.