Network verification fails on bare-metal: non-admin interfaces are down

Bug #1529985 reported by Artem Panchenko on 2015-12-29
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
High
Artem Panchenko
8.0.x
High
Artem Panchenko

Bug Description

If there are networks assigned to non-admin(pxe) interfaces on slaves, then network verification could fail, for example:

Repo availability verification using public network failed on following nodes cz5554, cz5553, cz5548, cz5547, cz5549.

Following repos are not available - http://mirror.fuel-infra.org/mos-repos/ubuntu/8.0/

. Check your public network settings and availability of the repositories from public network. Please examine nailgun and astute logs for additional details.
Node Name Node MAC address Node Interface Expected VLAN (not received)
cz5553 a0:2b:b8:1f:4a:ad eno2 untagged, 360
cz5554 a0:2b:b8:1f:4a:91 eno2 untagged, 360
cz5548 a0:d3:c1:ef:16:ed eno2 untagged
cz5549 a0:d3:c1:ef:32:cd eno2 untagged
cz5547 a0:d3:c1:ef:2c:d9 eno2 untagged

Checkers (urlaccesscheck/net_probe) fail because on new Ubuntu bootstrap non-pxe interfaces are DOWN by default. Network checker turns on such interfaces, but it either doesn't wait until link becomes ready or does it incorrectly. So urlaccesscheck tries to access links before public interface (link protocol) is UP:

http://paste.openstack.org/show/482833/

A delay before setting link UP could be caused by different factors related to real (non-virtual) network architecture, for example enabled STP protocol on switches.

Changed in fuel:
milestone: none → 9.0

Fix proposed to branch: master
Review: https://review.openstack.org/262338

Changed in fuel:
assignee: Fuel Python Team (fuel-python) → Artem Panchenko (apanchenko-8)
status: New → In Progress
tags: added: area-python
Dmitry Pyzhov (dpyzhov) on 2016-01-11
tags: added: team-bugfix

Reviewed: https://review.openstack.org/262338
Committed: https://git.openstack.org/cgit/openstack/network-checker/commit/?id=0589710474d4c4a6598fe371c9ed857c8c249897
Submitter: Jenkins
Branch: master

commit 0589710474d4c4a6598fe371c9ed857c8c249897
Author: Artem Panchenko <email address hidden>
Date: Wed Dec 30 02:14:07 2015 +0200

    Wait up to 30 seconds until network link is UP

    Network checker must wait up to 30 seconds for
    network link readiness, because by default all
    non-admin interfaces are turned off on slaves
    in 'discover' state.

    Change-Id: Idfdab5e81182c95836b40e460106f1e775658cf8
    Closes-bug: #1529985

Changed in fuel:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/271108
Committed: https://git.openstack.org/cgit/openstack/network-checker/commit/?id=a43cf96cd9532f10794dce736350bf5bed350e9d
Submitter: Jenkins
Branch: stable/8.0

commit a43cf96cd9532f10794dce736350bf5bed350e9d
Author: Artem Panchenko <email address hidden>
Date: Wed Dec 30 02:14:07 2015 +0200

    Wait up to 30 seconds until network link is UP

    Network checker must wait up to 30 seconds for
    network link readiness, because by default all
    non-admin interfaces are turned off on slaves
    in 'discover' state.

    Change-Id: Idfdab5e81182c95836b40e460106f1e775658cf8
    Closes-bug: #1529985
    (cherry picked from commit 0589710474d4c4a6598fe371c9ed857c8c249897)

Reviewed: https://review.openstack.org/272067
Committed: https://git.openstack.org/cgit/openstack/network-checker/commit/?id=9c192d54bc8c4f2196f9bee30b51ef19cab56dee
Submitter: Jenkins
Branch: master

commit 9c192d54bc8c4f2196f9bee30b51ef19cab56dee
Author: Artem Panchenko <email address hidden>
Date: Mon Jan 25 14:37:16 2016 +0200

    Fix typo in tests

    Change-Id: I53a91684dfab0060ea0bb958590e127b10e499d3
    Related-bug: #1529985

Artem Panchenko (apanchenko-8) wrote :

verified on bare-metal:

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "506"
  build_id: "506"
  fuel-nailgun_sha: "8e954abd70ef0083109f34289de2553dcda544d4"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "658be72c4b42d3e1436b86ac4567ab914bfb451b"
  fuel-nailgun-agent_sha: "b2bb466fd5bd92da614cdbd819d6999c510ebfb1"
  astute_sha: "b81577a5b7857c4be8748492bae1dec2fa89b446"
  fuel-library_sha: "ec7e212972ead554f21b52b9e165156665f659df"
  fuel-ostf_sha: "ab5fd151fc6c1aa0b35bc2023631b1f4836ecd61"
  fuel-mirror_sha: "351d568fa3b3e4dd062054b91d766aa54d379867"
  fuelmenu_sha: "234cb4cbb30fbd2df00f388c28f31606d9cae15f"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "a43cf96cd9532f10794dce736350bf5bed350e9d"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "94507c5e4dad6d8cfbd8f5d41aa8389d5335990a"

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers