Could not determine host ip address

Bug #1693689 reported by YAMAMOTO Takashi
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
BaGPipe
Fix Released
Undecided
Unassigned
Group Based Policy
Confirmed
Undecided
Unassigned
neutron
Fix Released
Undecided
Unassigned

Bug Description

it seems affecting the following jobs.

gate-neutron-dsvm-fullstack-ubuntu-xenial
gate-neutron-dsvm-functional-ubuntu-xenial-nv
gate-neutron-dsvm-functional-ubuntu-xenial
gate-networking-bagpipe-dsvm-fullstack-ubuntu-xenial-nv

to me it seems happening only on citycloud.

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%20%5C%22Could%20not%20determine%20host%20ip%20address.%5C%22

eg. http://logs.openstack.org/75/437775/10/check/gate-neutron-dsvm-functional-ubuntu-xenial/22dbc16/

2017-05-26 03:22:08.617415 | +++ functions-common:address_in_net:2286 : network=10.0.0.0
2017-05-26 03:22:08.618918 | +++ functions-common:address_in_net:2287 : local subnet
2017-05-26 03:22:08.620812 | +++++ functions-common:address_in_net:2288 : cidr2netmask 22
2017-05-26 03:22:08.621827 | +++++ functions-common:cidr2netmask:2304 : local 'maskpat=255 255 255 255'
2017-05-26 03:22:08.622784 | +++++ functions-common:cidr2netmask:2305 : local 'maskdgt=254 252 248 240 224 192 128'
2017-05-26 03:22:08.623706 | +++++ functions-common:cidr2netmask:2306 : set -- 255 255 252
2017-05-26 03:22:08.624637 | +++++ functions-common:cidr2netmask:2307 : echo 255.255.252.0
2017-05-26 03:22:08.626091 | ++++ functions-common:address_in_net:2288 : maskip 10.0.1.215 255.255.252.0
2017-05-26 03:22:08.627654 | ++++ functions-common:maskip:2361 : local ip=10.0.1.215
2017-05-26 03:22:08.628849 | ++++ functions-common:maskip:2362 : local mask=255.255.252.0
2017-05-26 03:22:08.629910 | ++++ functions-common:maskip:2363 : local l=10.0.1
2017-05-26 03:22:08.631051 | ++++ functions-common:maskip:2363 : local r=0.1.215
2017-05-26 03:22:08.632372 | ++++ functions-common:maskip:2363 : local n=255.255.252
2017-05-26 03:22:08.633835 | ++++ functions-common:maskip:2363 : local m=255.252.0
2017-05-26 03:22:08.634883 | ++++ functions-common:maskip:2364 : local subnet
2017-05-26 03:22:08.636020 | ++++ functions-common:maskip:2365 : subnet=10.0.0.0
2017-05-26 03:22:08.637027 | ++++ functions-common:maskip:2366 : echo 10.0.0.0
2017-05-26 03:22:08.639064 | +++ functions-common:address_in_net:2288 : subnet=10.0.0.0
2017-05-26 03:22:08.640696 | +++ functions-common:address_in_net:2289 : [[ 10.0.0.0 == 10.0.0.0 ]]
2017-05-26 03:22:08.642357 | +++ functions-common:get_default_host_ip:684 : echo
2017-05-26 03:22:08.643625 | ++ stackrc:source:815 : HOST_IP=
2017-05-26 03:22:08.644779 | ++ stackrc:source:816 : '[' '' == '' ']'
2017-05-26 03:22:08.645754 | ++ stackrc:source:817 : die 817 'Could not determine host ip address. See local.conf for suggestions on setting HOST_IP.'
2017-05-26 03:22:08.646787 | ++ functions-common:die:186 : local exitcode=0
2017-05-26 03:22:08.647956 | ++ functions-common:die:187 : set +o xtrace
2017-05-26 03:22:08.648111 | [Call Trace]
2017-05-26 03:22:08.648178 | ./stack.sh:191:source
2017-05-26 03:22:08.648217 | /opt/stack/new/devstack/stackrc:817:die
2017-05-26 03:22:08.650467 | [ERROR] /opt/stack/new/devstack/stackrc:817 Could not determine host ip address. See local.conf for suggestions on setting HOST_IP.
2017-05-26 03:22:09.655121 | + /opt/stack/new/neutron/neutron/tests/contrib/gate_hook.sh:main:1 : rm -rf /tmp/tmp.nAvEK2rZvR
2017-05-26 03:22:09.657520 | ERROR: the main setup script run by this job failed - exit code: 1
2017-05-26 03:22:09.657561 | please look at the relevant log files to determine the root cause
2017-05-26 03:22:09.657575 | Running devstack worlddump.py
2017-05-26 03:22:09.866930 | Cleaning up host
2017-05-26 03:22:09.867026 | ... this takes 3 - 4 minutes (logs at logs/devstack-gate-cleanup-host.txt.gz)
2017-05-26 03:22:45.893523 | [WARNING]: No hosts matched, nothing to do
2017-05-26 03:22:46.763984 | Generating static files at /home/jenkins/workspace/gate-neutron-dsvm-functional-ubuntu-xenial/logs/ara...
2017-05-26 03:22:47.622857 | Done.
2017-05-26 03:22:49.039304 | *** FAILED with status: 1
2017-05-26 03:22:49.039942 | [Zuul] Task exit code: 1
2017-05-26 03:22:50.923786 | [Zuul] Job complete, result: FAILURE

tags: added: gate-failure
tags: added: fullstack functional-tests
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/469189

Changed in neutron:
assignee: nobody → Ihar Hrachyshka (ihar-hrachyshka)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/471097

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.openstack.org/471097
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a99897ffb3e3a1247006b21103b54a71123a6b50
Submitter: Jenkins
Branch: master

commit a99897ffb3e3a1247006b21103b54a71123a6b50
Author: Ihar Hrachyshka <email address hidden>
Date: Mon Jun 5 18:54:39 2017 +0000

    Use rootwrap for fullstack test runner

    We plan to switch to devstack-gate for fullstack job, and it revokes
    direct sudo calls before executing tests, so we can't rely on sudo
    working anymore.

    This also moves functional-testing.filters to a more generic filename
    (testing.filters) because the filters are now deployed and used by
    fullstack target too.

    Related-Bug: #1557168
    Related-Bug: #1693689

    Change-Id: I1718ea51836adbb8ef8dea79822a722dcf111127

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/469189
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=96a821c48b7de0017d00c43a3f9da91585873354
Submitter: Jenkins
Branch: master

commit 96a821c48b7de0017d00c43a3f9da91585873354
Author: Ihar Hrachyshka <email address hidden>
Date: Tue May 30 10:10:09 2017 -0700

    tests: use devstack-gate to deploy dstat for functional/fullstack

    This should help with a gate failure where IPV4_ADDRS_SAFE_TO_USE would
    not be honoured by devstack, rendering a job failed on a particular
    cloud (like citycloud), like:

    'Could not determine host ip address. See local.conf for suggestions on
    setting HOST_IP.'

    Closes-Bug: #1693689
    Change-Id: Iee7cc4c129e6e9eab91107fda92f66c5379d6f95

tags: added: neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/475294

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 11.0.0.0b3

This issue was fixed in the openstack/neutron 11.0.0.0b3 development milestone.

Revision history for this message
Thomas Morin (tmmorin-orange) wrote :

Aaarg, hitting the same issue I digged logs and devstack gate script shell, came to the conclusion that neutron fullstack jobs should have solved this issue (since n8g-bagpipe fullstack gate hook was stolen from neutron's), ran a git blame on neutron's gate_hook.sh and ended up here, and found a mention on n8g-bagpipe failure in the bug description.

An "affected project: networking-bagpipe" would have spared me quite some time...
I just added n8g-bapipe and group-based-policy as affected projects.

But, well at least the solution is ready!

Changed in networking-bagpipe:
status: New → Confirmed
Changed in group-based-policy:
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-bagpipe (master)

Reviewed: https://review.openstack.org/502067
Committed: https://git.openstack.org/cgit/openstack/networking-bagpipe/commit/?id=8919a92225781588a2a8e795671cf8a0d9deb785
Submitter: Jenkins
Branch: master

commit 8919a92225781588a2a8e795671cf8a0d9deb785
Author: Thomas Morin <email address hidden>
Date: Fri Sep 8 16:15:03 2017 +0200

    devstack gate hook for fullstack job, fix IPV4_ADDRS_SAFE_TO_USE

    Set IPV4_ADDRS_SAFE_TO_USE to the same value used by devstack-vm-gate.sh
    to avoid issues in cloudcity.

    This is temporary until we use devstack-vm-gate.sh
    (see Ib2a4f1f258d39dcba370fe37799c56ad0e5ca8fd).

    Change-Id: I77ad056a0a1ec0e4d0282ae3dbb002641635f043
    Partial-Bug: 1693689

Changed in networking-bagpipe:
status: Confirmed → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (stable/ocata)

Change abandoned by Ihar Hrachyshka (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/475294
Reason: Switching functional and fullstack tests to rootwrap is non-trivial job. I think we can live with spurious failures, especially now that Ocata transitioned to life support II cycle.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to group-based-policy (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/505805

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to group-based-policy (master)

Reviewed: https://review.openstack.org/505346
Committed: https://git.openstack.org/cgit/openstack/group-based-policy/commit/?id=555dc66610cde24110d708e7f4ac01e9405e37dd
Submitter: Jenkins
Branch: master

commit 555dc66610cde24110d708e7f4ac01e9405e37dd
Author: Sumit Naiksatam <email address hidden>
Date: Tue Sep 19 10:34:40 2017 -0700

    Configure IPV4_ADDRS_SAFE_TO_USE in devstack gate hooks

    Set IPV4_ADDRS_SAFE_TO_USE to the same value used by devstack-vm-gate.sh
    to avoid the devstack job failing with the following messaage:

    'Could not determine host ip address. See local.conf for suggestions on
    setting HOST_IP.'

    This fix should be considered temporary until we
    switch to using devstack-vm-gate.sh

    Change-Id: I18924cd7d526874fcf2ac2e9caf0a42c457fc6a3
    Partial-Bug: 1693689

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to group-based-policy (stable/ocata)

Reviewed: https://review.openstack.org/505805
Committed: https://git.openstack.org/cgit/openstack/group-based-policy/commit/?id=b7451022051bfc6fa2b6ecb9e52415b6217d0767
Submitter: Jenkins
Branch: stable/ocata

commit b7451022051bfc6fa2b6ecb9e52415b6217d0767
Author: Sumit Naiksatam <email address hidden>
Date: Tue Sep 19 10:34:40 2017 -0700

    Configure IPV4_ADDRS_SAFE_TO_USE in devstack gate hooks

    Set IPV4_ADDRS_SAFE_TO_USE to the same value used by devstack-vm-gate.sh
    to avoid the devstack job failing with the following messaage:

    'Could not determine host ip address. See local.conf for suggestions on
    setting HOST_IP.'

    This fix should be considered temporary until we
    switch to using devstack-vm-gate.sh

    Change-Id: I18924cd7d526874fcf2ac2e9caf0a42c457fc6a3
    Partial-Bug: 1693689

tags: added: in-stable-ocata
Changed in networking-bagpipe:
status: Fix Committed → Fix Released
Revision history for this message
Slawek Kaplonski (slaweq) wrote : auto-abandon-script

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

Changed in neutron:
assignee: Ihar Hrachyshka (ihar-hrachyshka) → nobody
tags: added: timeout-abandon
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-bagpipe (master)

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: master
Review: https://review.opendev.org/500109
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.