Gate failures for neutron in test_dualnet_multi_prefix_slaac

Bug #1540983 reported by Davanum Srinivas (DIMS)
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack-Gate
Expired
Undecided
Unassigned
neutron
Expired
High
Unassigned

Bug Description

24 hits in 7 days for logstash query : message:"in test_dualnet_multi_prefix_slaac" AND voting:1

2016-02-02 14:35:49.054 | Captured traceback:
2016-02-02 14:35:49.054 | ~~~~~~~~~~~~~~~~~~~
2016-02-02 14:35:49.054 | Traceback (most recent call last):
2016-02-02 14:35:49.054 | File "tempest/test.py", line 113, in wrapper
2016-02-02 14:35:49.055 | return f(self, *func_args, **func_kwargs)
2016-02-02 14:35:49.055 | File "tempest/scenario/test_network_v6.py", line 246, in test_dualnet_multi_prefix_slaac
2016-02-02 14:35:49.055 | dualnet=True)
2016-02-02 14:35:49.055 | File "tempest/scenario/test_network_v6.py", line 155, in _prepare_and_test
2016-02-02 14:35:49.055 | sshv4_1, ips_from_api_1, sid1 = self.prepare_server(networks=net_list)
2016-02-02 14:35:49.055 | File "tempest/scenario/test_network_v6.py", line 128, in prepare_server
2016-02-02 14:35:49.055 | username=username)
2016-02-02 14:35:49.056 | File "tempest/scenario/manager.py", line 390, in get_remote_client
2016-02-02 14:35:49.056 | linux_client.validate_authentication()
2016-02-02 14:35:49.056 | File "tempest/common/utils/linux/remote_client.py", line 63, in validate_authentication
2016-02-02 14:35:49.056 | self.ssh_client.test_connection_auth()
2016-02-02 14:35:49.056 | File "/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/ssh.py", line 172, in test_connection_auth
2016-02-02 14:35:49.056 | connection = self._get_ssh_connection()
2016-02-02 14:35:49.056 | File "/opt/stack/new/tempest/.tox/full/local/lib/python2.7/site-packages/tempest_lib/common/ssh.py", line 87, in _get_ssh_connection
2016-02-02 14:35:49.057 | password=self.password)
2016-02-02 14:35:49.057 | tempest_lib.exceptions.SSHTimeout: Connection to the 172.24.5.141 via SSH timed out.
2016-02-02 14:35:49.057 | User: cirros, Password: None

Revision history for this message
Sean M. Collins (scollins) wrote :

This is most likely a symptom of another bug. Tempest SSHs into a created instance over v4, then tries to SSH over v6 to a second compute node that was created.

Changed in neutron:
importance: Undecided → Low
status: New → Confirmed
tags: added: gate-failure ipv6
Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

All gate failures should be treated as critical issues and they cause serious disruption to the normal development routine.

Changed in neutron:
importance: Low → Critical
Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

Obviously there are some that are more or less severe (frequent) than others. To be hitting 24 times in the last few days it seems this is spiking and we should look into it. I wonder if this falls under the category of bugs:

bug 1509004
bug 1477192

Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

Btw on second inspection, this failed once in the gate queue in the last 7 days.

Changed in openstack-gate:
status: New → Confirmed
Revision history for this message
Brian Haley (brian-haley) wrote :

http://logs.openstack.org/08/275508/1/check/gate-tempest-dsvm-neutron-full/5794f8e/logs/ is an example of this error still happening periodically.

And this is a link mtreinish posted in irc showing the status of the SLAAC test. Strangely this doesn't show the failure mentioned above, so more work is required:

http://status.openstack.org/openstack-health/#/test/tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_slaac?groupKey=project&resolutionKey=hour&duration=P3M

Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

0 gate occurrences in the past 7 days.

Changed in neutron:
importance: Critical → Undecided
status: Confirmed → Incomplete
Changed in openstack-gate:
status: Confirmed → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for OpenStack-Gate because there has been no activity for 60 days.]

Changed in openstack-gate:
status: Incomplete → Expired
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for neutron because there has been no activity for 60 days.]

Changed in neutron:
status: Incomplete → Expired
Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :
Changed in neutron:
status: Expired → Confirmed
importance: Undecided → Critical
Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

22 occurrences during the last 7 days.

tags: added: l3-ipam-dhcp
Revision history for this message
Carl Baldwin (carl-baldwin) wrote :

The query comes up empty for the last 7 days. I wonder if the partial fix for this one [1] got it. It is the same test and could be two manifestations of the same bug.

[1] https://bugs.launchpad.net/neutron/+bug/1609540

Revision history for this message
Carl Baldwin (carl-baldwin) wrote :
Changed in neutron:
status: Confirmed → Incomplete
Revision history for this message
Matthew Treinish (treinish) wrote :

The test has been skipped in the gate since Aug. 3rd with: https://review.openstack.org/#/c/275457/ so I don't think the lack of elastic search hits is necessarily indicative of a fix.

Changed in neutron:
assignee: nobody → Oleg Bondarev (obondarev)
Revision history for this message
Carl Baldwin (carl-baldwin) wrote :

I missed that the test was disabled. Changing to New and down-grading priority since it isn't a gate failure technically.

Changed in neutron:
status: Incomplete → New
importance: Critical → High
status: New → Confirmed
Revision history for this message
Oleg Bondarev (obondarev) wrote :

Trying to trigger a failure here: https://review.openstack.org/#/c/374106/

Revision history for this message
Miguel Lavalle (minsel) wrote :

I have been watching this bug in Kibana over the past 2 weeks, with a total of 43 hits. All of them were caused by:

1) Broken linuxbridge job, which was documented here: http://lists.openstack.org/pipermail/openstack-dev/2016-November/108164.html

2) The failures are due to bugs / errors in the corresponding patchset

As a consequence, marking it as incomplete

Changed in neutron:
status: Confirmed → Incomplete
Changed in neutron:
assignee: Oleg Bondarev (obondarev) → nobody
Changed in openstack-gate:
status: Expired → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for neutron because there has been no activity for 60 days.]

Changed in neutron:
status: Incomplete → Expired
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for OpenStack-Gate because there has been no activity for 60 days.]

Changed in openstack-gate:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers