ssh failed to run that caused master node unaccessability by 22 port

Bug #1539511 reported by Tatyanka on 2016-01-29
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Sergey Kulanov
Fuel Sustaining

Bug Description

Test failed with:
ERROR: Create environment and set up master node
Traceback (most recent call last):
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/", line 296, in testng_method_mistake_capture_func
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/compatability/", line 27, in capture_type_error
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/", line 350, in func
  File "/home/jenkins/workspace/9.0-liberty.ubuntu.smoke_neutron/fuelweb_test/helpers/", line 83, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/9.0-liberty.ubuntu.smoke_neutron/fuelweb_test/tests/", line 167, in setup_master
  File "/home/jenkins/workspace/9.0-liberty.ubuntu.smoke_neutron/fuelweb_test/models/", line 444, in setup_environment
  File "/home/jenkins/workspace/9.0-liberty.ubuntu.smoke_neutron/fuelweb_test/", line 59, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/9.0-liberty.ubuntu.smoke_neutron/fuelweb_test/models/", line 519, in wait_for_provisioning
    (self.d_env.admin_net), 22), timeout=timeout)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/devops/helpers/", line 108, in _wait
    return raising_predicate()
  File "/home/jenkins/workspace/9.0-liberty.ubuntu.smoke_neutron/fuelweb_test/models/", line 519, in <lambda>
    (self.d_env.admin_net), 22), timeout=timeout)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/devops/helpers/", line 56, in _tcp_ping
    s.connect((str(host), int(port)))
  File "/usr/lib/python2.7/", line 224, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 111] Connection refused

after connect to vm over vncviewer I see that there is no ssh service running, after it restart node became accessible over ssh

Looks like we do not start it at all due to errr in bootsrap_master.log:

Tatyanka (tatyana-leontovich) wrote :
Changed in fuel:
assignee: nobody → MOS Linux (mos-linux)
Sergey Kulanov (skulanov) wrote :
Download full text (4.0 KiB)

yep, need to revert

[root@fuel ~]# fuelmenu

Traceback (most recent call last):
  File "/usr/bin/fuelmenu", line 9, in <module>
    load_entry_point('fuelmenu==9.0.0', 'console_scripts', 'fuelmenu')()
  File "/usr/lib/python2.7/site-packages/fuelmenu/", line 458, in main
  File "/usr/lib/python2.7/site-packages/fuelmenu/", line 314, in setup
  File "/usr/lib/python2.7/site-packages/fuelmenu/", line 93, in __init__
  File "/usr/lib/python2.7/site-packages/fuelmenu/", line 254, in main
  File "/usr/lib64/python2.7/site-packages/urwid/", line 271, in run
  File "/usr/lib64/python2.7/site-packages/urwid/", line 241, in run_wrapper
    return fn()
  File "/usr/lib64/python2.7/site-packages/urwid/", line 336, in _run
  File "/usr/lib64/python2.7/site-packages/urwid/", line 707, in run
  File "/usr/lib64/python2.7/site-packages/urwid/", line 786, in _loop
  File "/usr/lib64/python2.7/site-packages/urwid/", line 387, in _update
  File "/usr/lib64/python2.7/site-packages/urwid/", line 487, in process_input
    k = self._topmost_widget.keypress(self.screen_size, k)
  File "/usr/lib64/python2.7/site-packages/urwid/", line 1102, in keypress
    return self.body.keypress( (maxcol, remaining), key )
  File "/usr/lib64/python2.7/site-packages/urwid/", line 986, in keypress
    key = focus_widget.keypress((maxcol,),key)
  File "/usr/lib64/python2.7/site-packages/urwid/", line 2240, in keypress
    key = w.keypress((mc,) + size[1:], key)
  File "/usr/lib64/python2.7/site-packages/urwid/", line 1559, in keypress
    key = self.focus.keypress(tsize, key)
  File "/usr/lib64/python2.7/site-packages/urwid/", line 384, in keypress
    return self._original_widget.keypress((maxcol, self.height), key)
  File "/usr/lib64/python2.7/site-packages/urwid/", line 836, in keypress
    return self._original_widget.keypress((maxcol,maxrow-top-bottom), key)
  File "/usr/lib64/python2.7/site-packages/urwid/", line 986, in keypress
    key = focus_widget.keypress((maxcol,),key)
  File "/usr/lib64/python2.7/site-packages/urwid/", line 534, in keypress
  File "/usr/lib64/python2.7/site-packages/urwid/", line 463, in _emit
    signals.emit_signal(self, name, self, *args)
  File "/usr/lib64/python2.7/site-packages/urwid/", line 120, in emit
    result |= bool(callback(*args_copy))
  File "/usr/lib/python2.7/site-packages/fuelmenu/modules/", line 50, in save_and_quit
  File "/usr/lib/python2.7/site-packages/fuelmenu/modules/", line 56, in save
    results, modulename = self.parent.global_save()
  File "/usr/lib/python2.7/site-packages/fuelmenu/", line 299, in global_save
    if module.apply(None):
  File "/usr/lib/python2.7/site-packages/fuelmenu/modules/fueluser....


Fix proposed to branch: master

Changed in fuel:
assignee: MOS Linux (mos-linux) → Sergey Kulanov (skulanov)
status: New → In Progress
Roman Vyalov (r0mikiam) on 2016-01-29
tags: added: area-python

Change abandoned by Sergey Kulanov (<email address hidden>) on branch: master
Reason: fixed by

Sergey Kulanov (skulanov) wrote :
Changed in fuel:
status: In Progress → Fix Committed

Seems, it is reproduced on ci[0]:
cat /var/log/fuelmenu.log
2016-03-15 21:12:24,482 DEBUG Executing command: docker ps -a
2016-03-15 21:12:24,532 DEBUG Command executed with exit code: 1
2016-03-15 21:12:24,536 DEBUG Executing command: dhcpcheck discover --timeout 5 -f json --ifaces enp0s3
2016-03-15 21:12:35,897 DEBUG Command executed with exit code: 0
2016-03-15 21:12:35,898 DEBUG No DHCP servers found
2016-03-15 21:12:35,898 DEBUG Executing command: arping -D -c3 -w1 -I enp0s3 -s
2016-03-15 21:12:37,916 DEBUG Command executed with exit code: 0
2016-03-15 21:12:37,944 ERROR Unable to read YAML: /etc/fuel/astute.yaml

I assume that by lines in third comment: "IOError: [Errno 2] No such file or directory: '/etc/fuel/astute.yaml'"


It has not been reproduced. Setting to 'fix released'

Changed in fuel:
status: Fix Committed → Fix Released
Sergey Kulanov (skulanov) wrote :


I'd rather vote for creating another bug with "migration flow" cause this bug should not be reproduced during provisioning/deployment master node

Also could you please also add logs from the reverted environment, why do you think it's related to fuel-menu?

Vadim Rovachev (vrovachev) wrote :


sorry for my wrong 0 comment.

In test result we see that test fails on check migrate.

Log from master node:
[root@nailgun ~]# cat /var/log/fuelmenu.log
2016-07-18 11:58:12,222 DEBUG No DHCP servers found
2016-07-18 11:58:14,246 ERROR Unable to read YAML: /etc/fuel/astute.yaml
2016-07-18 11:58:14,263 ERROR Unable to read YAML: /etc/fuel/astute.yaml

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers