IBP: fuel-agent should handle socket.timeout exception

Bug #1458526 reported by Roman Vyalov
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
Critical
Alexander Gordeev

Bug Description

http://jenkins-product.srt.mirantis.net:8080/job/6.1.staging.ubuntu.bvt_2/367/console
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 414, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 463, in wrapper
    return func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 470, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/helpers/decorators.py", line 356, in wrapper
    return func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/models/fuel_web_client.py", line 727, in deploy_cluster_wait
    self.assert_task_success(task, interval=interval)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/__init__.py", line 48, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/6.1.staging.ubuntu.bvt_2/fuelweb_test/models/fuel_web_client.py", line 267, in assert_task_success
    task['status'], 'ready', name=task["name"]
AssertionError: Task 'deploy' has incorrect status. error != ready

Roman Vyalov (r0mikiam)
Changed in fuel:
assignee: nobody → MOS QA Team (mos-qa)
importance: Undecided → Critical
importance: Critical → High
milestone: none → 6.1
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :

why this issue is assigned on QA team, if we look at the env or in logs, we can see that cluster has error state, according to next problem
http://paste.openstack.org/show/235626/

Changed in fuel:
assignee: MOS QA Team (mos-qa) → Fuel provisioning team (fuel-provisioning)
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :
Changed in fuel:
assignee: Fuel provisioning team (fuel-provisioning) → Aleksandr Gordeev (a-gordeev)
summary: - [staging] jenkins jobs for staging mirrors 6.1 failed with 'FAIL: Deploy
- ceph HA with RadosGW for objects'
+ image building failed. Incorrect handling of Getting 'release' file via
+ http
tags: added: feature-image-based
Changed in fuel:
status: New → Confirmed
Revision history for this message
Alexander Gordeev (a-gordeev) wrote : Re: image building failed. Incorrect handling of Getting 'release' file via http

https://github.com/stackforge/fuel-web/blob/master/fuel_agent/fuel_agent/utils/build_utils.py#L429
fuel-agent got unexpected (in terms of exception handling) network connectivity related issue. Should be fixed.

Roman Vyalov (r0mikiam)
Changed in fuel:
importance: High → Critical
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/185436

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
Alexander Gordeev (a-gordeev) wrote : Re: image building failed. Incorrect handling of Getting 'release' file via http

to be precise it's python-requests related issue. It doesn't handle socket.timeout proreply.

summary: - image building failed. Incorrect handling of Getting 'release' file via
- http
+ IBP: fuel-agent should handle socket.timeout exception
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/185436
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=78250a2c44a84ea0ebca9942a63b6658ebbc3b6a
Submitter: Jenkins
Branch: master

commit 78250a2c44a84ea0ebca9942a63b6658ebbc3b6a
Author: Alexander Gordeev <email address hidden>
Date: Mon May 25 20:46:08 2015 +0300

    IBP: Fix socket.timeout catching

    Recent version of python-requests doesn't catch socket.timeout()
    This patch resolves the issue by explicit adding socket.timeout to
    the list of expected exceptions to be caught.

    Change-Id: Ifcadf9efbec633175bb1bd31cf7f58732c2e78c4
    Closes-Bug: #1458526

Changed in fuel:
status: In Progress → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.