[SRU] powering multiple nodes errors with max retries

Bug #1311140 reported by Blake Rouse on 2014-04-22
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
python-seamicroclient (Ubuntu)
Critical
Unassigned
Trusty
Critical
Unassigned

Bug Description

When powering multiple nodes on at once, in different sub-processes. The following error is generated and the node is not powered off or on accordingly.

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/lib/python2.7/dist-packages/provisioningserver/custom_hardware/seamicro.py", line 324, in power_control_seamicro15k_v2
    server.power_on(using_pxe=True)
  File "/usr/lib/python2.7/dist-packages/seamicroclient/v2/servers.py", line 28, in power_on
    self.manager.power_on(self, using_pxe)
  File "/usr/lib/python2.7/dist-packages/seamicroclient/v2/servers.py", line 114, in power_on
    self._action('power-on', server, action_params)
  File "/usr/lib/python2.7/dist-packages/seamicroclient/v2/servers.py", line 219, in _action
    return self.api.client.put(url, body=body)
  File "/usr/lib/python2.7/dist-packages/seamicroclient/client.py", line 165, in put
    return self._cs_request(url, 'PUT', **kwargs)
  File "/usr/lib/python2.7/dist-packages/seamicroclient/client.py", line 155, in _cs_request
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/seamicroclient/client.py", line 141, in _time_request
    resp, body = self.request(url, method, **kwargs)
  File "/usr/lib/python2.7/dist-packages/seamicroclient/client.py", line 115, in request
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 486, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 378, in send
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='192.168.220.102', port=80): Max retries exceeded with url: /v2.0/servers/21/0 (Caused by <class 'httplib.BadStatusLine'>: '')

[Impact]
This bug affects the ability of MAAS (and really any other client/user) to power on multiple (at scale) nodes in a Seamicro15000 chassis using their python-seamicroclient APIT, which is what MAAS uses. This prevents MAAS from being able to power manage many nodes at the time.

[Test Case]
1. Install MAAS. (sudo apt-get install maas)
2. Register SM15k. (maas admin node-group probe-and-enlist-hardware <clkuster-uuid) model=seamicro15k mac=<mac> username=<user> password=<pass> power_control=restapi2
3. Power on Many nodes at the time (juju deploy -n 10)
4. Fail.

With the fix:

3. power on many nodes at the time will succeed.

[Regression Potential]
Minimal. This only affects power on/off of nodes. This has been tested against hardware at one of our labs.

Changed in python-seamicroclient (Ubuntu):
status: New → Confirmed
Changed in python-seamicroclient (Ubuntu):
importance: Undecided → Critical
summary: - powering multiple nodes errors with max retries
+ [SRU] powering multiple nodes errors with max retries
Andres Rodriguez (andreserl) wrote :

Please, also copy this to ubuntu+1 (ubuntu r)

description: updated
Andres Rodriguez (andreserl) wrote :

err ubuntu 'u'

Hello Blake, or anyone else affected,

Accepted python-seamicroclient into trusty-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/python-seamicroclient/0.2.1-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python-seamicroclient (Ubuntu Trusty):
status: New → Triaged
importance: Undecided → Critical
status: Triaged → Fix Committed
tags: added: verification-needed
Andres Rodriguez (andreserl) wrote :

I've tested the fix and it works as expected. Please copy such package into Utopic!

Thanks.

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-seamicroclient - 0.2.1-0ubuntu1

---------------
python-seamicroclient (0.2.1-0ubuntu1) trusty-proposed; urgency=medium

  * New upstream bugfix release.
    - This release only fixes connection issues when trying to power on/off
      multiple nodes at the same time. (LP: #1311140)
  * debian/patches/fix-tests.patch: Fix tests.
 -- Andres Rodriguez <email address hidden> Tue, 22 Apr 2014 12:18:19 -0400

Changed in python-seamicroclient (Ubuntu Trusty):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for python-seamicroclient has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-seamicroclient - 0.2.1-0ubuntu1

---------------
python-seamicroclient (0.2.1-0ubuntu1) trusty-proposed; urgency=medium

  * New upstream bugfix release.
    - This release only fixes connection issues when trying to power on/off
      multiple nodes at the same time. (LP: #1311140)
  * debian/patches/fix-tests.patch: Fix tests.
 -- Andres Rodriguez <email address hidden> Tue, 22 Apr 2014 12:18:19 -0400

Changed in python-seamicroclient (Ubuntu):
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers