http error 409 instead of 503

Bug #1820307 reported by Sam Lee
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Expired
Undecided
Unassigned
MAAS
Invalid
Undecided
Unassigned

Bug Description

On occasion when high enough concurrent allocate machine API calls (using github.com/juju/gomaasapi) hits the region controller, the controller responds on some with Error 409: Conflict Error.

```
Error: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head>
    <title>Error: Conflict Error</title>
  </head>
  <body>
    <h2>
      Conflict error. Try your request again, as it will most likely succeed.
    </h2>
  </body>
</html>
```
Simply re-running the batch of API calls resumes allocating the ones that didn't allocate previously.

If instead the controller would return an error of 503: Service Unavailable, the gomaasapi has built-in retry logic.

Revision history for this message
Blake Rouse (blake-rouse) wrote :

A 409 means that a conflict has occurred. MAAS internally retries 10 times to try and handle conflicts with IP allocation when allocating a machine with AUTO IP addresses. If you are getting failures in this case then the its now the clients turn to retry the action.

This is a bug in the gomaasapi for not retrying the action.

Revision history for this message
Andres Rodriguez (andreserl) wrote : Re: [Bug 1820307] Re: http error 409 instead of 503

Sam,

Do you happen to know the average number of concurrent requests that are
being made in the API for this endpoint when this error appears?

On Fri, Mar 15, 2019 at 1:30 PM Blake Rouse <email address hidden>
wrote:

> A 409 means that a conflict has occurred. MAAS internally retries 10
> times to try and handle conflicts with IP allocation when allocating a
> machine with AUTO IP addresses. If you are getting failures in this case
> then the its now the clients turn to retry the action.
>
> This is a bug in the gomaasapi for not retrying the action.
>
> ** Also affects: juju
> Importance: Undecided
> Status: New
>
> --
> You received this bug notification because you are subscribed to MAAS.
> https://bugs.launchpad.net/bugs/1820307
>
> Title:
> http error 409 instead of 503
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1820307/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: product=juju; status=New; importance=Undecided;
> assignee=None;
> Launchpad-Bug: product=maas; status=New; importance=Undecided;
> assignee=None;
> Launchpad-Bug-Information-Type: Public
> Launchpad-Bug-Private: no
> Launchpad-Bug-Security-Vulnerability: no
> Launchpad-Bug-Commenters: blake-rouse sjwl
> Launchpad-Bug-Reporter: Sam Lee (sjwl)
> Launchpad-Bug-Modifier: Blake Rouse (blake-rouse)
> Launchpad-Message-Rationale: Subscriber (MAAS)
> Launchpad-Message-For: andreserl
>
--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer

Revision history for this message
Sam Lee (sjwl) wrote :

We don't use AUTO IP, we use STATIC.

Revision history for this message
John A Meinel (jameinel) wrote :

Is this a request for us to treat 409 as a retry error?

Changed in juju:
status: New → Incomplete
Revision history for this message
Blake Rouse (blake-rouse) wrote :

IDK if you want to treat this bug as that issue, but gomaasapi should retry on 409 respecting the Retry-After header.

Revision history for this message
Björn Tillenius (bjornt) wrote :

Returning 503 would be incorrect, since the service is in fact available. If anything needs to happen, it's either to debug to see if we somehow can avoid the 409, or Juju needs to retry on 409.

Changed in maas:
status: New → Invalid
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for juju because there has been no activity for 60 days.]

Changed in juju:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.