MAAS rejects empty files

Reported by Tycho Andersen on 2013-07-24
88
This bug affects 15 people
Affects Status Importance Assigned to Milestone
MAAS
Critical
Raphaël Badin
1.2
Critical
Raphaël Badin
1.3
Critical
Raphaël Badin
maas (Ubuntu)
Critical
Unassigned
Precise
Critical
Unassigned
Quantal
Undecided
Unassigned
Raring
Critical
Unassigned

Bug Description

tycho@cabeiri:~$ juju bootstrap -v
2013-07-24 12:56:03 ERROR juju supercommand.go:235 command failed: cannot create bootstrap state file: gomaasapi: got error back from server: 400 BAD REQUEST
error: cannot create bootstrap state file: gomaasapi: got error back from server: 400 BAD REQUEST

Looks like there is a missing field in one of the requests; attached is a wireshark log for the bootstrap command.

[Impact]
This bug makes Go Juju unusable with MAAS.

[Test Case]
1. Install MAAS
2. Install Go Juju
3. juju bootstrap
(The package without the fix will fail to bootstrap. The package with the fix will bootstrap successfully)

[Regression Potential]
Minimal. This simply defaults a parameter that tells whether to store empty files or not.

Tycho Andersen (tycho-s) wrote :
Robert Ayres (robert-ayres) wrote :

Seems to be failing on versions after 1.11.2-3~1414~raring1.

Looks like Juju tries to upload an empty 'provider-state' file and MAAS doesn't like empty files :)

Raphaël Badin (rvb) wrote :

Creating an empty 'provider-state' file is part of the boostraping sequence now and MAAS rejects empty files… I don't see why MAAS should reject empty files so I'm reassigning this bug to MAAS.

affects: juju-core → maas
summary: - juju bootstrap dies with 400 bad request
+ MAAS rejects empty files
Changed in maas:
importance: Undecided → Critical
assignee: nobody → Raphaël Badin (rvb)
status: New → In Progress
Raphaël Badin (rvb) on 2013-07-24
Changed in maas:
milestone: none → 13.10
Raphaël Badin (rvb) on 2013-07-24
Changed in maas:
status: In Progress → Fix Committed
Changed in maas:
status: Fix Committed → Fix Released
Changed in maas:
status: Fix Released → Fix Committed
Jonathan Davies (jpds) on 2013-08-12
Changed in maas (Ubuntu):
status: New → Triaged
importance: Undecided → Critical
Changed in maas (Ubuntu):
status: Triaged → Fix Released
status: Fix Released → New
status: New → Fix Released
description: updated
Changed in maas (Ubuntu Precise):
importance: Undecided → Critical
Changed in maas (Ubuntu Precise):
assignee: nobody → Andres Rodriguez (andreserl)
Changed in maas (Ubuntu Precise):
assignee: Andres Rodriguez (andreserl) → nobody
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in maas (Ubuntu Precise):
status: New → Confirmed
Changed in maas (Ubuntu Quantal):
status: New → Confirmed
Changed in maas (Ubuntu Raring):
status: New → Confirmed

Hello Tycho, or anyone else affected,

Accepted maas into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/maas/1.2+bzr1373+dfsg-0ubuntu1~12.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in maas (Ubuntu Precise):
status: Confirmed → Fix Committed
tags: added: verification-needed
Runar Ingebrigtsen (ringe) wrote :

Any chance to see this fix in raring?

Ryan C (rc-f5) wrote :

Second the request for a fix on raring. I am "dead in the water" until then.

Runar Ingebrigtsen (ringe) wrote :

Reinstalled MAAS-server using 12.04, and got here:
http://maas.ubuntu.com/docs/juju-quick-start.html

The ~/.juju/environments.yaml looks like this:
juju: environments
environments:
  maas:
    type: maas
    maas-server: 'http://localhost/MAAS'
    maas-oauth: '${maas-api-key}'
    admin-secret: 'nothing'

Except the maas-oauth is the key from http://cloud/MAAS/account/prefs/

mer@cloud:~$ juju status
error: no default environment found
mer@cloud:~$ juju status -e maas -v
2013-08-19 09:45:07 ERROR juju supercommand.go:235 command failed: no instances found
error: no instances found

What does this mean? I have two nodes in my MAAS, so that shouldn't be the instances not found.

Runar, two things:
 1. Please don't comment on a bug with completely unrelated problems
 2. You need to "juju bootstrap" first.

Konrad Meier (konrad-meier) wrote :

I am waiting for a raring fix too. Is there anything I can do to help?

James Page (james-page) wrote :

I've tested this on my 12.04 MAAS and I can now bootstrap and manage environments using juju-core

tags: added: verification-done
removed: verification-needed
Changed in maas (Ubuntu Raring):
status: Confirmed → New
importance: Undecided → Critical
Brian Murray (brian-murray) wrote :

Hello Tycho, or anyone else affected,

Accepted maas into raring-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/maas/1.3+bzr1461+dfsg-0ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-done-precise
removed: verification-done
Changed in maas (Ubuntu Raring):
status: New → Fix Committed
tags: added: verification-needed
Nick Moffitt (nick-moffitt) wrote :

The precise-proposed maas packages did fix this issue for me. Looking forward to their promotion to updates!

Adam Gandelman (gandelman-a) wrote :

I was hitting the same issue on raring, and updating to the package in proposed fixed it for me.

tags: added: verification-done
removed: verification-needed
Kent Baxley (kentb) wrote :

I can also verify that the -proposed set allows me to avoid the bootstrap issues on Precise and juju-core 1.13. So far no new breakage / regressions have occurred with this update.

Ryan C (rc-f5) wrote :

This has fixed the problem for me too, but it potentially has introduced another one.

When upgrading maas from raring-updates, maas-dns is removed which is causing resolution failures due to my maas nodes pointing at my region controller for DNS (which is no longer listening for queries).

Trying to (re-)install maas-dns failes with the following error:

  manager@ubuntu-maas-01:~$ sudo apt-get install --dry-run maas-dns
  Reading package lists... Done
  Building dependency tree
  Reading state information... Done
  Some packages could not be installed. This may mean that you have
  requested an impossible situation or if you are using the unstable
  distribution that some required packages have not yet been created
  or been moved out of Incoming.
  The following information may help to resolve the situation:

  The following packages have unmet dependencies:
   maas-dns : Depends: maas-region-controller (= 1.3+bzr1461+dfsg-0ubuntu2.1) but 1.3+bzr1461+dfsg-0ubuntu2.2 is to be installed
  E: Unable to correct problems, you have held broken packages.

Looks like a minor version difference, but I'm very new to apt and would need guidance if there is perhaps an easy way to fix it.

Matt Rae (mattrae) wrote :

verified that this resolved the 400 BAD REQUEST, during bootstrap

Ryan C (rc-f5) wrote :
Tim (tsm) wrote :

I've applied http://launchpad.net/ubuntu/+source/maas/1.2+bzr1373+dfsg-0ubuntu1~12.04.2 from precise proposed into a fresh maas install and I get the same error.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package maas - 1.2+bzr1373+dfsg-0ubuntu1~12.04.2

---------------
maas (1.2+bzr1373+dfsg-0ubuntu1~12.04.2) precise-proposed; urgency=low

  * MAAS Stable Release Update, debian/patches:
    - 99_filestorage_empty_files_lp1204507.patch: Fix to allow the storage
      of empty files when using Juju Go, otherwise machines will fail to
      bootstrap. (LP: #1204507)
    - 99_fix_highbank_localboot_lp1172966.patch: Fix to PXE LOCALBOOT on
      highbank servers by removing a PXE message. Otherwise highbank will
      fail to pxe boot. (LP: #1172966)
    - 99_no_ipmi_detection_kvm_lp1064527.patch: Fix to ensure that IPMI
      detection does not happen on KVM VM's, otherwise enlistment and
      commissioning process will take too long. (LP: #1064527)
    - 99_update_cluster_info_cli_lp1172193.patch: Fix to allow admins to
      update cluster information from the API/CLI and not only restrict it
      to the WebUI. (LP: #1172193)
    - 99_fix_ipmi_power_command_lp1171418: Fix to ensure that ipmi commands
      are always executed regardless of the state of the machine.
      (LP: #1171418)
    - 99_default_timezone_utc_lp1211447.patch: Default to UTC for the
      deployed nodes. (LP: #1211447)
 -- Andres Rodriguez <email address hidden> Mon, 12 Aug 2013 12:18:34 -0400

Changed in maas (Ubuntu Precise):
status: Fix Committed → Fix Released

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package maas - 1.3+bzr1461+dfsg-0ubuntu2.2

---------------
maas (1.3+bzr1461+dfsg-0ubuntu2.2) raring-proposed; urgency=low

  * debian/patches:
    - 99_filestorage_empty_files_lp1204507.patch: Fix to allow the storage
      of empty files when using Juju Go, otherwise machines will fail to
      bootstrap. (LP: #1204507)
    - 99_default_timezone_utc_lp1211447.patch: Default to UTC for the
      deployed nodes. (LP: #1211447)
 -- Andres Rodriguez <email address hidden> Wed, 21 Aug 2013 10:57:37 -0400

Changed in maas (Ubuntu Raring):
status: Fix Committed → Fix Released
Changed in maas:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers