manual provider doesn't install mongo from the cloud archive

Bug #1238934 reported by Kapil Thangavelu
26
This bug affects 5 people
Affects Status Importance Assigned to Milestone
juju-core
Fix Released
High
Andrew Wilkins

Bug Description

this fails to start juju-db cause the distro version in precise doesn't support mongodb w/ ssl. the cloud-archive needs to be added in as part of manual bootstrap.

ALSO the mongodb package version is running.

Related branches

description: updated
Jorge Castro (jorge)
Changed in juju-core:
status: New → Confirmed
description: updated
Curtis Hovey (sinzui)
Changed in juju-core:
status: Confirmed → Triaged
milestone: none → 1.17.0
tags: added: manual
Curtis Hovey (sinzui)
tags: added: manual-provider
removed: manual
Curtis Hovey (sinzui)
Changed in juju-core:
importance: Undecided → High
Revision history for this message
Andrew Wilkins (axwalk) wrote :

This is a bit concerning, as the initialisation code generates a cloud-init script just like the other providers. I just had a thought, that perhaps mongodb was already installed on the target machine. Can you please confirm/deny?

Revision history for this message
Andrew Wilkins (axwalk) wrote :

Scratch that, I see the problem. Part of the cloud-init stuff has to be done "by hand" for manual provisioning, and that bit doesn't consider apt-add-repository.

Andrew Wilkins (axwalk)
Changed in juju-core:
status: Triaged → In Progress
assignee: nobody → Andrew Wilkins (axwalk)
Revision history for this message
Andrew Wilkins (axwalk) wrote :

I'm investigating whether we can just install and use cloud-init in manual provisioning.

Revision history for this message
Andrew Wilkins (axwalk) wrote :

Looks like it will be doable:
 - if cloud-init isn't installed already, install it. We'll want to remove the upstart jobs immediately, or the machine will not be happy when it restarts and there's no EC2 metadata service.
 - generate a YAML config file with the user-data in it
 - run "sudo cloud-init -f <yaml-file> modules"

It would be ideal if there were a python-cloud-init package as well as a cloud-init package, then we wouldn't have to worry about the upstart jobs.

Revision history for this message
Andrew Wilkins (axwalk) wrote :

Actually, one option is to first:
 - sudo mkdir -p /etc/cloud/cloud.cfg.d
 - sudo echo 'datasource_list: [ None ]' > /etc/cloud/cloud.cfg.d/99_juju.cfg

Then restart shouldn't be affected too badly.

Revision history for this message
Andrew Wilkins (axwalk) wrote :

Having trouble getting any of that to work. I've emailed smoser to ask for some advice.

Revision history for this message
Kapil Thangavelu (hazmat) wrote : Re: [Bug 1238934] Re: manual provider doesn't install mongo from the cloud archive

I'd suggest alternatively using cloudinit less in this context and adding
the ppa installation to the script part so that there's less deviation here
from a standard install, it also causes things to fail earlier at the point
of error if there's an issue. I had a discussion with Julian Edwards about
the same yesterday in reference to a different issue.

On Tue, Oct 15, 2013 at 5:57 AM, Andrew Wilkins <
<email address hidden>> wrote:

> Having trouble getting any of that to work. I've emailed smoser to ask
> for some advice.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1238934
>
> Title:
> manual provider doesn't install mongo from the cloud archive
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1238934/+subscriptions
>

Revision history for this message
Dave Cheney (dave-cheney) wrote :

On Sat, Oct 12, 2013 at 9:03 PM, Andrew Wilkins
<email address hidden> wrote:
> Scratch that, I see the problem. Part of the cloud-init stuff has to be
> done "by hand" for manual provisioning, and that bit doesn't consider
> apt-add-repository.

That would also include the boot-cmd parts that we use to knobble the
packaged mongodb upstart script.

>
> --
> You received this bug notification because you are subscribed to juju-
> core.
> Matching subscriptions: MOAR JUJU SPAM!
> https://bugs.launchpad.net/bugs/1238934
>
> Title:
> manual provider doesn't install mongo from the cloud archive
>
> Status in juju-core:
> Triaged
>
> Bug description:
> this fails to start juju-db cause the distro version in precise
> doesn't support mongodb w/ ssl. the cloud-archive needs to be added in
> as part of manual bootstrap.
>
> ALSO the mongodb package version is running.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1238934/+subscriptions

Revision history for this message
Scott Moser (smoser) wrote :

The way I'd like to handle this is to have cloud-init actually functional on the system after installation via juju.

I'd like to provide a command in cloud-init which would do what you want it to do. We can add that to cloud-init for 14.04. For 13.10 and prior, then we'd have 2 paths:
 a.) SRU this new function/feature
 b.) juju carry it and "inject" when it installs cloud-init it on hosts < 14.04.

As an example Usage for this tool:

 * cloud-init manual-purge:
   remove/clean /var/lib/cloud-init and potentially /var/log/cloud-init files.

 * cloud-init manual-seed-nocloud "--user-data-file=my-user-data.txt" \
     --metadata=my-metadata.yaml
   where user-data is straight forward and metadata would be as described in doc/examples/seed/meta-data [1]
   This would/could also configure /etc/cloud/cloud.cfg.d/99-local.cfg to define only the localcloud datasource
   [1] http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/seed/meta-data

 * cloud-init manual-execute
   This would run through each of the cloud-init stages, and would consume the .

This is definitely something that can be supported on 12.04 and further and would be generally useful other places. Juju is not the first user to ask "how can I re-run cloud-init stuff".

Nothing here is hard, just has to be done. I really do think this is "the right way to do it".

Revision history for this message
Andrew Wilkins (axwalk) wrote :

@smoser: thanks, that sounds good to me.

Another thing to consider is splitting the cloud-init package into python-cloud-init for the Python packages, and cloud-init for the upstart scripts and configuration. Then juju can get just what it needs, without worrying about polluting the machine with unnecessary (and dangerous) upstart scripts.

Since it's going to take a little while to do this properly, I'm going to do what hazmat suggested for now; we'll manually add in the bootcmds, PGP key extraction, and apt-add-repository calls. I'll create a tech-debt bug to address this properly.

Revision history for this message
Scott Moser (smoser) wrote :

well, honestly, it wouldn't take you to long to do the above.
Its a simplish stand alone python program with those subcommands. Not something i have time to do right now, but not terribly hard to do either. I guess I wouldn't actually recommend adding it as subcommands to cloud-init multi-call binary, but a different command (so that it wouild be more seperatable).

I think you might as well use cloud-init as part of init. It should be generally safe execpt for being annoying in the event of a misconfigured /etc/network/interfaces.

Also note that bug 1240667 is related.

Revision history for this message
Andrew Wilkins (axwalk) wrote :

No, not difficult to implement, but getting it into the machine is another matter. If there were already a library-only package for cloud-init it'd make things easier.

Anyway, I've made a note on lp:1215777 to fix this properly, which I suspect will happen soon enough for some other work. In the mean time, I've got an MP that fixes the immediate problem.

Curtis Hovey (sinzui)
tags: added: cloud-archive
Curtis Hovey (sinzui)
tags: added: ssh-provider
removed: manual-provider
Revision history for this message
itzg (itzgeoff) wrote :

I am also seeing this behavior with a MAAS environment. After working around Bug #1243861, the juju-db service fails to start with the same symptom as seen in /var/log/upstart/juju-db.log

error command line: unknown option sslOnNormalPorts

Since this bug is out for code review, should I report a new bug?

Here are the specifics about the versions in use:

dev@maas-master:/var/lib/maas/ephemeral/precise/ephemeral/amd64/20131010$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 13.10
Release: 13.10
Codename: saucy
dev@maas-master:/var/lib/maas/ephemeral/precise/ephemeral/amd64/20131010$ cat info
release=precise
label=release
serial=20131010
arch=amd64
name=maas-precise-12.04-amd64-20131010
dev@maas-master:/var/lib/maas/ephemeral/precise/ephemeral/amd64/20131010$ juju version
1.16.0-saucy-amd64

Revision history for this message
Andrew Wilkins (axwalk) wrote :

@itzg: this bug/fix is specific to the manual provider, so if you're seeing it in MAAS it's certainly something different (though perhaps very similar?)

Please do file another bug.

Andrew Wilkins (axwalk)
Changed in juju-core:
status: In Progress → Fix Committed
Curtis Hovey (sinzui)
tags: added: manual-provider
removed: ssh-provider
Curtis Hovey (sinzui)
Changed in juju-core:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers