manual provider doesn't install mongo from the cloud archive

[Kapil Thangavelu]

this fails to start juju-db cause the distro version in precise doesn't support mongodb w/ ssl. the cloud-archive needs to be added in as part of manual bootstrap.

ALSO the mongodb package version is running.

[Andrew Wilkins (axwalk)]

This is a bit concerning, as the initialisation code generates a cloud-init script just like the other providers. I just had a thought, that perhaps mongodb was already installed on the target machine. Can you please confirm/deny?

[Andrew Wilkins (axwalk)]

Scratch that, I see the problem. Part of the cloud-init stuff has to be done "by hand" for manual provisioning, and that bit doesn't consider apt-add-repository.

[Andrew Wilkins (axwalk)]

I'm investigating whether we can just install and use cloud-init in manual provisioning.

[Andrew Wilkins (axwalk)]

Looks like it will be doable:
 - if cloud-init isn't installed already, install it. We'll want to remove the upstart jobs immediately, or the machine will not be happy when it restarts and there's no EC2 metadata service.
 - generate a YAML config file with the user-data in it
 - run "sudo cloud-init -f <yaml-file> modules"

It would be ideal if there were a python-cloud-init package as well as a cloud-init package, then we wouldn't have to worry about the upstart jobs.

[Andrew Wilkins (axwalk)]

Actually, one option is to first:
 - sudo mkdir -p /etc/cloud/cloud.cfg.d
 - sudo echo 'datasource_list: [ None ]' > /etc/cloud/cloud.cfg.d/99_juju.cfg

Then restart shouldn't be affected too badly.

[Andrew Wilkins (axwalk)]

Having trouble getting any of that to work. I've emailed smoser to ask for some advice.

[Kapil Thangavelu (hazmat)]

I'd suggest alternatively using cloudinit less in this context and adding
the ppa installation to the script part so that there's less deviation here
from a standard install, it also causes things to fail earlier at the point
of error if there's an issue. I had a discussion with Julian Edwards about
the same yesterday in reference to a different issue.

On Tue, Oct 15, 2013 at 5:57 AM, Andrew Wilkins <
<email address hidden>> wrote:

> Having trouble getting any of that to work. I've emailed smoser to ask
> for some advice.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1238934
>
> Title:
> manual provider doesn't install mongo from the cloud archive
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1238934/+subscriptions
>

[Dave Cheney (dave-cheney)]

On Sat, Oct 12, 2013 at 9:03 PM, Andrew Wilkins
<email address hidden> wrote:
> Scratch that, I see the problem. Part of the cloud-init stuff has to be
> done "by hand" for manual provisioning, and that bit doesn't consider
> apt-add-repository.

That would also include the boot-cmd parts that we use to knobble the
packaged mongodb upstart script.

>
> --
> You received this bug notification because you are subscribed to juju-
> core.
> Matching subscriptions: MOAR JUJU SPAM!
> https://bugs.launchpad.net/bugs/1238934
>
> Title:
> manual provider doesn't install mongo from the cloud archive
>
> Status in juju-core:
> Triaged
>
> Bug description:
> this fails to start juju-db cause the distro version in precise
> doesn't support mongodb w/ ssl. the cloud-archive needs to be added in
> as part of manual bootstrap.
>
> ALSO the mongodb package version is running.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1238934/+subscriptions

[Scott Moser (smoser)]

The way I'd like to handle this is to have cloud-init actually functional on the system after installation via juju.

I'd like to provide a command in cloud-init which would do what you want it to do. We can add that to cloud-init for 14.04. For 13.10 and prior, then we'd have 2 paths:
 a.) SRU this new function/feature
 b.) juju carry it and "inject" when it installs cloud-init it on hosts < 14.04.

As an example Usage for this tool:

 * cloud-init manual-purge:
   remove/clean /var/lib/cloud-init and potentially /var/log/cloud-init files.

 * cloud-init manual-seed-nocloud "--user-data-file=my-user-data.txt" \
     --metadata=my-metadata.yaml
   where user-data is straight forward and metadata would be as described in doc/examples/seed/meta-data [1]
   This would/could also configure /etc/cloud/cloud.cfg.d/99-local.cfg to define only the localcloud datasource
   [1] http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/seed/meta-data

 * cloud-init manual-execute
   This would run through each of the cloud-init stages, and would consume the .

This is definitely something that can be supported on 12.04 and further and would be generally useful other places. Juju is not the first user to ask "how can I re-run cloud-init stuff".

Nothing here is hard, just has to be done. I really do think this is "the right way to do it".

[Andrew Wilkins (axwalk)]

@smoser: thanks, that sounds good to me.

Another thing to consider is splitting the cloud-init package into python-cloud-init for the Python packages, and cloud-init for the upstart scripts and configuration. Then juju can get just what it needs, without worrying about polluting the machine with unnecessary (and dangerous) upstart scripts.

Since it's going to take a little while to do this properly, I'm going to do what hazmat suggested for now; we'll manually add in the bootcmds, PGP key extraction, and apt-add-repository calls. I'll create a tech-debt bug to address this properly.

[Scott Moser (smoser)]

well, honestly, it wouldn't take you to long to do the above.
Its a simplish stand alone python program with those subcommands. Not something i have time to do right now, but not terribly hard to do either. I guess I wouldn't actually recommend adding it as subcommands to cloud-init multi-call binary, but a different command (so that it wouild be more seperatable).

I think you might as well use cloud-init as part of init. It should be generally safe execpt for being annoying in the event of a misconfigured /etc/network/interfaces.

Also note that bug 1240667 is related.

[Andrew Wilkins (axwalk)]

No, not difficult to implement, but getting it into the machine is another matter. If there were already a library-only package for cloud-init it'd make things easier.

Anyway, I've made a note on lp:1215777 to fix this properly, which I suspect will happen soon enough for some other work. In the mean time, I've got an MP that fixes the immediate problem.

[itzg (itzgeoff)]

I am also seeing this behavior with a MAAS environment. After working around Bug #1243861, the juju-db service fails to start with the same symptom as seen in /var/log/upstart/juju-db.log

error command line: unknown option sslOnNormalPorts

Since this bug is out for code review, should I report a new bug?

Here are the specifics about the versions in use:

dev@maas-master:/var/lib/maas/ephemeral/precise/ephemeral/amd64/20131010$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 13.10
Release: 13.10
Codename: saucy
dev@maas-master:/var/lib/maas/ephemeral/precise/ephemeral/amd64/20131010$ cat info
release=precise
label=release
serial=20131010
arch=amd64
name=maas-precise-12.04-amd64-20131010
dev@maas-master:/var/lib/maas/ephemeral/precise/ephemeral/amd64/20131010$ juju version
1.16.0-saucy-amd64

[Andrew Wilkins (axwalk)]

@itzg: this bug/fix is specific to the manual provider, so if you're seeing it in MAAS it's certainly something different (though perhaps very similar?)

Please do file another bug.