cannot bootstrap with mongodb3 and manual-provider

[Curtis Hovey]

Juju with the mongdb3 support is not able to bootstrap with mongodb3 using the packages from ppa:juju/experimental. This same juju and packages can upgrade from mongodb 2.4 to 3.2. To reproduce.

1. Provision an instance
2. Get the ip and create a manual env for it.
3. juju bootstrap --upload-tools
4. Verify bootstrap was successful.
5. ssh to the host and verify it is running mongodb 2.4
6. juju destroy-controller

7. ssh to the host and run

sudo apt-get update
sudo apt-get install -y software-properties-common
sudo apt-add-repository -y ppa:juju/experimental
sudo apt-get update
sudo apt-get install juju-mongodb3 juju-mongo-tools3

8. juju bootstrap --upload-tools
9. Verify bootstrap was successful.
10. ssh to the host and verify it is running mongodb 3.2

UPDATE
The packages do not have wiredtiger compiled in them. Juju assumes wiredtiger is available. We are changing the package to compile wiredtiger (not arm64 or ppc64el support at this time) and Juju will explicitly state is wants wiredtiger.

Upgrades work because Juju preserves the old db storage engine.

[Curtis Hovey (sinzui)]

This is the output of sudo systemctl -l status juju-db

● juju-db.service - juju state database
   Loaded: loaded (/var/lib/juju/init/juju-db/juju-db.service; enabled; vendor preset: enabled)
   Active: failed (Result: start-limit) since Thu 2016-02-11 16:04:04 UTC; 14min ago
  Process: 5490 ExecStart=/usr/lib/juju/mongo3/bin/mongod --dbpath /var/lib/juju/db --sslOnNormalPorts --sslPEMKeyFile /var/lib/juju/server.pem --sslPEMKeyPassword ignored --port 37017 --syslog --journal --replSet juju --ipv6 --quiet --oplogSize 512 --auth --keyFile /var/lib/juju/shared-secret --noprealloc --smallfiles (code=exited, status=100)
 Main PID: 5490 (code=exited, status=100)

Feb 11 16:04:04 3c3f5ed7-1165-4281-b162-34182c21cd4a systemd[1]: juju-db.service: Main process exited, code=exited, status=100/n/a
Feb 11 16:04:04 3c3f5ed7-1165-4281-b162-34182c21cd4a systemd[1]: juju-db.service: Unit entered failed state.
Feb 11 16:04:04 3c3f5ed7-1165-4281-b162-34182c21cd4a systemd[1]: juju-db.service: Failed with result 'exit-code'.
Feb 11 16:04:04 3c3f5ed7-1165-4281-b162-34182c21cd4a systemd[1]: juju-db.service: Service hold-off time over, scheduling restart.
Feb 11 16:04:04 3c3f5ed7-1165-4281-b162-34182c21cd4a systemd[1]: Stopped juju state database.
Feb 11 16:04:04 3c3f5ed7-1165-4281-b162-34182c21cd4a systemd[1]: juju-db.service: Start request repeated too quickly.
Feb 11 16:04:04 3c3f5ed7-1165-4281-b162-34182c21cd4a systemd[1]: Failed to start juju state database.
Feb 11 16:04:04 3c3f5ed7-1165-4281-b162-34182c21cd4a systemd[1]: juju-db.service: Unit entered failed state.
Feb 11 16:04:04 3c3f5ed7-1165-4281-b162-34182c21cd4a systemd[1]: juju-db.service: Failed with result 'start-limit'.

[Curtis Hovey (sinzui)]

I marked this bug medium the upgrade-mongo3 branch because I don't think this issue should block merging into master. If testing shows there are no regressions and upgrade-mongo command works, We can merge. We would then update mongo packages or juju to fix this issue.