If controllers are deployed with additional role 'ceph-osd', fuel-octane fails to upgrade the primary controller with unknown error:
[root@fuel ~]# octane -v upgrade-node --isolated $SEED_ID 7
DEPRECATION WARNING: /etc/fuel/client/config.yaml exists and will be used as the source for settings. This behavior is deprecated. Please specify the path to your custom settings file in the FUELCLIENT_CUSTOM_SETTINGS environment variable.
2015-10-22 14:26:25 INFO urllib3.connectionpool Starting new HTTP connection (1): 10.20.0.2
2015-10-22 14:26:25 DEBUG urllib3.connectionpool Setting read timeout to None
2015-10-22 14:26:25 DEBUG urllib3.connectionpool "GET /api/v1/version HTTP/1.1" 200 None
2015-10-22 14:26:25 DEBUG keystoneclient.auth.identity.v2 Making authentication request to http://10.20.0.2:8000/keystone/v2.0/tokens
2015-10-22 14:26:25 INFO urllib3.connectionpool Starting new HTTP connection (1): 10.20.0.2
2015-10-22 14:26:25 DEBUG urllib3.connectionpool Setting read timeout to None
2015-10-22 14:26:25 DEBUG urllib3.connectionpool "POST /keystone/v2.0/tokens HTTP/1.1" 200 None
2015-10-22 14:26:25 DEBUG keystoneclient.auth.identity.v2 Making authentication request to http://10.20.0.2:8000/keystone/v2.0/tokens
2015-10-22 14:26:25 INFO urllib3.connectionpool Starting new HTTP connection (1): 10.20.0.2
2015-10-22 14:26:25 DEBUG urllib3.connectionpool Setting read timeout to None
2015-10-22 14:26:25 DEBUG urllib3.connectionpool "POST /keystone/v2.0/tokens HTTP/1.1" 200 None
2015-10-22 14:26:25 INFO urllib3.connectionpool Starting new HTTP connection (1): 10.20.0.2
2015-10-22 14:26:25 DEBUG urllib3.connectionpool Setting read timeout to None
2015-10-22 14:26:25 DEBUG urllib3.connectionpool "GET /api/v1/nodes/7/ HTTP/1.1" 200 None
2015-10-22 14:26:25 INFO urllib3.connectionpool Starting new HTTP connection (1): 10.20.0.2
2015-10-22 14:26:25 DEBUG urllib3.connectionpool Setting read timeout to None
2015-10-22 14:26:25 DEBUG urllib3.connectionpool "GET /api/v1/clusters/2/ HTTP/1.1" 200 None
2015-10-22 14:26:25 DEBUG stevedore.extension found extension EntryPoint.parse('ceph-osd = octane.handlers.upgrade.ceph_osd:CephOsdUpgrade')
2015-10-22 14:26:25 DEBUG stevedore.extension found extension EntryPoint.parse('controller = octane.handlers.upgrade.controller:ControllerUpgrade')
2015-10-22 14:26:25 DEBUG stevedore.extension found extension EntryPoint.parse('compute = octane.handlers.upgrade.compute:ComputeUpgrade')
2015-10-22 14:26:25 DEBUG stevedore.extension found extension EntryPoint.parse('cinder = octane.handlers.upgrade.cinder:CinderUpgrade')
2015-10-22 14:26:25 DEBUG stevedore.extension found extension EntryPoint.parse('ceph-osd = octane.handlers.upgrade.ceph_osd:CephOsdUpgrade')
2015-10-22 14:26:25 DEBUG stevedore.extension found extension EntryPoint.parse('controller = octane.handlers.upgrade.controller:ControllerUpgrade')
2015-10-22 14:26:25 DEBUG stevedore.extension found extension EntryPoint.parse('compute = octane.handlers.upgrade.compute:ComputeUpgrade')
2015-10-22 14:26:25 DEBUG stevedore.extension found extension EntryPoint.parse('cinder = octane.handlers.upgrade.cinder:CinderUpgrade')
2015-10-22 14:26:25 INFO urllib3.connectionpool Starting new HTTP connection (1): 10.20.0.2
2015-10-22 14:26:25 DEBUG urllib3.connectionpool Setting read timeout to None
2015-10-22 14:26:25 DEBUG urllib3.connectionpool "GET /api/v1/clusters/3/ HTTP/1.1" 200 None
2015-10-22 14:26:25 INFO octane.util.ssh Creating new SSH connection to node 7
2015-10-22 14:26:25 DEBUG paramiko.transport starting thread (client mode): 0x354bcd0L
2015-10-22 14:26:25 INFO paramiko.transport Connected (version 2.0, client OpenSSH_6.6.1p1)
2015-10-22 14:26:25 DEBUG paramiko.transport kex algos:[<email address hidden>', u'ecdh-sha2-nistp256', u'ecdh-sha2-nistp384', u'ecdh-sha2-nistp521', u'diffie-hellman-group-exchange-sha256', u'diffie-hellman-group-exchange-sha1', u'diffie-hellman-group14-sha1', u'diffie-hellman-group1-sha1'] server key:[u'ssh-rsa', u'ssh-dss', u'ecdsa-sha2-nistp256', u'ssh-ed25519'] client encrypt:[u'aes256-ctr', u'aes192-ctr', u'aes128-ctr', u'arcfour256', u'arcfour128'] server encrypt:[u'aes256-ctr', u'aes192-ctr', u'aes128-ctr', u'arcfour256', u'arcfour128'] client mac:[u'hmac-sha2-512', u'hmac-sha2-256', u'hmac-ripemd160', u'hmac-sha1'] server mac:[u'hmac-sha2-512', u'hmac-sha2-256', u'hmac-ripemd160', u'hmac-sha1'] client compress:[u'none', <email address hidden>'] server compress:[u'none', <email address hidden>'] client lang:[u''] server lang:[u''] kex follows?False
2015-10-22 14:26:25 DEBUG paramiko.transport Ciphers agreed: local=aes128-ctr, remote=aes128-ctr
2015-10-22 14:26:25 DEBUG paramiko.transport using kex diffie-hellman-group1-sha1; server key type ssh-rsa; cipher: local aes128-ctr, remote aes128-ctr; mac: local hmac-sha1, remote hmac-sha1; compression: local none, remote none
2015-10-22 14:26:25 DEBUG paramiko.transport Switch to new keys ...
2015-10-22 14:26:25 DEBUG paramiko.transport Adding ssh-rsa host key for 10.20.0.7: 66a9487004e2814e19b23c3681ceb54d
2015-10-22 14:26:25 DEBUG paramiko.transport Trying key d9a0cf55b58f9bf8e44821194c26e75f from /root/.ssh/id_rsa
2015-10-22 14:26:25 DEBUG paramiko.transport userauth is OK
2015-10-22 14:26:25 INFO paramiko.transport Authentication (publickey) successful!
2015-10-22 14:26:25 DEBUG paramiko.transport [chan 1] Max packet in: 34816 bytes
2015-10-22 14:26:25 DEBUG paramiko.transport [chan 1] Max packet out: 32768 bytes
2015-10-22 14:26:25 INFO paramiko.transport Secsh channel 1 opened.
2015-10-22 14:26:26 DEBUG paramiko.transport [chan 1] Sesch channel 1 request ok
2015-10-22 14:26:26 DEBUG paramiko.transport [chan 1] EOF sent (1)
2015-10-22 14:26:26 INFO octane.util.subprocess Started process ceph[at node-7]: ceph health
2015-10-22 14:26:26 DEBUG paramiko.transport [chan 1] EOF received (1)
2015-10-22 14:26:26 INFO octane.util.subprocess Process ceph finished with return value 0
2015-10-22 14:26:26 DEBUG octane.util.ceph Got status: HEALTH_OK
2015-10-22 14:26:26 INFO octane.handlers Method 'prepare' not implemented in handler ControllerUpgrade
2015-10-22 14:26:26 INFO urllib3.connectionpool Starting new HTTP connection (1): 10.20.0.2
2015-10-22 14:26:26 DEBUG urllib3.connectionpool Setting read timeout to None
2015-10-22 14:26:26 DEBUG urllib3.connectionpool "GET /api/v1/nodes/7/disks/ HTTP/1.1" 200 None
2015-10-22 14:26:26 INFO urllib3.connectionpool Starting new HTTP connection (1): 10.20.0.2
2015-10-22 14:26:26 DEBUG urllib3.connectionpool Setting read timeout to None
2015-10-22 14:26:26 DEBUG urllib3.connectionpool "PUT /api/v1/nodes/7/disks/ HTTP/1.1" 200 None
2015-10-22 14:26:26 INFO urllib3.connectionpool Starting new HTTP connection (1): 10.20.0.2
2015-10-22 14:26:26 DEBUG urllib3.connectionpool Setting read timeout to None
2015-10-22 14:26:26 DEBUG urllib3.connectionpool "GET /api/v1/nodes/ HTTP/1.1" 200 None
2015-10-22 14:26:26 ERROR octane
2015-10-22 14:26:26 DEBUG paramiko.transport EOF in transport thread
