Cannot create cluster using Spark Ubuntu images

[mj]

When attempting to create a cluster it fails with a keytool error:

Creating cluster failed for the following reason(s): An error occurred in thread 'configure-ssl-cert-35092d42-9149-41f2-9c83-6765df9bf004': RemoteCommandException: Error during command execution: "sudo su - -c "keytool -import -alias sahara-0 -keystore `cut -f2 -d \"=\" /etc/profile.d/99-java.sh | head -1`/lib/security/cacerts -file /tmp/cert.pem -noprompt -storepass changeit"" Return code: 1 STDERR: stdin: is not a tty STDOUT: keytool error: java.lang.Exception: Input not an X.509 certificate Error ID: fd44fc55-ab1f-4e20-985f-edcb432af353 Error ID: 428aebc9-8faf-48cd-bdef-352a204c92ee Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/sahara/context.py", line 172, in _wrapper func(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/sahara/swift/swift_helper.py", line 101, in _install_ssl_certs r.execute_command(register_cmd % idx) File "/usr/lib/python2.7/dist-packages/sahara/utils/ssh_remote.py", line 748, in execute_command get_stderr, raise_when_error) File "/usr/lib/python2.7/dist-packages/sahara/utils/ssh_remote.py", line 820, in _run_s return self._run_with_log(func, timeout, *args, **kwargs) File "/usr/lib/python2.7/dist-packages/sahara/utils/ssh_remote.py", line 671, in _run_with_log return self._run(func, *args, **kwargs) File "/usr/lib/python2.7/dist-packages/sahara/utils/ssh_remote.py", line 816, in _run return procutils.run_in_subprocess(self.proc, func, args, kwargs) File "/usr/lib/python2.7/dist-packages/sahara/utils/procutils.py", line 57, in run_in_subprocess raise exceptions.SubprocessException(result['exception']) SubprocessException: RemoteCommandException: Error during command execution: "sudo su - -c "keytool -import -alias sahara-0 -keystore `cut -f2 -d \"=\" /etc/profile.d/99-java.sh | head -1`/lib/security/cacerts -file /tmp/cert.pem -noprompt -storepass changeit"" Return code: 1 STDERR: stdin: is not a tty STDOUT: keytool error: java.lang.Exception: Input not an X.509 certificate Error ID: fd44fc55-ab1f-4e20-985f-edcb432af353 Error ID: 428aebc9-8faf-48cd-bdef-352a204c92ee Error ID: ca5361f3-2bcc-4388-9017-ac9248478830

To replicate this add the spark image available at http://sahara-files.mirantis.com/images/upstream//mitaka/sahara-mitaka-spark-1.6.0-ubuntu.qcow2 (http://sahara-files.mirantis.com/imag...). Create two node group templates one with both the name node and spark master. The second contains both the datanode and spark worker processes. Next create a cluster template that spins up 1 instance of the template with the name node and spark master and 5 instances of the data node and spark worker template.

[Nikita Konovalov (nkonovalov)]

Could you please provide a certificate example that is being used for SSL endpoints.

The java exception suggests it uses non X.509 format.

[mj (maut)]

I am using the default settings. I cannot actually see in the available list of settings where you provide a certificate.

[Vitalii Gridnev (vgridnev)]

Seems like a duplicate of the https://bugs.launchpad.net/fuel/+bug/1650284 .

[mj (maut)]

As far as I can tell SSL is not enabled by default and as such I am not using it. The only reference I can find to this is

dfs.https.enable

which is disabled by default. Given this I don't believe it is a duplicate of https://bugs.launchpad.net/fuel/+bug/1650284.

[Vitalii Gridnev (vgridnev)]

No, as I see, you have enabled SSL for OpenStack services, so that is why you are receiving such messages. In order to have correct integration of your cluster with Swift, Sahara put certificate on cluster node ant inject that into the store of certificates. Config 'dfs.https.enable' doesn't influence on that behavior.

[mj (maut)]

Are you able to provide some more details of how to do this fix or point me towards some documentation? This is the first ever cluster I have tried to create in OpenStack.

[Vitalii Gridnev (vgridnev)]

On all controllers you can find 2 configuration values which can be described as follows:

[object_store_acces]
public_identity_ca_file = /path/to/ca/file
public_object_store_ca_file = /path/to/ca/file

Most probably these files will contain some private key as described at [0]. So, I can recommend you to copy that file, then you can remove private key from the copy. Then edit sahara configurations:

[object_store_acces]
public_identity_ca_file = /path/to/ca/file_copy
public_object_store_ca_file = /path/to/ca/file_copy

Finally restart sahara-api and sahara-engine services.

[0] https://bugs.launchpad.net/fuel/+bug/1650284

[Vitalii Gridnev (vgridnev)]

What is the particular reason to keep this duplicated bug instead of opening https://bugs.launchpad.net/fuel/+bug/1650284 to needed series?