lava-tool keyring backend fails on ec2/headless

[Deepti B. Kalakeri]

I tried submitting the job to lava using the following steps.

1) Added the authentication token to a local file ex: token_file_for_ciadmin

2) Added the following details into the keyringrc.cfg file as this is needed as a workaround for bug 917570
[backend]
default-keyring=simplekeyring.SimpleKeyring
keyring-path=/home/ubuntu/keyring-0.3/demo/

3) Used auth-add to one time authenticate using the following command:
lava-tool auth-add --token-file token_file_for_ciadmin http://<email address hidden>/lava-server/RPC2/

4) But, when I do a submit-job I get the following error:
lava-tool submit-job http://<email address hidden>/lava-server/RPC2/ test_submission.json
EXPERIMENTAL - SUBJECT TO CHANGE (See --experimental-notice for more info)
Traceback (most recent call last):
  File "/usr/bin/lava-tool", line 9, in <module>
    load_entry_point('lava-tool==0.2', 'console_scripts', 'lava-tool')()
  File "/usr/lib/pymodules/python2.6/lava_tool/dispatcher.py", line 85, in main
    run_with_dispatcher_class(LavaDispatcher)
  File "/usr/lib/pymodules/python2.6/lava_tool/dispatcher.py", line 81, in run_with_dispatcher_class
    raise SystemExit(cls().dispatch())
  File "/usr/lib/pymodules/python2.6/lava_tool/dispatcher.py", line 74, in dispatch
    return command.invoke()
  File "/usr/lib/pymodules/python2.6/lava_scheduler_tool/commands.py", line 43, in invoke
    server.scheduler.submit_job(command_text)
  File "/usr/lib/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.6/xmlrpclib.py", line 1243, in request
    headers
xmlrpclib.ProtocolError: <ProtocolError for <email address hidden>/lava-server/RPC2/: 401 UNAUTHORIZED>

This error occurs with both http/https protocol.

But, when I use the token in the submit-job, the command works
for ex: lava-tool submit-job http://ciadmin:<token_key>@validation.linaro.org/lava-server/RPC2/ test_submission.json

Thanks!!!
Deepti.

[Michael Hudson-Doyle (mwhudson)]

As hrw pointed out in another report, we need to distinguish in the
errors somehow why you're getting the 401 response: is it because there
was a username and no token, or a username and an invalid token?

[Deepti B. Kalakeri (deeptik)]

On Wed, Jan 18, 2012 at 3:37 AM, Michael Hudson-Doyle <
<email address hidden>> wrote:

> As hrw pointed out in another report, we need to distinguish in the
> errors somehow why you're getting the 401 response: is it because there
> was a username and no token, or a username and an invalid token?
>

Yes I tried submitting a job using the lava-tool submit-job with just the
username as I had already registered the token key using lava-tool auth-add
command.

>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/917671
>
> Title:
> lava-tool submit-job fails with 401 error
>
> Status in LAVA Command-line Tool:
> New
>
> Bug description:
> I tried submitting the job to lava using the following steps.
>
> 1) Added the authentication token to a local file ex:
> token_file_for_ciadmin
>
> 2) Added the following details into the keyringrc.cfg file as this is
> needed as a workaround for bug 917570
> [backend]
> default-keyring=simplekeyring.SimpleKeyring
> keyring-path=/home/ubuntu/keyring-0.3/demo/
>
> 3) Used auth-add to one time authenticate using the following command:
> lava-tool auth-add --token-file token_file_for_ciadmin
> http://<email address hidden>/lava-server/RPC2/
>
> 4) But, when I do a submit-job I get the following error:
> lava-tool submit-job
> http://<email address hidden>/lava-server/RPC2/test_submission.json
> EXPERIMENTAL - SUBJECT TO CHANGE (See --experimental-notice for more info)
> Traceback (most recent call last):
> File "/usr/bin/lava-tool", line 9, in <module>
> load_entry_point('lava-tool==0.2', 'console_scripts', 'lava-tool')()
> File "/usr/lib/pymodules/python2.6/lava_tool/dispatcher.py", line 85,
> in main
> run_with_dispatcher_class(LavaDispatcher)
> File "/usr/lib/pymodules/python2.6/lava_tool/dispatcher.py", line 81,
> in run_with_dispatcher_class
> raise SystemExit(cls().dispatch())
> File "/usr/lib/pymodules/python2.6/lava_tool/dispatcher.py", line 74,
> in dispatch
> return command.invoke()
> File "/usr/lib/pymodules/python2.6/lava_scheduler_tool/commands.py",
> line 43, in invoke
> server.scheduler.submit_job(command_text)
> File "/usr/lib/python2.6/xmlrpclib.py", line 1199, in __call__
> return self.__send(self.__name, args)
> File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
> verbose=self.__verbose
> File "/usr/lib/python2.6/xmlrpclib.py", line 1243, in request
> headers
> xmlrpclib.ProtocolError: <ProtocolError for
> <email address hidden>/lava-server/RPC2/: 401 UNAUTHORIZED>
>
>
> This error occurs with both http/https protocol.
>
> But, when I use the token in the submit-job, the command works
> for ex: lava-tool submit-job http://ciadmin:<token_key>@
> validation.linaro.org/lava-server/RPC2/ test_submission.json
>
> Thanks!!!
> Deepti.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/lava-tool/+bug/917671/+subscriptions
>

--
Thanks and Regards,
Deepti

[Данило Шеган (danilo)]

This is blocking fixing of bug #917503 for us: can it please be looked at?

[Paul Larson (pwlars)]

There was a fix that went in just yesteray for the problem of needing to get a better error message here. See https://code.launchpad.net/~mwhudson/lava-scheduler/more-helpful-401s-bug-917390/+merge/90036

Did we ever determine the cause of this failure? If not, it might help to do a mid-cycle rollout of the scheduler and retry this to see if we can get more information.

[Zygmunt Krynicki (zyga)]

mwhudson, plars: this is a different bug most likely, the error message is a nice addition apart from the fact that is still does not work. The issue is (probably) related to keyring backend in a headless ec2 environment.

Re-read what deepti said at the bottom of the report:

Quote:

This error occurs with both http/https protocol.

But, when I use the token in the submit-job, the command works
for ex: lava-tool submit-job http://ciadmin:<token_key>@validation.linaro.org/lava-server/RPC2/ test_submission.json

End Quote:

If the token is passed directly and bypassed the keyring system everything works correcly.

[Michael Hudson-Doyle (mwhudson)]

The keyringrc.cfg from the bug report says this:

[backend]
default-keyring=simplekeyring.SimpleKeyring
keyring-path=/home/ubuntu/keyring-0.3/demo/

but the one from bug 917570 has this:

[backend]
default-keyring=keyring.backend.UncryptedFileKeyring

This latter one works for me in local testing. Can you confirm what the file being used in EC2 is?

[Deepti B. Kalakeri (deeptik)]

The keyringrc.cfg on ec2 instance had default-keyring=simplekeyring.SimpleKeyring and it was failing to submit the job.
I changed the backend information to contain default-keyring=keyring.backend.UncryptedFileKeyring and now the submit-job works.

We can close this bug now. Thanks for pointing out the correct configuration.

Thanks!!!
Deepti.

[Michael Hudson-Doyle (mwhudson)]

Oh phew.