Comment 10 for bug 161047

If I understand correctly your problem, I think you can use the ssh_config setting 'PreferredAuthentications while using ssh (ssh -o PreferredAuthentications password) to have it prefer the password over the key when you don't want it to use the key. Another option is to disable the ssh-agent bg process which caches your keys and will use them automatically, if this is disabled, then I'm pretty sure you'll need to explicitly set the key with the -i command.

Hope this helps.