Comment 26 for bug 505278

This isn't a bug, it's a feature. Read the gnome-keyring website carefully, https://wiki.gnome.org/Projects/GnomeKeyring/Ssh

[quote]
This assumes some familiarity with the ssh-add command. See its man page for more info.
    You can use ssh-add to manually add keys for use in the SSH agent. These will be in addition to the automatically loaded keys.
    The ssh-add -D will remove any keys you've added manually.
    The ssh-add -D will lock any automatically loaded keys.
    ssh-add -l and ssh-add -L will always list automatically loaded keys.
[/quote]

This is exactly what happens in 14.04; automatically loaded keys get locked, manually added keys get removed from the agent.

Automatically loaded keys are:
[quote]
The SSH agent automatically loads files in ~/.ssh which have corresponding *.pub paired files. Additional SSH keys can be manually loaded and managed via the ssh-add command.
[/quote]

On a side note, it seems 14.04 also starts the openssh 'ssh-agent' automatically, so effectively running two agents by default (is this intentional?). Ssh-agent stores its socket in /tmp. Try something like:

SSH_AUTH_SOCK=/tmp/ssh-ABCDEF123456/agent.12345 ssh-add