Comment 5 for bug 374427

To give more detail, here's what your ~/.ssh/config file needs to look like:

Host *
IdentityFile id_rsa.pub
Host hostname_with_special_keys.com
IdentifyFile id_rsa.hostname_with_special_keys.pub
Host another_host.com
IdentityFile id_rsa.another_host.pub

The actual hostnames and file names are arbitrary, and do not need to match. By doing this, you won't need to specify -i on the commandline each time.