openssl s_client behaves strangely without CAPath

[Sebastian Otaegui]

Binary package hint: openssl

1) lsb_release -rd
Description: Ubuntu 8.04.2
Release: 8.04

2) apt-cache policy openssl
openssl:
  Installed: 0.9.8g-4ubuntu3.7
  Candidate: 0.9.8g-4ubuntu3.7
  Version table:
 *** 0.9.8g-4ubuntu3.7 0
        500 http://us.archive.ubuntu.com hardy-updates/main Packages
        500 http://security.ubuntu.com hardy-security/main Packages
        100 /var/lib/dpkg/status
     0.9.8g-4ubuntu3 0
        500 http://us.archive.ubuntu.com hardy/main Packages

3) openssl s_client -connect gmail.com:443 command should look into the CA directory to verify the cert of the site.
4) example output:
Bad behaviour:
openssl s_client -quiet -connect gmail.com:443
depth=1 /C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
verify error:num=20:unable to get local issuer certificate
verify return:0
Bad behaviour:
openssl s_client -quiet -connect gmail.com:443 -CApath /dev/null
depth=2 /C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
verify return:1
depth=1 /C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
verify return:1
depth=0 /C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com
verify return:1

It looks the openssl does not honor the -CApath parameter and takes the default, but if you dont specify the -CApath it doesnt look the CA directory at all

[Kees Cook (kees)]

Thanks for the report. This is the expected behavior of the openssl commandline tool -- it does not presume to select a set of CAs by default.

[Sebastian Otaegui (feniix)]

Hello Kees,
probably not a security issue, but take a look at the behaviour of:

openssl s_client -connect www.google.com:443 -CApath /tmp | grep 'return
code'
depth=2 /C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification
Authority
verify return:1
depth=1 /C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
verify return:1
depth=0 /C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.com
verify return:1
    Verify return code: 0 (ok)

I am passing a totally bogus CApath and its not throwing any warning or
error.
Is that really the expected behaviour?

Thanks

[Kees Cook (kees)]

Hrm, this behavior is a bit weird. Using -CApath seems to set -CAfile to the the default of /etc/ssl/certs/ca-certificates.crt. Compare the outputs of:

$ echo "" | strace -f -s 1024 -e trace=file openssl s_client -connect www.google.com:443 2>&1 | egrep '^open|return code'
...
    Verify return code: 20 (unable to get local issuer certificate)
$ echo "" | strace -f -s 1024 -e trace=file openssl s_client -connect www.google.com:443 -CApath /dev/null 2>&1 | egrep '^open|return code'
...
open("/etc/ssl/certs/ca-certificates.crt", O_RDONLY) = 3
...
    Verify return code: 0 (ok)
$ echo "" | strace -f -s 1024 -e trace=file openssl s_client -connect www.google.com:443 -CAfile /dev/null 2>&1 | egrep '^open|return code'
...
open("/dev/null", O_RDONLY) = 3
...
    Verify return code: 20 (unable to get local issuer certificate)

[Sebastian Otaegui (feniix)]

I have a slightly different behaviour on karmic:

otaeguis@otaeguis-home:~$ echo "" | strace -f -s 1024 -e trace=file openssl s_client -connect www.google.com:443 -CApath /dev/null 2>&1 | egrep '^open|return code'
...
open("/usr/lib/ssl/cert.pem", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
...
open("/usr/lib/ssl/certs/7651b327.0", O_RDONLY|O_LARGEFILE) = 4
...
    Verify return code: 0 (ok)

Please note I haven't modified the standard openssl configuration.

Best regards

[Robert Clark (robert-clark)]

openssl s_client is typically used for testing / verify certificates - as it states in the man pages, this should only be used for testing.

There's no use case that I can see for using s_client without at least one CA certificate. The default behaviour of openssl in fedora is to use the system installed CA bundle, which is what any user would expect. At the very least openssl should warn you that you're attempting to connect without using any CA files.

[pdf (pdffs)]

Just blew two hours trying to work out why my certs were broken. They weren't, but OpenSSL on Debian/Ubuntu is extremely stupid.

[pdf (pdffs)]

What appears to be happening is that when CApath is set to anything, it will actually fall back to '${OPENSSLDIR}/certs' and succeed, if the required cert hashes are not found at the CApath specified on the CLI. But by default, only the CAfile codepath is activated, and the default CAfile is set to '${OPENSSLDIR}/cert.pem', which is completely useless.

If the default CAfile was set to '${OPENSSLDIR}/certs/ca-certificates.crt' at build time, things would work as expected for pretty much everyone.

[Håkon A. Hjortland (post-hakn)]

From openssl 1.0.1-4ubuntu5.12 (I hope I traced the chain of functions correctly):

apps/s_client.c:
--------------------------------------------------------------------------------
        if ((!SSL_CTX_load_verify_locations(ctx,CAfile,CApath)) ||
                (!SSL_CTX_set_default_verify_paths(ctx)))
                {
                /* BIO_printf(bio_err,"error setting default verify locations\n"); */
                ERR_print_errors(bio_err);
                /* goto end; */
                }
--------------------------------------------------------------------------------
(CAfile and CApath are the command line option values (NULL if not given).)

ssl/ssl_lib.c:
--------------------------------------------------------------------------------
int SSL_CTX_load_verify_locations(SSL_CTX *ctx, const char *CAfile,
                const char *CApath)
        {
        return(X509_STORE_load_locations(ctx->cert_store,CAfile,CApath));
        }
--------------------------------------------------------------------------------

crypto/x509/x509_d2.c:
--------------------------------------------------------------------------------
int X509_STORE_load_locations(X509_STORE *ctx, const char *file,
                const char *path)
        {
        X509_LOOKUP *lookup;

        if (file != NULL)
                {
                lookup=X509_STORE_add_lookup(ctx,X509_LOOKUP_file());
                if (lookup == NULL) return(0);
                if (X509_LOOKUP_load_file(lookup,file,X509_FILETYPE_PEM) != 1)
                    return(0);
                }
        if (path != NULL)
                {
                lookup=X509_STORE_add_lookup(ctx,X509_LOOKUP_hash_dir());
                if (lookup == NULL) return(0);
                if (X509_LOOKUP_add_dir(lookup,path,X509_FILETYPE_PEM) != 1)
                    return(0);
                }
        if ((path == NULL) && (file == NULL))
                return(0);
        return(1);
        }
--------------------------------------------------------------------------------

I think the problem is that (path == NULL) && (file == NULL) is treated as an error. That causes the s_client code to abort before it calls SSL_CTX_set_default_verify_paths. If (file != NULL) or (path != NULL) and no other errors are produced, SSL_CTX_set_default_verify_paths will get called. That's why we observe that "-CApath /nonsense" adds the default path. Additionally, loading an arbitrary CA file will work too:
openssl s_client -quiet -CAfile /etc/ssl/certs/Visa_eCommerce_Root.pem -connect google.com:443

It seems strange that default locations are loaded even when -CAfile or -CApath is given, so in my opinion SSL_CTX_set_default_verify_paths should only be called when (CAfile == NULL) && (CApath == NULL).

[Adrien Nader (adrien)]

I'm not seeing that behaviour on a 23.04 system and I expect it to be the same since 22.04 at least. As such I'm going to mark this as Fix Released.