Activity log for bug #1574911

Date Who What changed Old value New value Message
2016-04-26 00:45:45 OwN bug added bug
2016-04-26 02:53:45 OwN vsftpd (Ubuntu): status New Invalid
2017-03-17 17:50:57 OwN vsftpd (Ubuntu): status Invalid New
2017-03-20 15:02:32 Christian Ehrhardt  vsftpd (Ubuntu): status New Confirmed
2017-03-21 15:39:31 Nish Aravamudan bug task added pam-mysql (Ubuntu)
2017-03-27 16:34:54 Joshua Powers bug added subscriber Ubuntu Server Team
2017-05-15 15:00:05 Andreas Hasenack pam-mysql (Ubuntu): status New Confirmed
2017-05-15 15:00:10 Andreas Hasenack vsftpd (Ubuntu): status Confirmed Invalid
2017-05-15 15:39:24 Andreas Hasenack bug watch added https://github.com/NigelCunningham/pam-MySQL/issues/29
2017-05-15 18:42:50 Andreas Hasenack bug watch added https://github.com/jedisct1/pure-ftpd/issues/58
2017-05-16 14:11:48 Andreas Hasenack summary vsftpd 500 oops stack smashing detected - Ubuntu 16.04 my_make_scrambled_password() is not a replacement for make_scrambled_password()
2017-05-16 20:54:13 Andreas Hasenack description Ubuntu 16.04 x64 and Ubuntu 16.04 x86 VSFTPD Version: vsftpd_3.0.3-3ubuntu2.debian When trying to use a fixed version of libpam-mysql (the one's I patched here: https://bugs.launchpad.net/ubuntu/+source/pam-mysql/+bug/1574900) with VSFTPD, authentication passes (no failed status in the /var/log/auth.log file meaning libpam-mysql is working), but then VSFTPD fails to login for a virtual user and displays the following error: Looking up localhost Trying localhost:21 Connected to localhost:21 220 Welcome to vsFTPd Server USER test 331 Please specify the password. PASS xxxx *** stack smashing detected ***: /usr/sbin/vsftpd terminated 500 OOPS: priv_sock_get_result Disconnecting from site localhost Here is my vsftpd.conf: listen=YES anonymous_enable=NO local_enable=YES write_enable=YES local_umask=0002 file_open_mode=0775 dirmessage_enable=YES xferlog_enable=YES connect_from_port_20=YES nopriv_user=ftp chroot_local_user=YES secure_chroot_dir=/var/run/vsftpd pam_service_name=vsftpd rsa_cert_file=/etc/ssl/certs/vsftpd.pem guest_enable=YES guest_username=ftp local_root=/var/www/vhosts/$USER user_sub_token=$USER virtual_use_local_privs=YES user_config_dir=/etc/vsftpd_user_conf local_max_rate=2000000 # bytes per sec, 2Mbytes per sec max_clients=50 # to avoid DOS attack, if you have a huge server, increase this.. ftpd_banner=Welcome to vsFTPd Server allow_writeable_chroot=YES seccomp_sandbox=NO Contents of /etc/pam.d/vsftpd: auth required pam_mysql.so user=ehcp passwd=MYPASSHERE host=localhost db=ehcp table=ftpaccounts usercolumn=ftpusername passwdcolumn=password crypt=2 account required pam_mysql.so user=ehcp passwd=MYPASSHERE host=localhost db=ehcp table=ftpaccounts usercolumn=ftpusername passwdcolumn=password crypt=2 Not seeing anything in vsftpd's log that is helpful or in the syslog. Same exact setup works fine in Ubuntu 14.04 when applying this patch in VSFTPD: http://askubuntu.com/questions/126625/libgcc-s-so-1-must-be-installed-for-pthread-cancel-to-work#answer-404523 The above patch should really be included in all versions of VSFTPD for Ubuntu / Debian too. Here's hoping to smoother vsftpd package releases in newer versions of Ubuntu. This list of VSFTPD fixes per Ubuntu release will need to grow for Ubuntu 16.04: http://ehcpforce.tk/faq/index.php?sid=33661&lang=en&action=artikel&cat=1&id=3&artlang=en artful libpam-mysql-0.8.0-1 pam_mysql, when crypt=2 is set in its configuration, it expects the password to be hashed according to the server-side PASSWORD() SQL function. From its README: 2 (or "mysql") = Use MySQL PASSWORD() function. It is possible that the encryption function used by PAM-MySQL is different from that of the MySQL server, as PAM-MySQL uses the function defined in MySQL's C-client API instead of using PASSWORD() SQL function in the query. pam_mysql is indeed using an incorrect hash function: it's using my_make_scrambled_password() as a replacement for make_scrambled_password() to locally hash the given password and compare it with what is stored in the database: char buf[42]; my_make_scrambled_password(buf, passwd, strlen(passwd)); vresult = strcmp(row[0], buf); row[0] is the result of the SQL query that fetches the user's password hash There are two problems with this: a) my_make_scrambled_password() writes CRYPT_MAX_PASSWORD_SIZE bytes to buf, and that's way more than 42. From the mysql source code: #define CRYPT_SALT_LENGTH 20 #define CRYPT_MAGIC_LENGTH 3 #define CRYPT_PARAM_LENGTH 13 #define SHA256_HASH_LENGTH 43 #define CRYPT_MAX_PASSWORD_SIZE (CRYPT_SALT_LENGTH + \ SHA256_HASH_LENGTH + \ CRYPT_MAGIC_LENGTH + \ CRYPT_PARAM_LENGTH) 42 is the length of the hexified hash produced by make_scrambled_password(), not my_make_scrambled_password(). b) the output of my_make_scrambled_password() is not a hex string like "*2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19", but something like "$5$9Ws#033Q.TZtI4^?X#026y@@{e2$OxTGgW3PiJUVZ/AChiJgAdIWQ2u2B8kA/hHZgqNj.y.". So even if buf had the correct size, the comparison would never match what's produced by PASSWORD() on the server side. As the documentation admitted could happen. If my_make_scrambled_password() is not found in the system mysqlclient library, pam_mysql will reimplement it, and funnily enough this reimplementation actually mimicks the desired behavior of make_scrambled_password() and produces an hexified hash compatible with the server's PASSWORD() function and with the right length of 42 bytes. So, if mysqlclient doesn't export my_make_scrambled_password(), pam_mysql will work because it will use its own implementation. But in the ubuntu case, my_make_scrambled_password() is exported and used, and leads to this bug. To reproduce this problem, setup mysql, vsftpd and libpam-mysql on artful as explained in bug #1574900.
2017-05-16 21:04:15 Andreas Hasenack description artful libpam-mysql-0.8.0-1 pam_mysql, when crypt=2 is set in its configuration, it expects the password to be hashed according to the server-side PASSWORD() SQL function. From its README: 2 (or "mysql") = Use MySQL PASSWORD() function. It is possible that the encryption function used by PAM-MySQL is different from that of the MySQL server, as PAM-MySQL uses the function defined in MySQL's C-client API instead of using PASSWORD() SQL function in the query. pam_mysql is indeed using an incorrect hash function: it's using my_make_scrambled_password() as a replacement for make_scrambled_password() to locally hash the given password and compare it with what is stored in the database: char buf[42]; my_make_scrambled_password(buf, passwd, strlen(passwd)); vresult = strcmp(row[0], buf); row[0] is the result of the SQL query that fetches the user's password hash There are two problems with this: a) my_make_scrambled_password() writes CRYPT_MAX_PASSWORD_SIZE bytes to buf, and that's way more than 42. From the mysql source code: #define CRYPT_SALT_LENGTH 20 #define CRYPT_MAGIC_LENGTH 3 #define CRYPT_PARAM_LENGTH 13 #define SHA256_HASH_LENGTH 43 #define CRYPT_MAX_PASSWORD_SIZE (CRYPT_SALT_LENGTH + \ SHA256_HASH_LENGTH + \ CRYPT_MAGIC_LENGTH + \ CRYPT_PARAM_LENGTH) 42 is the length of the hexified hash produced by make_scrambled_password(), not my_make_scrambled_password(). b) the output of my_make_scrambled_password() is not a hex string like "*2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19", but something like "$5$9Ws#033Q.TZtI4^?X#026y@@{e2$OxTGgW3PiJUVZ/AChiJgAdIWQ2u2B8kA/hHZgqNj.y.". So even if buf had the correct size, the comparison would never match what's produced by PASSWORD() on the server side. As the documentation admitted could happen. If my_make_scrambled_password() is not found in the system mysqlclient library, pam_mysql will reimplement it, and funnily enough this reimplementation actually mimicks the desired behavior of make_scrambled_password() and produces an hexified hash compatible with the server's PASSWORD() function and with the right length of 42 bytes. So, if mysqlclient doesn't export my_make_scrambled_password(), pam_mysql will work because it will use its own implementation. But in the ubuntu case, my_make_scrambled_password() is exported and used, and leads to this bug. To reproduce this problem, setup mysql, vsftpd and libpam-mysql on artful as explained in bug #1574900. artful libpam-mysql-0.8.0-1 pam_mysql, when crypt=2 is set in its configuration, it expects the password to be hashed according to the server-side PASSWORD() SQL function. From its README: 2 (or "mysql") = Use MySQL PASSWORD() function. It is possible that the encryption function used by PAM-MySQL is different from that of the MySQL server, as PAM-MySQL uses the function defined in MySQL's C-client API instead of using PASSWORD() SQL function in the query. pam_mysql is indeed using an incorrect hash function: it's using my_make_scrambled_password() as a replacement for make_scrambled_password() to locally hash the given password and compare it with what is stored in the database:   char buf[42];   my_make_scrambled_password(buf, passwd, strlen(passwd));   vresult = strcmp(row[0], buf); row[0] is the result of the SQL query that fetches the user's password hash There are two problems with this: a) my_make_scrambled_password() writes CRYPT_MAX_PASSWORD_SIZE bytes to buf, and that's way more than 42. From the mysql source code: #define CRYPT_SALT_LENGTH 20 #define CRYPT_MAGIC_LENGTH 3 #define CRYPT_PARAM_LENGTH 13 #define SHA256_HASH_LENGTH 43 #define CRYPT_MAX_PASSWORD_SIZE (CRYPT_SALT_LENGTH + \                                  SHA256_HASH_LENGTH + \                                  CRYPT_MAGIC_LENGTH + \                                  CRYPT_PARAM_LENGTH) 42 is the length of the hexified hash produced by make_scrambled_password(), not my_make_scrambled_password(). b) the output of my_make_scrambled_password() is not a hex string like "*2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19", but something like "$5$9Ws#033Q.TZtI4^?X#026y@@{e2$OxTGgW3PiJUVZ/AChiJgAdIWQ2u2B8kA/hHZgqNj.y.". So even if buf had the correct size, the comparison would never match what's produced by PASSWORD() on the server side. As the documentation admitted could happen. If my_make_scrambled_password() is not found in the system mysqlclient library, pam_mysql will reimplement it, and funnily enough this reimplementation actually mimicks the desired behavior of make_scrambled_password() and produces an hexified hash compatible with the server's PASSWORD() function and with the right length of 42 bytes. So, if mysqlclient doesn't export my_make_scrambled_password(), pam_mysql will work because it will use its own implementation. But in the ubuntu case, my_make_scrambled_password() is exported and used, and leads to this bug. To reproduce this problem, setup mysql, vsftpd and libpam-mysql on artful as explained in bug #1574900. I cannot explain why vsftpd doesn't crash in this scenario in artful: gcc's stack protector isn't triggered, nor is a segfault. In debugging I can see the buf variable getting way more than 42 bytes written to it, and if I add another stack variable next to it, it gets corrupted. But no crashes, just an authentication error.
2017-05-16 21:06:11 Andreas Hasenack description artful libpam-mysql-0.8.0-1 pam_mysql, when crypt=2 is set in its configuration, it expects the password to be hashed according to the server-side PASSWORD() SQL function. From its README: 2 (or "mysql") = Use MySQL PASSWORD() function. It is possible that the encryption function used by PAM-MySQL is different from that of the MySQL server, as PAM-MySQL uses the function defined in MySQL's C-client API instead of using PASSWORD() SQL function in the query. pam_mysql is indeed using an incorrect hash function: it's using my_make_scrambled_password() as a replacement for make_scrambled_password() to locally hash the given password and compare it with what is stored in the database:   char buf[42];   my_make_scrambled_password(buf, passwd, strlen(passwd));   vresult = strcmp(row[0], buf); row[0] is the result of the SQL query that fetches the user's password hash There are two problems with this: a) my_make_scrambled_password() writes CRYPT_MAX_PASSWORD_SIZE bytes to buf, and that's way more than 42. From the mysql source code: #define CRYPT_SALT_LENGTH 20 #define CRYPT_MAGIC_LENGTH 3 #define CRYPT_PARAM_LENGTH 13 #define SHA256_HASH_LENGTH 43 #define CRYPT_MAX_PASSWORD_SIZE (CRYPT_SALT_LENGTH + \                                  SHA256_HASH_LENGTH + \                                  CRYPT_MAGIC_LENGTH + \                                  CRYPT_PARAM_LENGTH) 42 is the length of the hexified hash produced by make_scrambled_password(), not my_make_scrambled_password(). b) the output of my_make_scrambled_password() is not a hex string like "*2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19", but something like "$5$9Ws#033Q.TZtI4^?X#026y@@{e2$OxTGgW3PiJUVZ/AChiJgAdIWQ2u2B8kA/hHZgqNj.y.". So even if buf had the correct size, the comparison would never match what's produced by PASSWORD() on the server side. As the documentation admitted could happen. If my_make_scrambled_password() is not found in the system mysqlclient library, pam_mysql will reimplement it, and funnily enough this reimplementation actually mimicks the desired behavior of make_scrambled_password() and produces an hexified hash compatible with the server's PASSWORD() function and with the right length of 42 bytes. So, if mysqlclient doesn't export my_make_scrambled_password(), pam_mysql will work because it will use its own implementation. But in the ubuntu case, my_make_scrambled_password() is exported and used, and leads to this bug. To reproduce this problem, setup mysql, vsftpd and libpam-mysql on artful as explained in bug #1574900. I cannot explain why vsftpd doesn't crash in this scenario in artful: gcc's stack protector isn't triggered, nor is a segfault. In debugging I can see the buf variable getting way more than 42 bytes written to it, and if I add another stack variable next to it, it gets corrupted. But no crashes, just an authentication error. artful libpam-mysql-0.8.0-1 TL;DR pam_mysql in artful will in the best case scenario just fail to authenticate users whose passwords were hashed with the server-side PASSWORD() SQL function. There is a buffer overflow happening, but it doesn't trigger a crash for some reason. Detailed explanation follows. pam_mysql, when crypt=2 is set in its configuration, it expects the password to be hashed according to the server-side PASSWORD() SQL function. From its README: 2 (or "mysql") = Use MySQL PASSWORD() function. It is possible that the encryption function used by PAM-MySQL is different from that of the MySQL server, as PAM-MySQL uses the function defined in MySQL's C-client API instead of using PASSWORD() SQL function in the query. pam_mysql is indeed using an incorrect hash function: it's using my_make_scrambled_password() as a replacement for make_scrambled_password() to locally hash the given password and compare it with what is stored in the database:   char buf[42];   my_make_scrambled_password(buf, passwd, strlen(passwd));   vresult = strcmp(row[0], buf); row[0] is the result of the SQL query that fetches the user's password hash There are two problems with this: a) my_make_scrambled_password() writes CRYPT_MAX_PASSWORD_SIZE bytes to buf, and that's way more than 42. From the mysql source code: #define CRYPT_SALT_LENGTH 20 #define CRYPT_MAGIC_LENGTH 3 #define CRYPT_PARAM_LENGTH 13 #define SHA256_HASH_LENGTH 43 #define CRYPT_MAX_PASSWORD_SIZE (CRYPT_SALT_LENGTH + \                                  SHA256_HASH_LENGTH + \                                  CRYPT_MAGIC_LENGTH + \                                  CRYPT_PARAM_LENGTH) 42 is the length of the hexified hash produced by make_scrambled_password(), not my_make_scrambled_password(). b) the output of my_make_scrambled_password() is not a hex string like "*2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19", but something like "$5$9Ws#033Q.TZtI4^?X#026y@@{e2$OxTGgW3PiJUVZ/AChiJgAdIWQ2u2B8kA/hHZgqNj.y.". So even if buf had the correct size, the comparison would never match what's produced by PASSWORD() on the server side. As the documentation admitted could happen. If my_make_scrambled_password() is not found in the system mysqlclient library, pam_mysql will reimplement it, and funnily enough this reimplementation actually mimicks the desired behavior of make_scrambled_password() and produces an hexified hash compatible with the server's PASSWORD() function and with the right length of 42 bytes. So, if mysqlclient doesn't export my_make_scrambled_password(), pam_mysql will work because it will use its own implementation. But in the ubuntu case, my_make_scrambled_password() is exported and used, and leads to this bug. To reproduce this problem, setup mysql, vsftpd and libpam-mysql on artful as explained in bug #1574900. I cannot explain why vsftpd doesn't crash in this scenario in artful: gcc's stack protector isn't triggered, nor is a segfault. In debugging I can see the buf variable getting way more than 42 bytes written to it, and if I add another stack variable next to it, it gets corrupted. But no crashes, just an authentication error.
2017-05-17 16:08:51 Andreas Hasenack bug watch added http://bugs.mysql.com/bug.php?id=86357
2017-09-06 19:10:17 Andreas Hasenack bug added subscriber Andreas Hasenack
2017-09-19 18:19:14 Launchpad Janitor merge proposal linked https://code.launchpad.net/~ahasenack/ubuntu/+source/pam-mysql/+git/pam-mysql/+merge/331007
2017-09-19 18:24:42 Andreas Hasenack pam-mysql (Ubuntu): assignee Andreas Hasenack (ahasenack)
2017-09-19 18:24:44 Andreas Hasenack pam-mysql (Ubuntu): status Confirmed In Progress
2017-09-19 18:24:57 Andreas Hasenack pam-mysql (Ubuntu): importance Undecided High
2017-09-20 16:28:11 Launchpad Janitor pam-mysql (Ubuntu): status In Progress Fix Released