Regression in 2.11.1-0ubuntu7.14; segfault in getservbyname

Bug #1352504 reported by Alex Vandiver on 2014-08-04
422
This bug affects 33 people
Affects Status Importance Assigned to Milestone
eglibc (Ubuntu)
Undecided
Unassigned
Lucid
Critical
Marc Deslauriers
Precise
Undecided
Unassigned
Trusty
Undecided
Unassigned
Utopic
Undecided
Unassigned

Bug Description

After taking security updates to 2.11.1-0ubuntu7.14 on Lucid, calls to getservbyname() are causing segfaults; backtrace attached. I suspect a failure in debian/patches/any/CVE-2013-4357.diff

nscd is installed and in use as a caching layer for openldap, which use used for passwd, group, and shadow but not services.

Needless to say, a security update that causes a regression which makes 'apt-get' segfault is quite unfortunate.

Please apply https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=c8fc0c91 which I believe will fix the segfault.

TJ (tj) on 2014-08-04
Changed in eglibc (Ubuntu):
status: New → In Progress
importance: Undecided → Critical
assignee: nobody → TJ (tj)
TJ (tj) wrote :

The proposed bug-fix source is currently building in my PPA at:

https://launchpad.net/~tj/+archive/ubuntu/ppa

TJ (tj) wrote :

Package fails to build due to a test suite error:

...
root@hephaestion:~/eglibc-2.11.1# cat scripts/data/localplt-i386-linux-gnu.data
libc.so: _Unwind_Find_FDE
libc.so: calloc
libc/crypt/libcrypt.so | \
          LC_ALL=C sort | \
          diff -u ../scripts/data/localplt-i386-linux-gnu.data - > /tmp/buildd/eglibc-2.11.1/build-tree/i386-libc/elf/check-localplt.out
make[3]: *** [/tmp/buildd/eglibc-2.11.1/build-tree/i386-libc/elf/check-localplt.out] Error 1
make[3]: Target `tests' not remade because of errors.
make[3]: Leaving directory `/tmp/buildd/eglibc-2.11.1/elf'
make[2]: *** [elf/tests] Error 2

libc.so: free
libc.so: malloc
libc.so: memalign
libc.so: realloc
libm.so: matherr
root@hephaestion:~/eglibc-2.11.1# cat /tmp/buildd/eglibc-2.11.1/build-tree/i386-libc/elf/check-localplt.out
--- ../scripts/data/localplt-i386-linux-gnu.data 2006-08-17 01:18:26.000000000 +0000
+++ - 2014-08-04 23:51:17.224429361 +0000
@@ -1,7 +1,9 @@
 libc.so: _Unwind_Find_FDE
 libc.so: calloc
+libc.so: feof
 libc.so: free
 libc.so: malloc
 libc.so: memalign
+libc.so: memmem
 libc.so: realloc
 libm.so: matherr

I cannot see any call on memmem or feof in the patch I've introduced, nor any commits in upstream close to this one to explain why these are being exposed.

TJ (tj) wrote :

Working from the reference at:

https://sourceware.org/glibc/wiki/Testing/Check-localplt

The test-suite failures appear to have been introduced in the patches for CVE-2014-0475 (2.11.1-0ubuntu7.14) and the patch for bindresvport (2.9-21).

# objdump -DR build-tree/i386-libc/libc.s > libc.dis
# egrep -n 'memmem@plt>' libc.dis
41049:000169d0 <memmem@plt>:
54050: 21063: e8 68 59 ff ff call 169d0 <memmem@plt>

# egrep -B 50 -n 'call.*<memmem@plt>' libc.dis | egrep '<.*>:'
54009-00020fd0 <_nl_find_locale>:

grep -rn memmem debian/patches/*
debian/patches/any/cvs-issue12092.diff:2:Subject: [PATCH] Fix strstr and memmem algorithm.
debian/patches/any/CVE-2014-0475.diff:43:+ if (__builtin_expect ((memmem (name, namelen,

# egrep -n 'feof@plt>' libc.dis
41019:00016970 <feof@plt>:
315267: fa8dc: e8 8f c0 f1 ff call 16970 <feof@plt>

# egrep -B 150 -n 'call.*<feof@plt>' libc.dis | egrep '<.*>:'
315131-000fa6c0 <bindresvport>:

grep -rn feof debian/patches/*
debian/patches/any/local-bindresvport_blacklist.diff:51:+ while (!feof (fp))

It appears feof() should be feof_unlocked().

Can't be sure about memmem().

Bart (marc-lecrosnier-enensys) wrote :

"apt-get remove nscd" solves the problem ...
Waiting for a fix

Chris Vigelius (chris-vigelius) wrote :

For all those who are hit by this and need to get their production server up again ASAP, here's an emergency workaround (downgrading libc6 to 2.11.1-0ubuntu7.13):

x86:
dpkg -i /var/cache/apt/archives/libc-bin_2.11.1-0ubuntu7.13_i386.deb &&
dpkg -i /var/cache/apt/archives/libc6_2.11.1-0ubuntu7.13_i386.deb

x64:
sudo dpkg -i /var/cache/apt/archives/libc-bin_2.11.1-0ubuntu7.13_amd64.deb &&
sudo dpkg -i /var/cache/apt/archives/libc6_2.11.1-0ubuntu7.13_amd64.deb

If you don't have the packages in /var/cache/apt/archives, you can get them from http://archive.ubuntu.com/ubuntu/pool/main/e/eglibc/

Bart Swennen (bswennen) wrote :

No need to uninstall or downgrade packages:

service nscd stop

also solves the problem for now.

Tom (2-launchpad-pertho-net) wrote :

Stopping nscd is not an option for those of us using LDAP authentication. It's not really a fix.

Blinker (blinker1985) wrote :

Same story here. As a temporary resolution i disabled nscd-daemon on all Ubuntu 10.04 servers.

Changed in eglibc (Ubuntu):
assignee: TJ (tj) → Marc Deslauriers (mdeslaur)
Changed in eglibc (Ubuntu Trusty):
status: New → Invalid
Changed in eglibc (Ubuntu Utopic):
status: In Progress → Invalid
Changed in eglibc (Ubuntu Precise):
status: New → Invalid
Changed in eglibc (Ubuntu Lucid):
importance: Undecided → Critical
Changed in eglibc (Ubuntu Utopic):
importance: Critical → Undecided
assignee: Marc Deslauriers (mdeslaur) → nobody
Changed in eglibc (Ubuntu Lucid):
assignee: nobody → Marc Deslauriers (mdeslaur)
information type: Public → Public Security
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in eglibc (Ubuntu Lucid):
status: New → Confirmed
Marc Deslauriers (mdeslaur) wrote :

I am currently preparing an updated package, and will put it in the following PPA:

https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+packages

Once it is built, as soon as someone can test it and make sure it solves the issue, I will publish it.

Does anyone have steps to reproduce the issue?

Marc Deslauriers (mdeslaur) wrote :

Is everyone that is hitting this issue running nscd with ldap?

Bart (marc-lecrosnier-enensys) wrote :

I am using nscd with ldap

Blinker (blinker1985) wrote :
Download full text (8.0 KiB)

This is nscd with LDAP.

I can reproduce the problem:

root@web:~# /etc/init.d/nscd start ; apt-get update
 * Starting Name Service Cache Daemon nscd [ OK ]
0% [Working]*** glibc detected *** /usr/lib/apt/methods/http: munmap_chunk(): invalid pointer: 0x00007f270db89458 ***
======= Backtrace: =========
/lib/libc.so.6(+0x78cc6)[0x7f270cd8ccc6]
/lib/libc.so.6(+0x125ec9)[0x7f270ce39ec9]
/lib/libc.so.6(+0x12628b)[0x7f270ce3a28b]
/lib/libc.so.6(getservbyname_r+0x183)[0x7f270ce1cde3]
/lib/libc.so.6(+0xb9ff2)[0x7f270cdcdff2]
/lib/libc.so.6(+0xbad50)[0x7f270cdced50]
/lib/libc.so.6(getaddrinfo+0x162)[0x7f270cdd18f2]
/usr/lib/apt/methods/http[0x40b337]
/usr/lib/apt/methods/http[0x406748]
/usr/lib/apt/methods/http[0x407d39]
/usr/lib/apt/methods/http[0x40a0cb]
/lib/libc.so.6(__libc_start_main+0xfd)[0x7f270cd32cad]
/usr/lib/apt/methods/http[0x4041c9]
======= Memory map: ========
00400000-0040e000 r-xp 00000000 08:01 2381348 /usr/lib/apt/methods/http
0060d000-0060e000 r--p 0000d000 08:01 2381348 /usr/lib/apt/methods/http
0060e000-0060f000 rw-p 0000e000 08:01 2381348 /usr/lib/apt/methods/http
017c1000-017e2000 rw-p 00000000 00:00 0 [heap]
7f270c90d000-7f270c90f000 r-xp 00000000 08:01 1839337 /lib/libdl-2.11.1.so
7f270c90f000-7f270cb0f000 ---p 00002000 08:01 1839337 /lib/libdl-2.11.1.so
7f270cb0f000-7f270cb10000 r--p 00002000 08:01 1839337 /lib/libdl-2.11.1.so
7f270cb10000-7f270cb11000 rw-p 00003000 08:01 1839337 /lib/libdl-2.11.1.so
7f270cb11000-7f270cb13000 r-xp 00000000 08:01 1839348 /lib/libutil-2.11.1.so
7f270cb13000-7f270cd12000 ---p 00002000 08:01 1839348 /lib/libutil-2.11.1.so
7f270cd12000-7f270cd13000 r--p 00001000 08:01 1839348 /lib/libutil-2.11.1.so
7f270cd13000-7f270cd14000 rw-p 00002000 08:01 1839348 /lib/libutil-2.11.1.so
7f270cd14000-7f270ce93000 r-xp 00000000 08:01 1839349 /lib/libc-2.11.1.so
7f270ce93000-7f270d093000 ---p 0017f000 08:01 1839349 /lib/libc-2.11.1.so
7f270d093000-7f270d097000 r--p 0017f000 08:01 1839349 /lib/libc-2.11.1.so
7f270d097000-7f270d098000 rw-p 00183000 08:01 1839349 /lib/libc-2.11.1.so
7f270d098000-7f270d09d000 rw-p 00000000 00:00 0
7f270d09d000-7f270d0b3000 r-xp 00000000 08:01 1844905 /lib/libgcc_s.so.1
7f270d0b3000-7f270d2b2000 ---p 00016000 08:01 1844905 /lib/libgcc_s.so.1
7f270d2b2000-7f270d2b3000 r--p 00015000 08:01 1844905 /lib/libgcc_s.so.1
7f270d2b3000-7f270d2b4000 rw-p 00016000 08:01 1844905 /lib/libgcc_s.so.1
7f270d2b4000-7f270d336000 r-xp 00000000 08:01 1839340 /lib/libm-2.11.1.so
7f270d336000-7f270d535000 ---p 00082000 08:01 1839340 /lib/li...

Read more...

Ro (robert-markula) wrote :

@Marc: Yes, running nscd with ldap.

Frits Letteboer (f-letteboer) wrote :

I'm using nscd with MySQL

Steve Mumford (steve-mumford) wrote :

Also using nscd with LDAP; first noticed the issue with MySQL failing

Marc Deslauriers (mdeslaur) wrote :

The amd64 packages have finished building in the following PPA:

https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+packages

Could someone who can reproduce please test them and see if they solve the issue?

Frits Letteboer (f-letteboer) wrote :

@mdeslaur this package works for me, no more crashes. Thanks!

Confirmed that fixes the problem for me.

Bobby Walker (gwaland-y) wrote :

Verified that this corrects the issue in my environments as well.

Blaine Fleming (blaine-t) wrote :

I can confirm that this update fixes the problem for me in multiple environments.

Kelly Davis (kldavis4) wrote :

This is also impacting cfagent:

sudo cfagent -v
*** glibc detected *** cfagent: free(): invalid pointer: 0x00007fe23a80b45d ***
======= Backtrace: =========
/lib/libc.so.6(+0x78cc6)[0x7fe239377cc6]
/lib/libc.so.6(cfree+0x73)[0x7fe23937e303]
/lib/libc.so.6(+0x125ec9)[0x7fe239424ec9]
/lib/libc.so.6(+0x12628b)[0x7fe23942528b]
/lib/libc.so.6(getservbyname_r+0x183)[0x7fe239407de3]
/lib/libc.so.6(getservbyname+0x7b)[0x7fe239407b5b]
cfagent[0x418cc3]
cfagent[0x40792b]
cfagent[0x40851e]
/lib/libc.so.6(__libc_start_main+0xfd)[0x7fe23931dcad]
cfagent[0x4047f9]

Blinker (blinker1985) wrote :

@mdeslaur Fix confirmed!

Bart (marc-lecrosnier-enensys) wrote :

ppa packages for amd64 tested :
fix OK !

tux (wondersir) wrote :

how are you applying this PPA? aptitude/apt are having glibc problems as well

Blaine Fleming (blaine-t) wrote :

@wondersir Stop nscd then apply the updates via aptitude/apt

Bobby Walker (gwaland-y) wrote :

@kldavis4 We've verified that stopping nscd will allow cfagent to function again.

Marc Deslauriers (mdeslaur) wrote :

Thanks for testing everyone!

As soon as all the build are finished, I'll release it as a security regression fix.

Sorry for the inconvenience.

tux (wondersir) wrote :

@Blaine Fleming - ah, thanks!
For anyone else in a panic, here's what I did.

   # /etc/init.d/nscd stop
   # sudo add-apt-repository ppa:ubuntu-security-proposed/ppa
   # aptitude update
   # aptitude safe-upgrade

and you shoudl see updates from the new ppa:

   ...
   The following packages will be upgraded:
     libc-bin libc-dev-bin libc6 libc6-dev libc6-i386 nscd
   6 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
   Need to get 12.2MB of archives. After unpacking 4,096B will be used.
   Do you want to continue? [Y/n/?] y
   Writing extended state information... Done
   ...

Ulli Horlacher (framstag) wrote :

WE, who are subscribed to this bug, know that we have to disable nscd, but
what about all others?
Their (automatic) update mechanism is defunct and they will not get
ANY security updates (including the glibc fix), until they manually stop
nscd!
This is really a BAD situation!

See also
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1352876

Bobby Walker (gwaland-y) wrote :

@framstag I'm going to assume that canonical is going to make some noise to let people know that they'll need manual intervention? Beyond spreading the word there's not much else that can be done.

TJ (tj) wrote :

Before signing off for sleep this morning I investigated how to install the new packages on a system that fails apt-get. It seems the best solution is a pair of {pre,post}.inst script stanzas keyed to the broken version only, which stop and start the services that cause the SIGSEGV whilst the package upgrade is in progress.

Marc Deslauriers (mdeslaur) wrote :

The regression will be announced on the ubuntu-security-announce mailing list.

tux (wondersir) wrote :

@Marc Deslauriers - thanks for getting this fixed so quick!

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package eglibc - 2.11.1-0ubuntu7.15

---------------
eglibc (2.11.1-0ubuntu7.15) lucid-security; urgency=medium

  * SECURITY REGRESSION: segfault when using nscd (LP: #1352504)
    - debian/patches/lp1352504.diff: don't free non-malloced memory and fix
      memory leak in nscd/nscd_getserv_r.c.
 -- Marc Deslauriers <email address hidden> Tue, 05 Aug 2014 07:57:55 -0400

Changed in eglibc (Ubuntu Lucid):
status: Confirmed → Fix Released
Frank Delin (franklin-delin) wrote :

Thanks for the patch. What do I need to do to apply it, the regular update then upgrade doesn't seem to see it

Marc Deslauriers (mdeslaur) wrote :

You need to wait until the packages are available in the archive...probably in ~20 minutes, then the regular update and dist-upgrade should work after you're stopped nscd.

Dan Woodard (dan-e-woodard) wrote :

Also using ldap authentication with nscd. Confirmed stop service nscd allows apt-get update and appears to solve the problem.
root@messina:~# uname -a
Linux messina.iqanalog.com 2.6.32-57-generic #119-Ubuntu SMP Wed Feb 19 01:01:55 UTC 2014 x86_64 GNU/Linux

Dan Woodard (dan-e-woodard) wrote :

Did the update and now with service nscd running, things seem fine. Thanks!

Scott Balneaves (sbalneav) wrote :

Confirmed 2.11.1-0ubuntu7.15 works here as well. Thanks.

Kelly Davis (kldavis4) wrote :

I am still not seeing 2.11.1-0ubuntu7.15 when I run 'apt-get install libc6' it says it is already the newest version. dpkg -l shows these 2.11.1-0ubuntu7.14 is installed:

ii libc6 2.11.1-0ubuntu7.14 Embedded GNU C Library: Shared libraries

Do we need to just keep waiting or am I missing something?

Marc Deslauriers (mdeslaur) wrote :

you need to run "apt-get update" to refresh the list of packages from the mirror, and then "apt-get dist-upgrade" to install all available updates.

Tom (2-launchpad-pertho-net) wrote :

2.11.1-0ubuntu7.15 fixes all my problems with nscd and LDAP logins. Apt works again.
Many thanks for pushing the patch out!

To post a comment you must log in.
This report contains Public Security information  Edit
Everyone can see this security related information.

Other bug subscribers

Bug attachments