Ubuntu
procps package

Bug #1150413
Comment #0

Comment 0 for bug 1150413

Revision history for this message

Andrew Lowther (alowther) wrote on 2013-03-06:

Both ps and pgrep exit with an error like "xrealloc: realloc(1073741824) failedCannot allocate memory" if there is a process owned by a user with a large number of groups.

I suspect this was introduced with a recent kernel patch which no longer limits the number of groups returned by /proc/<pid>/status to 32. It affects Ubuntu kernel 3.2.0-38.60 and newer.

Links
* related kernel bug report - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1132789
* kernel patch - https://lists.ubuntu.com/archives/kernel-team/2013-January/024446.html
* kernel 3.2.0-38.60 changelog - https://launchpad.net/ubuntu/precise/+source/linux/3.2.0-38.60

strace ps output where it fails when it gets to the process owned by a user with a large number of groups
open("/proc/11860/status", O_RDONLY) = 6
read(6, "Name:\tsu\nState:\tS (sleeping)\nTgid:\t11860\nPid:\t11860\nPPid:\t7912\nTracerPid:\t0\nUid:\t1001\t1001\t1001\t1001\nGid:\t1351\t1351\t1351\t1351\nFDSize:\t256\nGroups:\t1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 13", 1023) = 1023
close(6) = 0
mmap(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbcc1e4c000
mremap(0x7fbcc1e4c000, 135168, 266240, MREMAP_MAYMOVE) = 0x7fbcc1e0b000
mremap(0x7fbcc1e0b000, 266240, 528384, MREMAP_MAYMOVE) = 0x7fbcc28cf000
mremap(0x7fbcc28cf000, 528384, 1052672, MREMAP_MAYMOVE) = 0x7fbcc27ce000
mremap(0x7fbcc27ce000, 1052672, 2101248, MREMAP_MAYMOVE) = 0x7fbcc1c6c000
mremap(0x7fbcc1c6c000, 2101248, 4198400, MREMAP_MAYMOVE) = 0x7fbcc186b000
mremap(0x7fbcc186b000, 4198400, 8392704, MREMAP_MAYMOVE) = 0x7fbcc106a000
mremap(0x7fbcc106a000, 8392704, 16781312, MREMAP_MAYMOVE) = 0x7fbcc0069000
mremap(0x7fbcc0069000, 16781312, 33558528, MREMAP_MAYMOVE) = 0x7fbcbe068000
mremap(0x7fbcbe068000, 33558528, 67112960, MREMAP_MAYMOVE) = 0x7fbcba067000
mremap(0x7fbcba067000, 67112960, 134221824, MREMAP_MAYMOVE) = 0x7fbcb2066000
mremap(0x7fbcb2066000, 134221824, 268439552, MREMAP_MAYMOVE) = 0x7fbca2065000
mremap(0x7fbca2065000, 268439552, 536875008, MREMAP_MAYMOVE) = 0x7fbc82064000
mremap(0x7fbc82064000, 536875008, 1073745920, MREMAP_MAYMOVE) = -1 EFAULT (Bad address)
mmap(NULL, 1073745920, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
brk(0x4133c000) = 0x1333000
mmap(NULL, 1073876992, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
open("/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 6
read(6, "0\n", 8192) = 2
close(6) = 0
mmap(NULL, 134217728, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7fbbfa042000
munmap(0x7fbbfa042000, 33284096) = 0
munmap(0x7fbc00000000, 33824768) = 0
mprotect(0x7fbbfc000000, 135168, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 1073745920, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
write(2, "xrealloc: realloc(1073741824) failed", 36xrealloc: realloc(1073741824) failed) = 36
write(2, "Cannot allocate memory\n", 23Cannot allocate memory
) = 23
exit_group(1) = ?

Steps I was able to use to reproduce the problem with all local users and groups. The number of groups needed to break ps may be different on other systems.
root@alowther-d02:~# for i in $(seq 180); do groupadd group$i ; done
root@alowther-d02:~# useradd user1
root@alowther-d02:~# su - user1 -c ps
No directory, logging in with HOME=/
PID TTY TIME CMD
5182 pts/0 00:00:00 su
5183 pts/0 00:00:00 sh
5185 pts/0 00:00:00 ps
root@alowther-d02:~# for i in $(seq 180); do adduser user1 group$i; done > /dev/null
root@alowther-d02:~# su - user1 -c ps
xrealloc: realloc(1073741824) failedCannot allocate memory

System info - I also tried using the version of procps from Raring, but it still failed
root@alowther-d02:~# lsb_release -rd
Description: Ubuntu 12.04.2 LTS
Release: 12.04
root@alowther-d02:~# uname -a
Linux alowther-d02 3.2.0-38-generic #61-Ubuntu SMP Tue Feb 19 12:18:21 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
root@alowther-d02:~# dpkg -l procps
ii procps 1:3.2.8-11ubuntu6 /proc file system utilities
root@alowther-d02:~# apt-cache policy procps
procps:
  Installed: 1:3.2.8-11ubuntu6
  Candidate: 1:3.3.3-2ubuntu5
  Version table:
     1:3.3.3-2ubuntu5 0
        500 http://aptproxy/ubuntu/ raring/main amd64 Packages
*** 1:3.2.8-11ubuntu6 0
        500 http://aptproxy/ubuntu/ precise/main amd64 Packages
        100 /var/lib/dpkg/status

Both ps and pgrep exit with an error like "xrealloc: realloc(1073741824) failedCannot allocate memory" if there is a process owned by a user with a large number of groups.

I suspect this was introduced with a recent kernel patch which no longer limits the number of groups returned by /proc/<pid>/status to 32.  It affects Ubuntu kernel 3.2.0-38.60 and newer.

Links
* related kernel bug report - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1132789
* kernel patch -  https://lists.ubuntu.com/archives/kernel-team/2013-January/024446.html
* kernel 3.2.0-38.60 changelog - https://launchpad.net/ubuntu/precise/+source/linux/3.2.0-38.60

strace ps output where it fails when it gets to the process owned by a user with a large number of groups
open("/proc/11860/status", O_RDONLY)    = 6
read(6, "Name:\tsu\nState:\tS (sleeping)\nTgid:\t11860\nPid:\t11860\nPPid:\t7912\nTracerPid:\t0\nUid:\t1001\t1001\t1001\t1001\nGid:\t1351\t1351\t1351\t1351\nFDSize:\t256\nGroups:\t1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 13", 1023) = 1023
close(6)                                = 0
mmap(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbcc1e4c000
mremap(0x7fbcc1e4c000, 135168, 266240, MREMAP_MAYMOVE) = 0x7fbcc1e0b000
mremap(0x7fbcc1e0b000, 266240, 528384, MREMAP_MAYMOVE) = 0x7fbcc28cf000
mremap(0x7fbcc28cf000, 528384, 1052672, MREMAP_MAYMOVE) = 0x7fbcc27ce000
mremap(0x7fbcc27ce000, 1052672, 2101248, MREMAP_MAYMOVE) = 0x7fbcc1c6c000
mremap(0x7fbcc1c6c000, 2101248, 4198400, MREMAP_MAYMOVE) = 0x7fbcc186b000
mremap(0x7fbcc186b000, 4198400, 8392704, MREMAP_MAYMOVE) = 0x7fbcc106a000
mremap(0x7fbcc106a000, 8392704, 16781312, MREMAP_MAYMOVE) = 0x7fbcc0069000
mremap(0x7fbcc0069000, 16781312, 33558528, MREMAP_MAYMOVE) = 0x7fbcbe068000
mremap(0x7fbcbe068000, 33558528, 67112960, MREMAP_MAYMOVE) = 0x7fbcba067000
mremap(0x7fbcba067000, 67112960, 134221824, MREMAP_MAYMOVE) = 0x7fbcb2066000
mremap(0x7fbcb2066000, 134221824, 268439552, MREMAP_MAYMOVE) = 0x7fbca2065000
mremap(0x7fbca2065000, 268439552, 536875008, MREMAP_MAYMOVE) = 0x7fbc82064000
mremap(0x7fbc82064000, 536875008, 1073745920, MREMAP_MAYMOVE) = -1 EFAULT (Bad address)
mmap(NULL, 1073745920, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
brk(0x4133c000)                         = 0x1333000
mmap(NULL, 1073876992, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
open("/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 6
read(6, "0\n", 8192)                    = 2
close(6)                                = 0
mmap(NULL, 134217728, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x7fbbfa042000
munmap(0x7fbbfa042000, 33284096)        = 0
munmap(0x7fbc00000000, 33824768)        = 0
mprotect(0x7fbbfc000000, 135168, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 1073745920, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
write(2, "xrealloc: realloc(1073741824) failed", 36xrealloc: realloc(1073741824) failed) = 36
write(2, "Cannot allocate memory\n", 23Cannot allocate memory
) = 23
exit_group(1)                           = ?

Steps I was able to use to reproduce the problem with all local users and groups.  The number of groups needed to break ps may be different on other systems.
root@alowther-d02:~# for i in $(seq 180); do groupadd group$i ; done
root@alowther-d02:~# useradd user1
root@alowther-d02:~# su - user1 -c ps
No directory, logging in with HOME=/
  PID TTY TIME CMD
 5182 pts/0 00:00:00 su
 5183 pts/0 00:00:00 sh
 5185 pts/0 00:00:00 ps
root@alowther-d02:~# for i in $(seq 180); do adduser user1 group$i; done > /dev/null
root@alowther-d02:~# su - user1 -c ps
xrealloc: realloc(1073741824) failedCannot allocate memory

System info - I also tried using the version of procps from Raring, but it still failed
root@alowther-d02:~# lsb_release -rd
Description:    Ubuntu 12.04.2 LTS
Release:        12.04
root@alowther-d02:~# uname -a
Linux alowther-d02 3.2.0-38-generic #61-Ubuntu SMP Tue Feb 19 12:18:21 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
root@alowther-d02:~# dpkg -l procps
ii  procps                                                1:3.2.8-11ubuntu6                                     /proc file system utilities
root@alowther-d02:~# apt-cache policy procps
procps:
  Installed: 1:3.2.8-11ubuntu6
  Candidate: 1:3.3.3-2ubuntu5
  Version table:
     1:3.3.3-2ubuntu5 0
        500 http://aptproxy/ubuntu/ raring/main amd64 Packages
 *** 1:3.2.8-11ubuntu6 0
        500 http://aptproxy/ubuntu/ precise/main amd64 Packages
        100 /var/lib/dpkg/status

Ubuntuprocps package

Comment 0 for bug 1150413

Ubuntu
procps package