apt-get update generates a segmentation fault

[guillaume thouvenin]

The deployment of stable/0.8 failed with the following message:

Deployment has failed. Method granular_deploy. Failed to execute hook 'lma_infrastructure_alerting-0.8.0' Failed to run command cd / && apt-get update

When we connect to the LMA node and run the command "apt-get update" manually we've got a segmentation fault. It is the same if we use the command "apt-cache stats".

If we run the command "echo 'APT::Cache-Start 50331648;' >/etc/apt/apt.conf.d/99cache-start" if fixes the problem.

[guillaume thouvenin (guillaume-thouvenin)]

After looking more deeply the problem occurs when there are too many packages in the cache and apt failed to resize the cache. By default the value is 25165824. If you go over this value the remapping of the memory produces a SIGSEVG...

mremap(0x7f47dcbd2000, 25165824, 26214400, MREMAP_MAYMOVE) = 0x7f47db2d2000
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x7f47de3c7dec} ---
+++ killed by SIGSEGV +++

[Simon Pasquier (simon-pasquier)]

I've added the Fuel project to this bug since this isn't strictly related to LMA.

The bug itself is located in apt-get and could potentially affect Fuel even when no plugin is deployed. When "apt-get update" retrieves too many packages, it will attempt to extend the allocated memory by calling mremap() which eventually fails with a SIGSEGV...

[0] http://man7.org/linux/man-pages/man2/mremap.2.html

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/242438

[Simon Pasquier (simon-pasquier)]

@Dmitry, sorry for having unassigned the bug on the Fuel project but Gerrit had assigned it to Guillaume because of the review in the LMA project :-(

[Dmitry Teselkin (teselkin-d)]

Could you please try to increate Cache-Limit instead of Cache-Start and check if it helps?

[Dmitry Klenov (dklenov)]

@Simon, NP, I have assigned it back to MOS-linux team.

[Simon Pasquier (simon-pasquier)]

@DmitryT, we tried it but it doesn't help because by default Cache-Limit is 0 (eg no limit).

[Jay Pipes (jaypipes)]

How about setting Cache-Limit to a non-zero value so that you limit the amount of memory assigned to the cache?

[Aleksander Mogylchenko (amogylchenko)]

It should not be critical if the workaround is a one line fix.

[Simon Pasquier (simon-pasquier)]

"Unfortunately" we couldn't reproduce the bug anymore... We'll re-open the bug if the issue arises again.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Guillaume Thouvenin (<email address hidden>) on branch: master
Review: https://review.openstack.org/242438
Reason: We can not reproduce the problem.

[Vyacheslav Tabolin (slavchick)]

The same bug https://bugs.launchpad.net/fuel-plugins/+bug/1514043