mongodb-server crashes on kernel 3.13.0-66

[Dennis Dmitriev]

When puppet execute mongo.pp, node becomes inaccessible, with crash dump on VNC console.

Steps to reproduce:

1) Create Ubuntu cluster
2) Add base-os role using Fuel with default repositories
3) SSH to base-os node and install 'mongodb-server' package
4) Reboot the base-os node

Result:
 - When node is booting (or when 'mongod' process is started), the 'mongod' process is crashed (see screenshots next comments) and hungs as a zombie process:

root@node-2:~# ps auxf|grep mong
root 6024 0.0 0.0 10460 920 pts/5 S+ 12:27 0:00 \_ grep --color=auto mong
mongodb 4338 0.0 0.0 0 0 ? Zsl 12:23 0:00 [mongod] <defunct>

 - Few minutes later, node could completely hung.

Packages version:
mongodb-server 1:2.6.10-0u~u14.04+mos2
linux-image-generic 3.13.0.66.72

---------------------------------------------------

After downgrading Linux kernel to 3.13.0.65, 'mongod' start working fine:

linux-image-generic 3.13.0.65.71

Reproduced on CI with all system tests that use 'mongo' roles, for example:

https://product-ci.infra.mirantis.net/job/8.0.system_test.ubuntu.error_node_reinstallation/14/testReport/%28root%29/reinstall_failed_regular_controller_deployment/
https://product-ci.infra.mirantis.net/job/8.0.system_test.ubuntu.services_ha.ceilometer/14/testReport/%28root%29/deploy_ceilometer_ha_with_mongo/

[root@nailgun ~]# fuel --fuel-version
DEPRECATION WARNING: /etc/fuel/client/config.yaml exists and will be used as the source for settings. This behavior is deprecated. Please specify the path to your custom settings file in the FUELCLIENT_CUSTOM_SETTINGS environment variable.
api: '1.0'
astute_sha: 0f753467a3f16e4d46e7e9f1979905fb178e4d5b
auth_required: true
build_id: '134'
build_number: '134'
feature_groups:
- mirantis
fuel-agent_sha: e881f0dabd09af4be4f3e22768b02fe76278e20e
fuel-createmirror_sha: df6a93f7e2819d3dfa600052b0f901d9594eb0db
fuel-library_sha: 14b6b2d55840f5271693e26a6544cc43c91989c4
fuel-nailgun-agent_sha: d66f188a1832a9c23b04884a14ef00fc5605ec6d
fuel-nailgun_sha: a0f7a4d5dfc8a78846fe57fa6245829643167505
fuel-ostf_sha: 41aa5059243cbb25d7a80b97f8e1060a502b99dd
fuelmain_sha: 01c8a575e342f867552075f136826e9b39cda631
openstack_version: 2015.1.0-8.0
production: docker
python-fuelclient_sha: a8298df7de84960f32dccb1cd070255d5828a8ea
release: '8.0'
release_versions:
  2015.1.0-8.0:
    VERSION:
      api: '1.0'
      astute_sha: 0f753467a3f16e4d46e7e9f1979905fb178e4d5b
      build_id: '134'
      build_number: '134'
      feature_groups:
      - mirantis
      fuel-agent_sha: e881f0dabd09af4be4f3e22768b02fe76278e20e
      fuel-createmirror_sha: df6a93f7e2819d3dfa600052b0f901d9594eb0db
      fuel-library_sha: 14b6b2d55840f5271693e26a6544cc43c91989c4
      fuel-nailgun-agent_sha: d66f188a1832a9c23b04884a14ef00fc5605ec6d
      fuel-nailgun_sha: a0f7a4d5dfc8a78846fe57fa6245829643167505
      fuel-ostf_sha: 41aa5059243cbb25d7a80b97f8e1060a502b99dd
      fuelmain_sha: 01c8a575e342f867552075f136826e9b39cda631
      openstack_version: 2015.1.0-8.0
      production: docker
      python-fuelclient_sha: a8298df7de84960f32dccb1cd070255d5828a8ea
      release: '8.0'

[Dennis Dmitriev (ddmitriev)]

Crash while node is booting.

[Dennis Dmitriev (ddmitriev)]

Crash while node idle for 5-10 min (node becomes inaccessible)

[Dennis Dmitriev (ddmitriev)]

Crash at reboot. Node hung right after 'mongod' was crashed, without reboot.

[Matthew Mosesohn (raytrac3r)]

MOS Linux maintains the kernel package, not MOS Packaging team

[Artem Silenkov (asilenkov)]

I'm going to update mongodb to 3.0.6 today. You'd better wait and recheck.

[Dmitry Teselkin (teselkin-d)]

Why did you use 3.13.0-66? I can see only 3.13.0-65 in 'trusty' and 'trusty-updates' repositories (http://packages.ubuntu.com/search?keywords=linux-image-generic)

[Aleksander Mogylchenko (amogylchenko)]

Please recheck this behavior on real hardware and at least with latest 2.6.x series (which is 2.6.11). 2.6.10 was released 1.5 years ago, and something in kernel API could have changed since that time, so the best thing would be to check against 3.0.6.

[Artem Roma (aroma-x)]

No updates for more than 3 weeks, move it to invalid