vRouter: Continuous rmmod and modprobe causes kernel crash/corruption

Bug #1561589 reported by Anand H. Krishnan
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R2.20
Fix Committed
Critical
Anand H. Krishnan
R2.21.x
Fix Committed
Critical
Anand H. Krishnan
R2.22.x
Fix Committed
Critical
Anand H. Krishnan
R3.0
Fix Committed
Critical
Anand H. Krishnan
Trunk
Fix Committed
Critical
Anand H. Krishnan

Bug Description

Continuous loading and unloading of vRouter module causes system to crash.

Tags: vrouter
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/18736
Submitter: Anand H. Krishnan (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/18736
Committed: http://github.org/Juniper/contrail-vrouter/commit/8337190349a263e6ce4d3f1ba85b08e646c5e727
Submitter: Zuul
Branch: R3.0

commit 8337190349a263e6ce4d3f1ba85b08e646c5e727
Author: Anand H. Krishnan <email address hidden>
Date: Thu Mar 24 21:06:04 2016 +0530

Fix memory corruption at the time of unloading

Every alloc and free of memory in vRouter is accounted for by the memory
statistics subsystem of vRouter. While unloading the module, we are
freeing malloc statistics memory before packet drop statstics memory is
freed. While we do update the memory statistics only if the memory
statistics block pointer is not null, we do not seem to be zeroing the
pointer that holds the address of the memory block post release. Hence,
reverse the order of release and NULL-ify the memory statistics pointer.

Also, unregister the net device notifier before going ahead and freeing
the memory that the vhost driver keeps. If the notifier is not unregistered
first, it can potentially access freed memory that vhost driver maintains.

Change-Id: I79c4cdec4f0501e61abf38698e44e461a90c370b
Closes-BUG: #1561589

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/18763
Submitter: Anand H. Krishnan (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.21.x

Review in progress for https://review.opencontrail.org/18804
Submitter: Anand H. Krishnan (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/18763
Committed: http://github.org/Juniper/contrail-vrouter/commit/2cef39d7d18db1d2c6bc7d9e0c49b996387198a4
Submitter: Zuul
Branch: master

commit 2cef39d7d18db1d2c6bc7d9e0c49b996387198a4
Author: Anand H. Krishnan <email address hidden>
Date: Thu Mar 24 21:06:04 2016 +0530

Fix memory corruption at the time of unloading

Every alloc and free of memory in vRouter is accounted for by the memory
statistics subsystem of vRouter. While unloading the module, we are
freeing malloc statistics memory before packet drop statstics memory is
freed. While we do update the memory statistics only if the memory
statistics block pointer is not null, we do not seem to be zeroing the
pointer that holds the address of the memory block post release. Hence,
reverse the order of release and NULL-ify the memory statistics pointer.

Also, unregister the net device notifier before going ahead and freeing
the memory that the vhost driver keeps. If the notifier is not unregistered
first, it can potentially access freed memory that vhost driver maintains.

Change-Id: I79c4cdec4f0501e61abf38698e44e461a90c370b
Closes-BUG: #1561589

Changed in juniperopenstack:
milestone: none → r3.1.0.0-fcs
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/18804
Committed: http://github.org/Juniper/contrail-vrouter/commit/781df457cb0222cdffee7de38c41938e72b430bd
Submitter: Zuul
Branch: R2.21.x

commit 781df457cb0222cdffee7de38c41938e72b430bd
Author: Anand H. Krishnan <email address hidden>
Date: Mon Mar 28 11:02:03 2016 +0530

Unregister the net device notifier before vhost exit

Unregister the net device notifier before going ahead and freeing
the memory that the vhost driver keeps. If the notifier is not
unregistered first, it can potentially access freed memory that
vhost driver maintains.

Change-Id: I2a51af25f591f0646876cc6f289df0eb03fc3655
Closes-BUG: #1561589

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.22.x

Review in progress for https://review.opencontrail.org/18929
Submitter: Anand H. Krishnan (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.20

Review in progress for https://review.opencontrail.org/18930
Submitter: Anand H. Krishnan (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/18930
Committed: http://github.org/Juniper/contrail-vrouter/commit/e3cb87715cbe715a146d7f6d17da0a69ef92e185
Submitter: Zuul
Branch: R2.20

commit e3cb87715cbe715a146d7f6d17da0a69ef92e185
Author: Anand H. Krishnan <email address hidden>
Date: Mon Mar 28 11:02:03 2016 +0530

Unregister the net device notifier before vhost exit

Unregister the net device notifier before going ahead and freeing
the memory that the vhost driver keeps. If the notifier is not
unregistered first, it can potentially access freed memory that
vhost driver maintains.

Change-Id: I2a51af25f591f0646876cc6f289df0eb03fc3655
Closes-BUG: #1561589

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/18929
Committed: http://github.org/Juniper/contrail-vrouter/commit/a90d5a54cce33c557f88cc92141b2ffee189d4c4
Submitter: Zuul
Branch: R2.22.x

commit a90d5a54cce33c557f88cc92141b2ffee189d4c4
Author: Anand H. Krishnan <email address hidden>
Date: Mon Mar 28 11:02:03 2016 +0530

Unregister the net device notifier before vhost exit

Unregister the net device notifier before going ahead and freeing
the memory that the vhost driver keeps. If the notifier is not
unregistered first, it can potentially access freed memory that
vhost driver maintains.

Change-Id: I2a51af25f591f0646876cc6f289df0eb03fc3655
Closes-BUG: #1561589

Nischal Sheth (nsheth)
information type: Proprietary → Public
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.