Comment 18 for bug 571093

Revision history for this message
Nigel Jones (dev-nigelj) wrote : Re: multipath + libvirtd eats away more memory over time

Some more investigation:

--technical bits-- (non tech lower down)

libvirtd debug logging (attached) shows running multipath -F/multipath -v4 does the following:

14:02:28.570: debug : udevEventHandleCallback:1363 : udev action: 'add'
14:02:28.570: info : udevGetDeviceProperty:111 : udev reports device '252:0' does not have property 'DRIVER'
14:02:28.570: info : udevGetDeviceProperty:111 : udev reports device '252:0' does not have property 'PCI_CLASS'
14:02:28.570: info : udevGetDeviceProperty:111 : udev reports device '252:0' does not have property 'INTERFACE'
14:02:28.570: info : udevGetDeviceType:1082 : Could not determine device type for device with sysfs path '252:0'

[snip]
14:02:28.592: debug : udevEventHandleCallback:1363 : udev action: 'add'
14:02:28.592: info : udevGetDeviceProperty:111 : udev reports device 'dm-0' does not have property 'DRIVER'
14:02:28.592: debug : udevGetDeviceProperty:131 : Found property key 'DEVNAME' value '/dev/dm-0' for device with sysname 'dm-0'
14:02:28.592: info : udevGetDeviceProperty:111 : udev reports device 'dm-0' does not have property 'ID_BUS'
14:02:28.592: info : udevGetDeviceProperty:111 : udev reports device 'dm-0' does not have property 'ID_SERIAL'
14:02:28.592: info : udevGetDeviceSysfsAttr:200 : udev reports device 'dm-0' does not have sysfs attr 'device/vendor'
14:02:28.592: info : udevGetDeviceSysfsAttr:200 : udev reports device 'dm-0' does not have sysfs attr 'device/model'
14:02:28.592: info : udevGetDeviceProperty:111 : udev reports device 'dm-0' does not have property 'ID_TYPE'
14:02:28.592: info : udevKludgeStorageType:912 : Could not find definitive storage type for device with sysfs path '/sys/devices/virtual/block/dm-0', trying to guess it
14:02:28.592: info : udevKludgeStorageType:924 : Could not determine storage type for device with sysfs path '/sys/devices/virtual/block/dm-0'
[snip]
14:02:28.594: debug : udevEventHandleCallback:1363 : udev action: 'remove'
14:02:28.594: info : udevRemoveOneDevice:1147 : Failed to find device to remove that has udev name '/sys/devices/virtual/bdi/252:0'
14:02:28.598: debug : udevEventHandleCallback:1363 : udev action: 'remove'
14:02:28.598: info : udevRemoveOneDevice:1147 : Failed to find device to remove that has udev name '/sys/devices/virtual/block/dm-0'

udevEventHandleCallback(...) will call udevAddOneDevice(device) in turn runs udevGetDeviceProperty(...) indirectly.

Assuming (and I'd need to check this) that udevGetDeviceProperty() returns PROPERTY_ERROR then I have a feeling we have found the problem as it does not look to have free'd the memory in the exit path - would have to check, especially if udevRemoveOneDevice also fails.

-- non technical bits --

It would be interesting to see if other peoples debug output from libvirt mimics my data. Instructions can be found at http://honk.sigxcpu.org/con/Debugging_libvirt.html

In particular instead of libvirtd -d, try:

LIBVIRT_DEBUG=1 libvirtd -v

I'm going to have a bit more of a play around soon to see if I can work this out a bit more.