Space leak

[Uday Reddy]

After using VM for an hour or so, I have an Emacs session that occupies about 100MB, even after killing all buffers. An empty Emacs session should only occupy about 10MB. So, space is leaking somewhere.

[John Hein (xpqheqdvq4)]

Uday Reddy wrote at 11:05 -0000 on Mar 2, 2011:
 > Public bug reported:
 >
 > After using VM for an hour or so, I have an Emacs session that occupies
 > about 100MB, even after killing all buffers. An empty Emacs session
 > should only occupy about 10MB. So, space is leaking somewhere.

It would be great to squash this bug. Happy to test patches here.

It's probably worse with large inboxes - I just dropped from 259 MB to
55 MB after vm-quit-no-change (after running for ~12 hours).

On the other hand, killing emacs and restarting, then opening the same
inbox puts me at 240 MB, so most of that is not a leak, just emacs
with a large file open.

Hmmm, with vm rev 1099...
 fresh emacs 23.2, M-x vm => 240 MB
 M-x vm-quit-no-change, M-x vm => 252 MB
 M-x vm-quit-no-change, M-x vm => 254 MB
 M-x vm-quit-no-change, M-x vm => 256 MB

[Uday Reddy (reddyuday)]

Hi John, do you know which version of Emacs and VM allowed the space to drop back to 55 MB? Do you think the new space leak is a difference coming from Emacs or VM?

[John Hein (xpqheqdvq4)]

Uday Reddy wrote at 20:49 -0000 on Mar 2, 2011:
 > Hi John, do you know which version of Emacs and VM allowed the
 > space to drop back to 55 MB?

For my previous report, I was using:
vm rev 1099
GNU Emacs 23.2.1 (i386-portbld-freebsd7.3, GTK+ Version 2.18.7)
 (in -nw mode)

 > Do you think the new space leak is a difference coming from Emacs
 > or VM?

Not sure, but when I used 'emacs -nw big_inbox', then C-x k, C-x C-f
big_inbox, it does not seem to leak (repeated a few times).

That is, when I just use emacs to open / close the file, no leak.

But when I use vm as described before to "open & close" the file, the
2 MB leak per iteration happens.

I also tried with vm 8.1.1 and the same emacs.
It had leaks, too.

[Tim Cross (tcross)]

On Thu, Mar 3, 2011 at 9:10 AM, John Hein <email address hidden> wrote:

> Uday Reddy wrote at 20:49 -0000 on Mar 2, 2011:
> > Hi John, do you know which version of Emacs and VM allowed the
> > space to drop back to 55 MB?
>
> For my previous report, I was using:
> vm rev 1099
> GNU Emacs 23.2.1 (i386-portbld-freebsd7.3, GTK+ Version 2.18.7)
> (in -nw mode)
>
>
> > Do you think the new space leak is a difference coming from Emacs
> > or VM?
>
> Not sure, but when I used 'emacs -nw big_inbox', then C-x k, C-x C-f
> big_inbox, it does not seem to leak (repeated a few times).
>
> That is, when I just use emacs to open / close the file, no leak.
>
> But when I use vm as described before to "open & close" the file, the
> 2 MB leak per iteration happens.
>
> I also tried with vm 8.1.1 and the same emacs.
> It had leaks, too.
>
> --
> You received this bug notification because you are subscribed to VM.
> https://bugs.launchpad.net/bugs/727718
>
> Title:
> Space leak
>
> Status in VM (View Mail) for Emacs:
> Confirmed
>
> Bug description:
> After using VM for an hour or so, I have an Emacs session that
> occupies about 100MB, even after killing all buffers. An empty Emacs
> session should only occupy about 10MB. So, space is leaking
> somewhere.
>

Can I ask how you are measuring the memory usage to verify it is actually a
leak? I will test on a couple of systems, but want to make sure we are
actually comparing what we think we are comparing.

Probably also worth mentioning that on some platforms, Linux being one IIRC,
 how memory was allocated will affect how it is released. I've forgotten the
full details, but from what I recall, if the memory being allocated is
obtained via mmap (i.e. was larger than page size), it will be returned to
the system when it is freed. However, if the memory was not obttained via
mmap, it will be held by the process until the process terminates and not
released back to the system. This memory would be available to be used again
by the process, but the memory footprint of the process would not change if
you do things like kill buffers etc

Of course this is very system dependent. One useful tool under Linux is
pmap, which gives more detail than using something like ps, which only gives
course grained stats.

I will see if I can gather some stats from both my 32bit and 64bit Linux
systems. I think I've got a couple of large mailboxes that could help.

Tim

[John Hein (xpqheqdvq4)]

Tim Cross wrote at 11:56 +1100 on Mar 3, 2011:
 > Can I ask how you are measuring the memory usage to verify it is actually a
 > leak? I will test on a couple of systems, but want to make sure we are
 > actually comparing what we think we are comparing.

I used the simple method: RSS in ps(1)

I also tested on linux:
 GNU Emacs 22.3.1 (i386-redhat-linux-gnu, GTK+ Version 2.14.7)
 vm 8.0.12

It also leaked doing 'M-x vm, x' according to the RSS eyeball test.
And it seemed to be by larger amounts (like 8 MB per iteration, but I
suspect that difference is not useful info for this bug).

The 'C-x C-f, C-x C-k' test did not leak RSS on that version of emacs either.

[Uday Reddy (reddyuday)]

Notes:

Space figures for an empty session, initially, after one folder visit, and after two folder visits (for the same folder).

(vm-garbage-collect)
(:conses (102590 . 13880)
 :syms (20328 . 54)
 :miscs (830 . 152)
 :chars 570345 :vector 329747 :floats (72 . 111)
 :intervals (348 . 189)
 :strings (21259 . 5705))

(vm-garbage-collect)
(:conses (663630 . 605600)
 :syms (32257 . 2608)
 :miscs (15443 . 18022)
 :chars 1639236 :vector 454670 :floats (345 . 261)
 :intervals (503 . 622)
 :strings (99122 . 90256))

(vm-garbage-collect)
(:conses (1113362 . 334402)
 :syms (32264 . 2807)
 :miscs (15467 . 36265)
 :chars 1833568 :vector 456351 :floats (345 . 328)
 :intervals (604 . 849)
 :strings (138140 . 122806))

[Uday Reddy (reddyuday)]

John Hein writes:

> The 'C-x C-f, C-x C-k' test did not leak RSS on that version of emacs
> either.

We shouldn't compare the text-mode visits with VM visits. Emacs deals
with text and data memory differently. The text memory is given back
to the OS quickly, whereas data memory is probably never given back.
The problem is not that the data memory is retained within Emacs.
But, rather that it seems to grow over time. That is what I mean by
a space leak.

Cheers,
Uday

[Uday Reddy (reddyuday)]

The same Emacs session as above, after a few hours of usage, reached this profile:

(vm-garbage-collect)
(:conses (1558351 . 313757)
 :syms (93011 . 347)
 :miscs (140492 . 28025)
 :chars 9264181 :vector 2283330 :floats (347 . 638)
 :intervals (2771 . 4880)
 :strings (467731 . 34316))

The total memory occupied was about 400MB, and Emacs complained that "memory was exhausted."