Message indicator and cursor not moving in sync in summary buffer

[Tim Cross]

Current VM trunk
Current Emacs 24.0.50

When hitting 'n' or 'p' to move to the next/previous message, the message indicator i.e. '->' is moving, but the emacs cursor is not. This results in some movement problems as things appear to get out of sync and you cannot move more than one message in either direction. Hitting 'n' will move the -> to the next message. Hitting it a second time will not move it further. As soon as you hit space to display the message, the -> jumps back to where the cursor is and you see the contents of the message you started on.

Note that you can move by using the down/up arrow keys or C-n/C-p, which moves the cursor. When you hit space, the -> jumps to the line where the cursor is (in the summary window) and displays that message.

Also noticed that switching between n and p seems to jump the line the cursor is on. Will try to clarify with the blow example.

If you have a summary buffer like the following where -> is the message indicator and ^ is where the cursor is

  182 Grant XXXX Jan 17 84/3320 "Windows Oncall."
^ -> 183 Grant XXXX Jan 17 91/3603 "Oncall Exchange Calendar"
  184 N To: tcross@XX Jan 17 24/559 "a test of broken things"
  185 N insight-hpsim@Jan 17 18/607 "Webes HealthCheck End-To-End Detection Failed"
  186 N AusCERT Jan 17 162/6766 "ESB-2011.0047 - ALERT [Appliance] Citrix Access Gateway: Root compromise -

hitting 'n' will move -> to message 184, but will leave the cursor at 183. Subsequent 'n' does not move the -> to 185 and hitting p will jump -> to 182, not back to 183 as you would expect. I am running with threads enabled, but running the various vm-toggle-thread-* functions does not seem to make a difference.

I did not notice this problem when I last updated VM, but first noticed it after an update of emacs 24.0.50, so this may be related to changes in emacs. However, I have updated emacs a number of times since noticing this issue and the problem persists. I could have missed this issue after the original update as it was on a machine I mainly use for dev work and don't use the email that much - only really noticed it last night and again on my work machine this moring. Noticed that it affects both the vm trunk and my custom branch, which I merged with vm trunk last week.

Tim

[Uday Reddy (reddyuday)]

I couldn't reproduce it on a couple of Emacs24 versions I tried,
including the 20110110 build on Windows.

Can you reproduce it with the default settings of VM?

Cheers,
Uday

[Tim Cross (tcross)]

On Tue, Jan 18, 2011 at 1:48 AM, Uday Reddy <email address hidden> wrote:

> I couldn't reproduce it on a couple of Emacs24 versions I tried,
> including the 20110110 build on Windows.
>
> Can you reproduce it with the default settings of VM?
>
> Cheers,
> Uday
>
> I have reduced my configuration to the bare minimum and the problem still
persists, though it is slightly modified and this has given some more
information.

Given the below summary buffer layout (slightly edited to protect the
guilty) where -> is the message indicator and ^ is the emacs cursor

  176 Raelene xxxx Jan 17 257/14553 ""
  177 Grant xxxx Jan 17 84/3320 "Windows Oncall."
  178 Brendan xxxxxxx Jan 12 374/21860 "Unicon, CAS and Shiboleth"
^->179 R To: Brendan xxxxx Jan 17 129/4823 "Unicon, CAS and Shiboleth"
  180 D Tim xxxxx Jan 17 46/1321 "Unicon, CAS and Shiboleth"
  181 U To: Tim xxxxx, Br Jan 17 46/1321 "Unicon, CAS and Shiboleth"
  182 N Tim xxxxx Jan 17 43/1250 "emacspeak-bmk-mgr
vanished?"

If I hit 'p', both the cursor and the -> will be moved to message 178 and
the message will display correctly. Hitting 'p' a second time will move to
message 177 as expected - all seems OK. However, if I now hit 'n',
everything moves to 178 as would be expectred, a second 'n' moves to 179,
looking good. Now, hitting 'n' for a thrid time and the problem is again
evident. The -> message indicator jumps to message 181, skipping the deleted
message 180, but the cursor stays on message 179. Hitting 'n' again and
neither moves - the message indicator stays at 181 and the cursor stays at
179. If I hit 'space' to display the message, the -> message indicator jumps
back to message 179 and that message is revealed.

The problem appears to be related to the 'U' and 'N' indicator i.e. unread
and new messages. If I use C-n to move to messages 181 and 182, hitting
space then reveals the contents of these messages. As they have now been
read, the 'U' and 'N' markers are removed and now I can move up/down with p
or n.

My vm config consists only of

; General settings
(setq vm-window-configuration-file "~/.vm.windows"
      vm-crash-box "/home/tcross/Mail-VM/INBOX.crash"
      vm-folder-directory "~/Mail-VM/"
      vm-primary-inbox "/home/tcross/Mail-VM/INBOX"
      vm-spool-file-suffixes '(".spool")
      vm-spool-files '(("/home/tcross/Mail-VM/INBOX"
                        "/home/tcross/Mail-VM/NEWMAIL"
                        "/home/tcross/Mail-VM/INBOX.crash")))

; MIME decode/encode helpers
(setq vm-mime-base64-decoder-program "/usr/bin/perl"
      vm-mime-base64-decoder-switches
      '("-MMIME::Base64" "-ne" "print decode_base64($_)")
      vm-mime-base64-encoder-program "/usr/bin/perl"
      vm-mime-base64-encoder-switches
      '("-MMIME::Base64" "-ne" "print encode_base64($_)")
      vm-mime-qp-decoder-program "/usr/bin/perl"
      vm-mime-qp-decoder-switches
      '("-MMIME::QuotedPrint" "-ne" "print decode_qp($_)")
      vm-mime-qp-encoder-program "/usr/bin/perl"
      vm-mime-qp-encoder-switches
      '("-MMIME::QuotedPrint" "-ne" "print encode_qp($_)"))

Emacs version is
GNU Emacs 24.0.50.1 (i686-p...

[Uday Reddy (reddyuday)]

I still can't reproduce any problem of this kind.

Some suggestions:

- Make sure that you are not loading any add-ons.

- Send me a vm-submit-bug-report after you get the problem, so that I
can double check the settings.

- Since the behaviour changed after you changed settings, you might
try and see which settings caused a change in the behaviour.

Cheers,
Uday

[Uday Reddy (reddyuday)]

I have looked through the code, and the behaviour you are reporting
should not happen. The cursor movement in summary happens in the
function vm-set-summary-pointer. It always moves the cursor before
updating the summary-pointer. So, cursor getting stuck while the
summary-pointer moves should some very weird bug.

You can try tracing through vm-set-summary-pointer to see if you can
find anything funny going on there.

Cheers,
Uday

[Tim Cross (tcross)]

More progress.

I am now almost certain the bug is due to changes in emacs. I built emacs from sources in revsion 102818 (Mon 2011-01-10 23:34:06 -0500) and the problem goes away. So it must be something which changed between that date an last Monday 17th. I will try and narrow it down to the exact revision number and look to see what bit of code changed. Hopefully, this will shed some light on the cause and whether something needs to be fixed in the emacs sources or VM.

[Tim Cross (tcross)]

OK, have now confirmed that it is a change in emacs 24.0.50 which is causing this problem. In particular, changes introduced in revision 102854. The offending log entry is

revno: 102854 [merge]
committer: Glenn Morris <email address hidden>
branch nick: trunk
timestamp: Sat 2011-01-15 12:38:27 -0800
message:
  Merge from emacs-23 branch, up to r100382.
  Note:
  emacs-23 r100373 is rendered unnecessary by pre-existing 2010-05-20
  trunk change.

Which is interesting as it means emacs 23.3 may have similar problems.

Still need to confirm exactly the specific change within that merge set that causes the problem, but I am suspicious of the following src/Changelog entry (which is from that merge)

-2011-01-15 Martin Rudalics <email address hidden>

        * window.c (inhibit_point_swap): New variable.
        (Fselect_window): If inhibit_point_swap is nonzero, avoid swapping
        point this time.
        (Fset_window_configuration): Set inhibit_point_swap to 1 instead
        of setting selected_window to nil (Bug#7728).

More analysis required.

[Uday Reddy (reddyuday)]

That shows the importance of pinning down the Emacs version more
precisely when reporting problems with the development versions.
M-x emacs-version gives us enough information, I think.

Cheers,
Uday

[Uday Reddy (reddyuday)]

Tim Cross writes:

> timestamp: Sat 2011-01-15 12:38:27 -0800
> message:
> Merge from emacs-23 branch, up to r100382.
> Note:
> emacs-23 r100373 is rendered unnecessary by pre-existing 2010-05-20
> trunk change.
>
> Which is interesting as it means emacs 23.3 may have similar problems.

I don't exactly know what exactly the message means, but I can confirm
that the problem exists with the current pretest:

  GNU Emacs 23.2.92.1 (i386-mingw-nt5.1.2600) of 2011-01-16 on 3249CTO

So, you need to report it to the Emacs team.

I don't exactly know what the problem is. Doing a debug trace doesn't
catch it and everything works normally. So, I can't narrow it down.
You were right, however, that the problem shows up in VM when you try
to move into a message with the 'U' attribute. It works fine with
other messages.

Please copy me into the Emacs bug report so that I can contribute
whatever I can find out.

Cheers,
Uday

[Tim Cross (tcross)]

Unfortunately, M-x emacs-version is not suficient enough - at least not under the Linux version. It only reports the emacs version and does not include the revision number. Essentially, the only change after updating or compiling from different revision numbers in the current trunk is the build date i.e.

GNU Emacs 24.0.50.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.22.0) of 2011-01-16 on puma

Getting the revision number for the current working tree from bzr seems a bit tricky as well - I've not been able to work it out yet. If I'm at the head, I can just go by the bzr log output and assume its the latest entry. However, if you revert your working tree to an earlier revision, this doesn't seem to be reflected in the log (you still get the full log). I'm sure the command is there, just have'nt been able to find it.

I will submit a bug report to emacs bug list. However, first of all I'm going to have to try nd come up with a recipe to reproduce it. Without that, it won't get acted upon. Simply describing the impact on VM won't be sufficient. I suspect this will take a bit of time. If I can include a simple bit of elisp that works before the change and doesn't afterwards, then there will be a good chance someone will look at it.

Tim

[Uday Reddy (reddyuday)]

I have now an idea now what may be happening.

The difference between unseen messages and seen messages is that
vm-show-current-message is called for seen messages only in the
default setting. So, this call is covering up the problem that is
occurring with Emacs. I don't yet know how, but I will continue to
investigate.

Cheers,
Uday

PS: I see that emacs-version doesn't help. It only records the date of
the build. You would need to get the emacs version info from bzr log.
If you revert to an earlier version, then you know which version you
have reverted to, and you would need to record that.

[Uday Reddy (reddyuday)]

Dear Tim, if you are able, can you try the patches given by Martin
Rudalics in the emacs-developers list, thread "bug in 23.2.92 with
anything" starting on 15 Jan?

If not, you might till he checks in the fixes and try again.

Cheers,
Uday

[Tim Cross (tcross)]

I built the latest emacs 23, which I believe has the patches you mentioned applied and things do indeed appear to be fixed. I'm running

GNU Emacs 23.2.92.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.22.0) of 2011-01-23 on puma revision 100403, thought it looks like the patch may have been applied in revision 100392.

I also tried a fresh build of current emacs trunk and that too appears to be fixed. The version I am using is

GNU Emacs 24.0.50.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.22.0) of 2011-01-23 on puma revision 102847, though I believe the fix was put in in revision 102941.

Note that I had to do considerable re-engineering of my emacs environment to enable running both 23 and 24, plus I've now given up using pre-packaged distribution sources in favor of direct git/bzr sources. This is something I planned to do for a while - the current issue just raised it up the priority path. However, due to all the changes, I've not yet got 100% confidence that everything is working as it should, so it would be good if others can also confirm that this problem no longer exists. Would probably be good to have confirmation on the Windows platform as well.

Tim

[Uday Reddy (reddyuday)]

We won't worry about problems with the Emacs trunk in general. However, the pretest versions are an issue. But I trust that they have it in hand now. If the next pretest still has the problem, then we will return to it.