Strange block character in long e-mail subject

[Pascal De Vuyst]

I'm using dapper 20060422 and mozilla-thunderbird 1.5-0ubuntu6.

Some e-mail subjects are shown in Thunderbird with strange block characters, see screenshot in attachement.
This seems only the case for e-mails with long subjects.

[Malcolm-smith (malcolm-smith)]

I can confirm this on Linux Thunderbird 0.7.3 (20040819).

According to RFC2822 section 2.2.3, a newline followed by whitespace should be
removed. The whitespace itself should NOT be removed.

However, certain other software seems to think that <newline><tab> should be
replaced with <space>. When Thunderbird displays such a message, it correctly
displays the tab as a small space in the message pane. But in the message list,
 it incorrectly displays an "NL" placeholder.

This does not happen with lines properly wrapped using <newline><space>, such as
are generated by Thunderbird itself.

[Kherron+mozilla (kherron+mozilla)]

*** Bug 278408 has been marked as a duplicate of this bug. ***

[Kherron+mozilla (kherron+mozilla)]

Confirming. I see this with TB 1.0 on windows XP. When a subject line is wrapped
in the message source, and the continued line is indented with a tab character,
the tab appears in the message list pane as a little filled rectangle.

When I double-click on the message to open it in a separate window, the title
bar of the window (based on the message subject) contains a little rectangle for
the tab character.

In the subject line of the preview pane or of the separate message window, the
tab appears as a wider-than-normal space.

The wider-than-normal space effect can also be reproduced in a trunk copy of TB
for solaris/gtk2, though the other two effects don't appear.

[Mcow (mcow)]

*** Bug 273096 has been marked as a duplicate of this bug. ***

[Mcow (mcow)]

Seamonkey bug 240924.

(In reply to comment #0)
> Expected Results:
> Should have said "this is atest"?

As noted in comment 1, this is not the expected result; the whitespace is
supposed to remain.

[Stewart (smjg)]

It's failing (Mozilla 2005022009 Mac OS X) even if it's just a space, not a tab. If there's one space
(which is how Mozilla itself breaks lines) then it appears as three spaces in the thread pane and
composition window, but only one space in the header of the message pane.

Should we make this part of this bug?

[Magnus Holmgren (holmgren)]

IMVHO, in this very specific case Thunderbird could break the RFC just a little
and convert such tabs to spaces, because:

1) tabs in subject lines don't do much sense anyway,
2) those black boxes are ugly, and
3) almost everyone else use tabs to indent folded lines, because
   a) in just about all other fields, tabs and spaces are equivalent, and
   b) it increases readability.

[Mcow (mcow)]

*** Bug 288660 has been marked as a duplicate of this bug. ***

[Mozilla-bakerweb (mozilla-bakerweb)]

*** Bug 310333 has been marked as a duplicate of this bug. ***

[Mcow (mcow)]

*** Bug 328536 has been marked as a duplicate of this bug. ***

[Pascal De Vuyst (pascal-devuyst)]

I'm using dapper 20060422 and mozilla-thunderbird 1.5-0ubuntu6.

Some e-mail subjects are shown in Thunderbird with strange block characters, see screenshot in attachement.
This seems only the case for e-mails with long subjects.

[Pascal De Vuyst (pascal-devuyst)]

Screenshot of Thunderbird showing strange block characters for long e-mail subjects.

[Dennis Kaarsemaker (dennis)]

Hmm, something goes wrong with tabs there... (character 0009 is a tab)

[Pascal De Vuyst (pascal-devuyst)]

I'm using Thunderbird 1.5.
The little rectangles still appear and make Thunderbird look unfinished and unprofessional.
Please convert these tabs characters by spaces as suggested by Magnus.

[Adam Conrad (adconrad)]

This happens, IIRC, on subjects that have been wrapped in the header due to line length, and they're wrapped like so:

Subject: Foo bar baz
<tab>more random stuff
<tab>and still more stuff
X-Next-Header: whatever

There's no question Tbird should be dealing with this better, but at least now you know why it's doing it.

[Pascal De Vuyst (pascal-devuyst)]

Thanks for the explanations.
This is a known problem that is more than a year old, I found the upstream mozilla bug (see above).

[Dopefishjustin (dopefishjustin)]

*** Bug 338737 has been marked as a duplicate of this bug. ***

[S-t-moz-bugzilla (s-t-moz-bugzilla)]

I am seeing this problem using Thunderbird 1.5.0.5 on Windows XP.

In my case, mailman is what is indenting the wrapped subject lines with TAB characters. mailman is very popular software!

Here is a dump of an example message header:

0002020 C c : sp nl S u b j e c t : sp [ B
0002040 u i l d b o t ] sp B u i l d B o
0002060 t sp S U C C E S S nl ht ( B l d _
0002100 g s s d k _ g s s d k _ t e s t
0002120 _ 2 0 0 6 _ 0 9 _ 1 2 _ 0 0 _ 0
0002140 1 _ 0 0 _ I n c r e m e n t a l
0002160 _ 2 ) nl X - B e e n T h e r e :

Given this issue still exists in Thunderbird 1.5.x, should the bug's version field be update to make this bug appear relevant?

[Mcow (mcow)]

*** Bug 331137 has been marked as a duplicate of this bug. ***

[Mozilla-bugs-ofcourseimright (mozilla-bugs-ofcourseimright)]

Me too. This bug leads to the odd case that when you cut and paste the subject line it does not match the subject line. Converting to a space would not violate the principle of least astonishment, IMHO.

[Cignangulo (cignangulo)]

*** Bug 359928 has been marked as a duplicate of this bug. ***

[Me-genady (me-genady)]

I have some messages with autowrapped subject and in which <nl><tab> should be removed and not replaced by a space. I don't know what the software is,
but I have this line in the headers:
X-Mailer: Internet Mail Service (5.5.2657.72)
I hope it helps.

[Mcow (mcow)]

*** Bug 363671 has been marked as a duplicate of this bug. ***

[Mcow (mcow)]

At the dupe, reporter notes that the odd character displays also in the preview text in the new message alert.

[Hilario J. Montoliu (hjmf) (hmontoliu)]

Assigned to Mozilla Team

[Chris Petersen (lists-forevermore)]

Not sure why RFC 2822 was recommended before, but RFC 822 is the one usually used for email standards. Its section 3.1.1. "LONG HEADER FIELDS" talks about multiple whitespace characters being allowed, but does clearly state that the whitespace character should be honored during unfolding.

However, the pictogram version of the tab character (a right-arrow in my case) that I see in TBird is really annoying, and given the prominent use of tabs in header fields to indent "for looks" I would personally like see Thunderbird be consistent and display them as a single space in the list view like it does in the message preview pane and "view" window.

[Philringnalda (philringnalda)]

(In reply to comment #20)
> Not sure why RFC 2822 was recommended before

Probably because the third line in 2822 is "Obsoletes: 822"

[Chris Petersen (lists-forevermore)]

That would be a good reason. :)

Either way, my understanding of this ticket is mainly that there is inconsistent behavior in TBird. The list displays the tab character visually as either a full tab or (in my case) an arrow character, and the message panes just treat it as any other old whitespace.

[Vseerror (vseerror)]

if i remember david correct, the message pane uses a slightly different source for the subject - semi related to bug 172104. perhaps these bugs could be fixed at the same time?

[Phillip Susi (psusi)]

Wow this bug is ancient. It can't be that hard to fix; I mean it's just a header parser fix to strip out the \n\t.

I can confirm that it is still annoying me in 2.0.0.6.

[Cignangulo (cignangulo)]

It might be fruitful to update in particular the 'version' label for the bug not to be ignored as out-of-date.

[Jay-hilliard (jay-hilliard)]

I can confirm another 800 users it's annoying (2.0.0.6)
It's even more annoying because we have mailman, and typically have longer Subject lines.

Please update the importance of this issue so it gets some attention! It's been years!

[Thomas (me-thomaskeller)]

Using webkit now, this long-standing annoyance is no longer a problem... ;)

[Daniel T Chen (crimsun)]

Is this symptom still reproducible in -updates/-security?

[Kevin-samson (kevin-samson)]

Still in 2.0.0.17. However this bug doesn't break threading view, even with subjects without the tab character, thank goodness.

Perhaps someone can just make a plugin to strip the subject?

[era (era)]

It's kind of fixed. See below.

vnix$ lsb_release -dr
Description: Ubuntu 8.10
Release: 8.10

vnix$ apt-cache policy thunderbird
thunderbird:
  Installed: 2.0.0.18+nobinonly-0ubuntu0.8.10.1
  Candidate: 2.0.0.18+nobinonly-0ubuntu0.8.10.1
  Version table:
 *** 2.0.0.18+nobinonly-0ubuntu0.8.10.1 0
        500 http://security.ubuntu.com intrepid-security/main Packages
        100 /var/lib/dpkg/status
     2.0.0.17+nobinonly-0ubuntu1 0
        500 http://fi.archive.ubuntu.com intrepid/main Packages

I sent myself the following test message.

vnix$ /usr/bin/sendmail -oi -t <<HERE
> From: era
> To: era
> Subject: Thunderbird
> tab test (LP#40759)
>
> I'm in ur mailz, messing ur MIMEz
> HERE

Both the list of message subjects and the actual message show a literal tab (a substantial amount of whitespace) between "Thunderbird" and "tab test". This is not quite the behavior I would have wished for (the consensus in the upstream bug seems to be that the tab should be converted to a regular space character, ASCII 32, and I would tend to agree) but the ugly block character is gone.

[Jh-mozilla (jh-mozilla)]

FWIW, the same happens when you want save an attachment with a long, folded filename, e.g.

Content-Disposition: attachment; filename="this_is_a_very_long_filename_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_xxx
 _xxxx_xxxx.xxxxxx.image"

instead of stripping the tab, the file is saved as "this_is_a_very_long_filename_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_xxx<tab>_xxxx_xxxx.xxxxxx.image", which makes no sense whatsoever...

I really hope this get's fixed soon...

[Vseerror (vseerror)]

(In reply to comment #29)
> FWIW, the same happens when you want save an attachment with a long, folded
> filename, e.g.

i believe there is a different bug on that.

[M-wada (m-wada)]

*** Bug 467097 has been marked as a duplicate of this bug. ***

[Phillip Susi (psusi)]

It is still here in thunderbird 3.0.3, though instead of showing the bad glyph trying to print the hard tab, the thread view just removes the tab, smashing the words before and after together. It should replace the tab with a space, which is what happens when you open the message in its own tab.

[Jay-hilliard (jay-hilliard)]

Good Grief, how many years until this is fixed?

[Nathan Tuggy (tuggyne)]

*** Bug 553280 has been marked as a duplicate of this bug. ***

[M-wada (m-wada)]

(In reply to comment #32)
> It is still here in thunderbird 3.0.3, though instead of showing the bad glyph
> the thread view just removes the tab, smashing the words before and after together.

It's Bug 553280(I dupe'ed to Bug 593337)/Bug 593337 for new phenomenon at thread pane with Tb 3.0(and Tb 3.1). See Bug 593337 Comment #15 for check result with trunk builds(Tb 3.2pre/Sm 2.2pre).
Next can be said.
1. This bug never occurs at thread pane after Tb 3.0(fixed by Tb 3.0),
   because Tab is removed upon Subject display at thread pane.
   And, by Tb 3.0, Bug 553280/Bug 593337 is generated.
2. Bug 553280/Bug 593337 is fixed by Tb 3.2pre/Sm 2.2 pre.

(In reply to comment #29)
> FWIW, the same happens when you want save an attachment with a long, folded
> filename, (snip)

For "Wrong Tab for folding at mid of name/filename parameter value", see Bug 593337 Comment #14, and open separate bug if you believe it's still big problem, please. <email address hidden>, please keep "one problem per a bug".

[Jay-hilliard (jay-hilliard)]

Thank you!!! I've seen that this appears to be fixed in TB 3+ (on Linux). I really appreciate you addressing this issue. Sweet!

[M-wada (m-wada)]

Closing as WORKSFORME.

[era (era)]

> status: Confirmed → Invalid

That's hardly right. The upstream status changed to RESOLVED WORKSFORME. Maybe the upstream status isn't semantically correct either, but it's not "Invalid", it's fixed.

[Phillip Susi (psusi)]

Looks like this finally got fixed.