Forwarding a message breaks character encoding

[shinyblue]

I received an email (MIME, multipart, UTF-8 text specified for both HTML and Plain parts). When I click Forward Message then the encoding gets broken.

It appears that the UTF-8 has been interpreted as Latin-1 (or at least not UTF8) and 'converted' into UTF8, even though it already was, so I see (what could be) double-encoded characters.

The mail was from Yahoo: X-Mailer: YahooMailWebService/0.8.140.532

The message displays fine. If I hit Reply, that's fine. If I start a new message and copy-and-paste, that works properly. It appears to be just the Forward button that's broken.

Both Forward and Shift-Forward exibit the same behaviour.

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: thunderbird 17.0.4+build1-0ubuntu0.12.04.1
ProcVersionSignature: Ubuntu 3.2.0-39.62-generic 3.2.39
Uname: Linux 3.2.0-39-generic x86_64
NonfreeKernelModules: wl
AddonCompatCheckDisabled: False
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.24.
ApportVersion: 2.0.1-0ubuntu17.1
Architecture: amd64
ArecordDevices:
 **** List of CAPTURE Hardware Devices ****
 card 0: PCH [HDA Intel PCH], device 0: ALC269VB Analog [ALC269VB Analog]
   Subdevices: 1/1
   Subdevice #0: subdevice #0
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: rich 1995 F.... pulseaudio
 /dev/snd/pcmC0D0p: rich 1995 F...m pulseaudio
BuildID: 20130308132820
Card0.Amixer.info:
 Card hw:0 'PCH'/'HDA Intel PCH at 0xc0600000 irq 43'
   Mixer name : 'Intel CougarPoint HDMI'
   Components : 'HDA:10ec0269,10250513,00100100 HDA:80862805,80860101,00100000'
   Controls : 26
   Simple ctrls : 12
Channel: Unavailable
Date: Mon Apr 8 10:33:02 2013
ForcedLayersAccel: False
IfupdownConfig:
 auto lo
 iface lo inet loopback
InstallationMedia: Ubuntu 12.04 LTS "Precise Pangolin" - Release amd64 (20120425)
IpRoute:
 default via 192.168.1.1 dev eth2 proto static
 169.254.0.0/16 dev eth2 scope link metric 1000
 192.168.1.0/24 dev eth2 proto kernel scope link src 192.168.1.6 metric 2
MarkForUpload: True
MostRecentCrashID: bp-022d6a4f-83a0-49e7-bac3-3739c2120718
PrefSources: prefs.js
Profiles: Profile0 (Default) - LastVersion=17.0.4/20130308132820 (In use)
RelatedPackageVersions:
 cinnamon 1.6.7-0ubuntu1~precise1
 totem-mozilla 3.4.3-0ubuntu1~precise1
 rhythmbox-mozilla 2.97-1ubuntu1~precise1
 gnome-shell 3.4.1-0ubuntu2
RunningIncompatibleAddons: False
SourcePackage: thunderbird
SubmittedCrashIDs: bp-022d6a4f-83a0-49e7-bac3-3739c2120718
UpgradeStatus: No upgrade log present (probably fresh install)
WifiSyslog: Apr 8 10:29:42 reech wpa_supplicant[1138]: WPA: Group rekeying completed with 00:0f:b5:b0:77:a6 [GTK=TKIP]
dmi.bios.date: 03/16/2012
dmi.bios.vendor: Acer
dmi.bios.version: V1.20
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: SJV70_HR
dmi.board.vendor: Acer
dmi.board.version: Base Board Version
dmi.chassis.type: 10
dmi.chassis.vendor: Acer
dmi.chassis.version: V1.20
dmi.modalias: dmi:bvnAcer:bvrV1.20:bd03/16/2012:svnAcer:pnAspire5750:pvrV1.20:rvnAcer:rnSJV70_HR:rvrBaseBoardVersion:cvnAcer:ct10:cvrV1.20:
dmi.product.name: Aspire 5750
dmi.product.version: V1.20
dmi.sys.vendor: Acer

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[rduke15 (rduke15)]

Same problem here, with Ubuntu 12.04 and TB 38.2.0.

The received message is really UTF8, and it's headers (and multipart heders) seem correct (the sender used Apple Mail 2.2).

If I open the message in the viewer, it displays correctly, but under View->Character Encoding, TB has selected "Western" instead of "Unicode". When forwarding the message, it gets double-encoded.

If in the viewer I change View->Character Encoding to "Unicode", the message still displays correctly, as if I didn't change anything, but now forwarding also works correctly and the encoding is preserved.

It feels as if TB would ignore the mail headers and try to autodetect the encoding, getting it wrong. This auto-detected charset is then only used with forwarding, not for displaying the mail in the viewer, or for replies. Very strange.