Ubuntu

Firefox can display a page with the wrong default encoding

Bug #228988 reported by André Pirard
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Mozilla Firefox
Invalid
Medium
Ubuntu
Won't Fix
Undecided
Unassigned
firefox (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

Since I'm using it, I see Firefox (2.X and 3.X) sometimes display pages using the wrong character encoding. Sometime a page will display correctly, later the same page won't, without any apparent reason why there's a difference.

Update:
------
Truly, there's something unpredictable in this behaviour of Firefox and of the presented test cases. Hence, I hereby mention a procedure that's reproducible on all my freshly installed systems but unfortunately not on all other ones (mainly US ones)
display http://atilf.atilf.fr
Click 'Entrez dans le TLF'
Fill in search field: 'erreur', click 'Valider 1'
Select the word 'ERREUR', a window pops up
Click on 'le TLF1' in that window
The pop up window invariably displays ISO8859-1 as UTF-8 as shown in
http://launchpadlibrarian.net/55706907/Screenshot-Mozilla%20Firefox.png
-----

I have first introduced Bug #206884 but, although it says exactly this, it turns out that several people misunderstood it, that it became a mess and that adding more explanation to it would make the mess even worse.
So I'm trying to make it clear and orderly in here, please read more detail in #206884.
And, obviously, make #206884 a duplicate of this one, not the opposite.

I made tests with http://atilf.atilf.fr/tlf.htm which often displays to me as shown in the attachment.
I even found a way to produce a problem.

This was Problem #1 and here is the procedure to produce it.

1) clear the said URL from history [any occurrence of the hostname in address bar dropdown]
2) open a new window or tag and do the following in it
3) set View|Character Encoding to UTF-8 (or anything but ISO889-1)
4) type that URL in the address bar and display page
5) the page displays using the wrong encoding (as the attachment, using UTF-8)

Repeat 1-5, setting ISO889-1 instead, and you get a correct display.
See an alternate procedure and more comments at the end of this text.

The obvious problem is that something done before the page displays produces an incorrect character set usage. In this case, we did it willingly but it may be the same problem that occurs unwillingly.
Obviously, all pages should contain all the information necessary to display correctly and ONLY AFTER a page was displayed should the user be able to try to display it using other encodings.

The header of that page is :
<HEAD>
<TITLE>
Le Trésor de la Langue Française Informatisé
</TITLE>
<link rel="stylesheet" type="text/css" href="atilf.css">
</HEAD>

We notice that this header doesn't specify the encoding.
As far as I know, the default encoding is ISO8858-1.
Hence, displaying the page in UTF-8 is always incorrect, unless the user changes the encoding after it was displayed.

Additional problem #2

I'll sketch it here. Please also read the bulk in #206884.

I was trying to find what could make determining the page encoding uncertain and I found this.

1) Content OptionsPreferences|Fonts & Colors|Character Encoding
The character encoding selected here will be used to display pages that do not specify which encoding to use.

This preference is unneeded, as, in my mind, the default is always ISO8859-1.
Allowing the user to change the default sure is a way to run into problems.
The Web will show you people telling others to set it back to ISO8859-1.
Yet ...

2) I tried to set this default setting to UTF-8 and, to my amazement, the test URL displayed correctly.

3) It may be that Firefox remembers the pages encodings in history (why I'm asking to clear it).
If that's a good feature once the user manually corrects the encoding for a page, it's less appropriate when a user who doesn't know how to change this encoding is hit by this occasional problem as the page encoding will be wrong for history's life instead of occasionally.

4) My discussion about how the standards badly define the encoding and the default encoding is for the Firefox developpers to notify those who write the standards.

PS:
Having spent hours trying to explain this simple problem and making tests, I just found another way to demonstrate it :

0) Use a freshly started, otherwise empty, Firefox
1) Again, clear the said URL from history [any occurrence of the hostname in address bar dropdown] as if it were the first time you use that URL
2) Use Google to search "Trésor Langue Française Informatisé", check that you get our URL
3) Right-click on link to "Open ... in new Tab"
4) the page displays using the wrong encoding (as the attachment, using UTF-8)

This is more like a user would do.
I thought that the UTF-8 setting in a freshly opened page or tab was because Google uses UTF-8, but I finally see that any empty tab ow window I open has UTF-8 set.
But, again, the problem is not what is set but that Firefox uses it as default instead of ISO8859-1.

Any page should display correctly whatever is done before it starts displaying.

See original description
Revision history for this message
André Pirard (a.pirard) wrote :
Changed in firefox:
status: New → Confirmed
Revision history for this message
John Vivirito (gnomefreak) wrote :

Please read https://wiki.ubuntu.com/MozillaTeam/Bugs/States?highlight=(mozilla) beforte setting status of mozilla bugs.

Changed in firefox:
status: Confirmed → Incomplete
Revision history for this message
John Vivirito (gnomefreak) wrote :

Forget last comment i forgot about didnt relize same author

Changed in firefox:
status: Incomplete → Confirmed
Changed in firefox-3.0:
status: New → Confirmed
Revision history for this message
John Vivirito (gnomefreak) wrote :

The reason way it was set to confirmed (the other bug) is because last night me and Alexander had a talk about your bug in IRC and he still was a little shakey on what you meant as was I but we think we got to the bottom of what you meant. Not everyone is from the same country as you are so sometimes explaining a bug more than once is needed. You say you dont have time for this or your exact words were "PLEASE READ IT CAREFULLY, it's the last time I write, I have lost enough time." Writing things more than once is needed at time as developers and bug triagers we have to write the same thing over and over again for people to grasp it sometimes. Please be patient with us as we are trying to understand your problem and fix it as needed, If we need more info please feel free to give it to us without the threat of "PLEASE READ IT CAREFULLY, it's the last time I write".

Now i still think you need to file this upstream or look for the bug upstream. I might get to it in next few weeks but i have over 1000 other bugs i have to do the same with. Next week if i remember i will talk to Alexander about what he wants to do with this as an upstream bug or our bug, as in who will be working on this us or upstream

Revision history for this message
André Pirard (a.pirard) wrote :

Ouch. I tried my second, bottom procedure again and it no longer worked.
I had repeated, though, it but it's probably not repeatable enough.
Remember the problem occurs now and then for unknown reasons.
Anyway, whatever the repeatability, the picture I sent shows something that should never happen.

Firefox probably fails to always set ISO8859-1 as the starting encoding of a page.
(which can be changed later by HTTP or inside the page)

Revision history for this message
John Vivirito (gnomefreak) wrote :

I have been trying to reproduce this and have failed to on many differnet sites. Alexander i know you are off today but can you please try the first steps to reproduce.

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] more tests

I know very little about upstream, but I have the feeling that "Affects
Mozilla Firefox" should be removed from alias Bug #206884 and set in
main Bug #228988. Thanks.

John said :
> If we need more info please feel free to give it to us without the
> threat of "PLEASE READ IT CAREFULLY, it's the last time I write".
 I meant "It's the last time I rewrite the full bug description from the
start", of course.
I continue to reply what I didn't write already.
I'm not threatening, I'm helping.
Ubuntu asks users just to report bugs and I analyze them.
I have no personal interest in this, I know how to put the displayed
encoding right, I'm doing this for those who can't, for the sake of Firefox.

I just tried the Googled-based procedure again (removed atilf from
history, restarted Firefox, shown Google search, right-clicked "Open in
tab") and there you have the picture : the new tab shows (page title)
that UTF-8 has been used to decode the page.
A page that "does not specify which encoding to use" should display with
ISO8859-1.
(and make tests only with pages that don't, of course)

Truly, there's something unpredictable in this behaviour of Firefox
regarding encoding.
A more predictable test to make is this :
- display http://atilf.atilf.fr/tlf.htm
- View|Character encoding|Unicode (UTF-8)
- restart Firefox
- type http://atilf.atilf.fr/tlf.htm in the address bar and display again
UTF-8 displayed again, and wrong :
A page that "does not specify which encoding to use" should display with
ISO8859-1.

Now, I must really do something else.

Revision history for this message
Alexander Sack (asac) wrote : Re: Firefox can display a page with the wrong encoding

in your description there is one point that makes your request that firefox should always do the right thing _without_ manual intervention void: you say "set View|Character Encoding to UTF-8" ... which is clearly a manual action and once you did that there is no real reason why firefox should not honour what you set. So, as long as you don't leave the no-manual action path everything will be fine. ONce you change it manual its ok to assume that you find that feature again to set it to auto again.

Changed in firefox-3.0:
status: Confirmed → Invalid
Changed in firefox:
status: Confirmed → Invalid
Revision history for this message
Alexander Sack (asac) wrote :

the other issue here is that it might not be obvious how to set it to auto again. thats why i retitled the other bug and thats what that bug is about.

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On 2008-05-11 23:29, Alexander Sack wrote :
> in your description there is one point that makes your request that
> firefox should always do the right thing _without_ manual intervention
> void: you say "set View|Character Encoding to UTF-8" ... which is
> clearly a manual action and once you did that there is no real reason
> why firefox should not honour what you set. So, as long as you don't
> leave the no-manual action path everything will be fine. ONce you change
> it manual its ok to assume that you find that feature again to set it to
> auto again.
>
> ** Changed in: firefox-3.0 (Ubuntu)
> Status: Confirmed => Invalid
>
 The first time that page (and some others) displayed using UTF-8 was
without "manual intervention" and on Windows if I recall well.
I set View | Encoding to ISO 8859-1 but it came back later again,
without manual intervention again, several times.
This is what persons who can read read at the beginning of my report.
I decided to open a bug because it had happened again.
Search Google with "Firefox UTF-8 8859 problème accents" you'll find
158000 hits

I have tried to reproduce the problem and I have patiently shown cases
where the wrong encoding was used, all that without manual intervention.

And now, you seem to make me a joker complaining with bad encoding
display after setting it wrong !!!!???

I just, additionally, showed that Firefox may use an incorrect encoding
because something wrong happened in the past, and I had no other way
than "manual intervention" to simulate that "someting wrong". Making all
the rest invalid because of that is strange logic.

You're ruining my work by making this report Invalid.

Furthermore, you changed someone else's report title to make it say say
what the author doesn't mean at all.
I have no problem at all understanding Character Encoding semantics,
probably because I worked with IETF task forces dealing with that.
So, if you, Mr Alexander Sack, find semantic problems, please open your
own bug report and have it say what you want to say. Let the other
people say what they want to say without trying to make believe they say
something else.
Please restore my bug title, and open your own with your title and write
what you want to say in it.

I wanted to spend my time to help Firefox but I'm loosing it.

I think iIt will be a long time before I write more than 5 lines in a
bug report.

Revision history for this message
Alexander Sack (asac) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

OK, lets give it a try, taking small steps. Maybe that works better.

When I first visit http://atilf.atilf.fr/tlf.htm in firefox 3 I get a
page the is properly displayed. ffox automatically detected windows
1252. Is that the wrong encoding? the accents look fine.

Now i go some site and and set UTF-8 explicitly (Menu -> View ->
Character Encoding -> UTF-8); then visit the site above and all works
fine.

Now I visit that site, clear the location bar, hit enter ... nothing
happens -> the browser is still on that page - its not blank or
anything, so its obvious that the user is _still_ on that page. Now i
set it to UTF-8 and the page breaks. Reloading that page will remember
your decision to mark it as UTF-8 before. I don't see whats wrong
about this.

Reopening the ffox 3 part.

 affects ubuntu/firefox-3.0
 status incomplete

If there is an issue at all it will definitly not be fixed in ffox 2
release series ...

 affects ubuntu/firefox
 status wontfix

 - Alexander

Alexander Sack (asac)
Changed in firefox:
status: Invalid → Won't Fix
Changed in firefox-3.0:
status: Invalid → Incomplete
Revision history for this message
jack (benbenny) wrote :

Dear Alex'
Please take my e-mail address from Bug Reporting List. I am already requested for UN subscribing Bug Reporting forum. Thanks. Sincerely, ben
 -------------- Original message ----------------------
From: Alexander Sack <email address hidden>
>
> OK, lets give it a try, taking small steps. Maybe that works better.
>
> When I first visit http://atilf.atilf.fr/tlf.htm in firefox 3 I get a
> page the is properly displayed. ffox automatically detected windows
> 1252. Is that the wrong encoding? the accents look fine.
>
> Now i go some site and and set UTF-8 explicitly (Menu -> View ->
> Character Encoding -> UTF-8); then visit the site above and all works
> fine.
>
> Now I visit that site, clear the location bar, hit enter ... nothing
> happens -> the browser is still on that page - its not blank or
> anything, so its obvious that the user is _still_ on that page. Now i
> set it to UTF-8 and the page breaks. Reloading that page will remember
> your decision to mark it as UTF-8 before. I don't see whats wrong
> about this.
>
> Reopening the ffox 3 part.
>
> affects ubuntu/firefox-3.0
> status incomplete
>
> If there is an issue at all it will definitly not be fixed in ffox 2
> release series ...
>
> affects ubuntu/firefox
> status wontfix
>
> - Alexander
>
> --
> Firefox can display a page with the wrong encoding
> https://bugs.launchpad.net/bugs/228988
> You received this bug notification because you are subscribed to firefox
> in ubuntu.

Revision history for this message
Alexander Sack (asac) wrote :

On Mon, May 12, 2008 at 05:29:34PM -0000, jack wrote:
> Dear Alex'
> Please take my e-mail address from Bug Reporting List. I am already requested for UN subscribing Bug Reporting forum. Thanks. Sincerely, ben

Only you can do that on your own. Visit the bugs page and hit
unsubscribe on the left side.

 - Alexander

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On 2008-05-12 17:25, Alexander Sack wrote :
> OK, lets give it a try, taking small steps. Maybe that works better.
>
> When I first visit http://atilf.atilf.fr/tlf.htm in firefox 3 I get a
> page the is properly displayed. ffox automatically detected windows
> 1252. Is that the wrong encoding? the accents look fine.
>
 You have reproduced the bug.
No, it's the wrong encoding.
In what I wrote, you can read that the encoding must be ISO 8859-1.
You don't see a display error because CP1252 is very similar to ISO 8859-1.
Why you got a CP1252 error where I never got anything else than a UTF-8
error is part of the mystery game.
> Reloading that page will remember
> your decision to mark it as UTF-8 before. I don't see whats wrong
> about this.
>
 Theoretically, it's wrong, as displaying a page must not depend on
anything in the past.
Practically, it's not inconvenient if the person who does that knows
what (s)he's doing.

But it's certainly wrong when what TB remembers is the consequence of
the bug you reproduced here above and if the victim understands nothing
about character sets to put that right, which is highly probable if the
persons they could ask write that encoding semantics are a fuzzy thing
and if they write pages to understand each other.
Remember I got that page wrong a long time myself before understanding.

In short, ISO 8859-1 must always be used to display a page that does not
specify an encoding, the user may temporarily experiment using other
encodings but if anything he does will permanently affect the display of
a page, that must be after a strong warning to make sure he understands
what he's doing and so that memorizing the result of a bug be impossible.

It looks like we made the last two steps, doesn't it?
I sincerely hope we're done.

Revision history for this message
André Pirard (a.pirard) wrote : Re: Firefox can display a page with the wrong encoding

May I know what is incomplete in this report?
You have seen the bug yourself.
Do you need more examples?
I have seen more UTF-8, but why add nothing new?

Watchful now, I've seen a page wrongly displayed with CP1252 too.
The strange thing is that when I returned to that page with the back arrow, it was then displayed using ISO8859-1.

Revision history for this message
Alexander Sack (asac) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On Thu, May 15, 2008 at 12:21:24AM -0000, André Pirard wrote:
> May I know what is incomplete in this report?
> You have seen the bug yourself.
> Do you need more examples?
> I have seen more UTF-8, but why add nothing new?
>
> Watchful now, I've seen a page wrongly displayed with CP1252 too.
> The strange thing is that when I returned to that page with the back arrow, it was then displayed using ISO8859-1.
>

CP1252 displays properly. Still don't understand what is wrong about
that?

 - Alexander

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On 2008-05-16 14:28, Alexander Sack wrote :
> CP1252 displays properly. Still don't understand what is wrong about
> that?
 OK, OK. I'll explain again and again then.
If Thunderbird displays unidentified pages with the wrong codepage, it
is a problem.
It's not because the CP1252 cases _you_ have seen produced no display
difference that it is not a problem, you may look at my picture and see
that the UTF-8 case _is_ a visible problem.
Unless, you looked at an ASCII-only display, need I say.

May I get a reply to my question?
May I know what is incomplete in this report?
Do you need anything else?

Revision history for this message
Alexander Sack (asac) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On Fri, May 16, 2008 at 02:23:29PM -0000, André Pirard wrote:
> On 2008-05-16 14:28, Alexander Sack wrote :
> > CP1252 displays properly. Still don't understand what is wrong about
> > that?
> OK, OK. I'll explain again and again then.
> If Thunderbird displays unidentified pages with the wrong codepage, it
> is a problem.
> It's not because the CP1252 cases _you_ have seen produced no display
> difference that it is not a problem, you may look at my picture and see
> that the UTF-8 case _is_ a visible problem.

This UTF-8 example is a non-issue, because you select it manually
in the menu. If you do this, you are left alone and changing the
semantic to not remember your manualy selection on next visit is not
going to be fixed.

> Unless, you looked at an ASCII-only display, need I say.
>
> May I get a reply to my question?
> May I know what is incomplete in this report?
> Do you need anything else?
>

This is not a bug. Its an explicit design decision to remember that
you _manually_ select the encoding for a site.

The auto detection does the correct thing in your example; so to get
this confirmed, provide an example where the auto-detection is broken.

 status wontfix
 title "firefox should forget manually selected page-encoding"

 - Alexander

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On 2008-05-17 10:51, Alexander Sack wrote :
> This is not a bug. Its an explicit design decision to remember that
> you _manually_ select the encoding for a site.
>
> The auto detection does the correct thing in your example; so to get
> this confirmed, provide an example where the auto-detection is broken.
>
> status wontfix
>
 I TOLD YOU THAT I SAW WRONG ENCODING DISPLAY WITOUT MANUAL INTERVERTION.
("I have tried to reproduce the problem and I have patiently shown cases
where the wrong encoding was used, all that without manual intervention.")
I TOLD YOU THAT I SAW WRONG ENCODING DISPLAY WITOUT MANUAL INTERVERTION.
I TOLD YOU THAT I SAW WRONG ENCODING DISPLAY WITOUT MANUAL INTERVERTION.
I TOLD YOU THAT I SAW WRONG ENCODING DISPLAY WITOUT MANUAL INTERVERTION.
I TOLD YOU THAT I SAW WRONG ENCODING DISPLAY WITOUT MANUAL INTERVERTION.
I TOLD YOU THAT I SAW WRONG ENCODING DISPLAY WITOUT MANUAL INTERVERTION.
I TOLD YOU THAT I SAW WRONG ENCODING DISPLAY WITOUT MANUAL INTERVERTION.
I TOLD YOU THAT I SAW WRONG ENCODING DISPLAY WITOUT MANUAL INTERVERTION.

AND YOU REPORTED YOU HAVE DONE THAT YOURSELF.
AND YOU REPORTED YOU HAVE DONE THAT YOURSELF.
AND YOU REPORTED YOU HAVE DONE THAT YOURSELF.
AND YOU REPORTED YOU HAVE DONE THAT YOURSELF.
AND YOU REPORTED YOU HAVE DONE THAT YOURSELF.
AND YOU REPORTED YOU HAVE DONE THAT YOURSELF.
AND YOU REPORTED YOU HAVE DONE THAT YOURSELF.

Now, if you say that you won't fix it, there's absolutely no reason that
I loose my time reporting this problem, try to find what happens and
help Firefox.

André.

Revision history for this message
ThiloPfennig (tpfennig) wrote : Re: Firefox can display a page with the wrong encoding

Although Andre was spamming I have to agree with the basic message. What Firefox is doing is that it uses UTF-8 as default for ANY web page, which is false. As in Germany many pages use ISO-8859-1 and many HTML pages do not specify an encoding. To start with the first example mentioned in the bug report if I access the page: http://atilf.atilf.fr/tlf.htm Firefox sets encoding to UTF-8.

I have read a bit more documents. The primary problem here is that too many pages on the web do not declare a character set.

 Secondly HTTP clearly defines that ISO-8859-1 is a falllback character set: " When no explicit charset parameter is provided by the sender, media subtypes of the "text" type are defined to have a default charset value of "ISO-8859-1" when received via HTTP." (http://tools.ietf.org/html/rfc2616#section-3.7.1 , RFC 2616 ). This had been the default character set for ages. So one can assume that if pages or servers do not send a character set, that they are old (HTML 2,3,...) where there was not even an alternative.

Revision history for this message
ThiloPfennig (tpfennig) wrote :

I have another example where things do not work well:
http://www.kodak.com/eknec/PageQuerier.jhtml?pq-path=12368&pq-locale=de_DE&CID=KOSBANNER&LOC=290808_CRM_F1_M863

It contains this tag:
                          <meta http-equiv="Content-Type" content="text/html;charset=utf-8">

But display is not set to UTF-8 but to ISO-8859-1 which is the opposit of what described above. Essentially 80% of all german pages I visit I broken since newer Firefox > 3.0.x - not sure since which exact version. Funny how few energy is invested if the most important application is partly unusable for propably all users who do not have the primary language as EN, EN_US or similar. I guess thisis because those users dont have the problem, because 99% of the characters are the same in those languages in ASCII or UTF-8.

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding
Download full text (5.1 KiB)

On 2008-08-27 01:42, ThiloPfennig wrote :
> Although Andre was spamming ...
 Spamming ?????
If you say that spamming is to have to repeat the same words over and
over to have simple facts understood or believed, you're right, and
you've started doing the same!

What I said is :

1) That it is nonsense to specify the encoding charset in HTTP (or
rather, get it from HTTP), because if you save the HTML file (or
transmit it by FTP or network sharing or ...), you lose that HTTP
information unless you modify the file when you save it, which had
better be done from the start.
Having the transport protocol specify the attributes of a file is great
but only if all transport protocols can do it and if the receiving
program knows where to keep that information.
That is far from being the case and hence the best way and place to
identify file attributes is within the file itself.
Obvious, isn't it?
If anyone disagrees, look at www.phpwact.org/php/i18n/charsets under
"Everybody gets it wrong" for a case of a joker HTTP saying that a file
specifying utf-8 is iso8859-1 and of a second joker believing the first
and obstinately displaying the joke charset.
Obviously, it's nonsense to have the sender look into a file to tell the
receiver what it can find by himself, especially if the sender's job is
only to transmit and if the receiver's job is to decode.
Is it a good idea for Firefox to believe someone (HTTP) saying what is
obviously wrong?

2) The default character set is ISO8859-1 and hence it is not a good
idea to allow the user to let configure it and hence to have Firefox
disregard the standards.
If there are encoding errors in pages, the thing to do is to correct
those pages and not to tweak the browser to display "correctly"
incorrect pages and "incorrectly" the correct ones.
Autodetecting what someone should have specified but did not specify is
always bad because it causes that someone to believe what he did was
right and to continue the same spreading mistake.

3) I have seen pages that do not specify the character set display other
character sets than ISO8859-1, generally UTF-8 but randomly in time and
character set.
So, I was glad to find a procedure to set Firefox in a situation when
the error occurred repeatedly.
I have been accused to have _caused_ the error.
To the best of my knowledge, nothing you can do before a page displays,
and certainly not what I did, is allowed to change the character set
that that page is displayed with.
Only _after_ a page was displayed would someone work around an encoding
mistake by _forcing_ View|Character encoding to redisplay the page with
another encoding.
What I did, forcing an encoding, is supposed do to change the display of
the last page, not the next one, isn't it? I wonder what so hard to
understand in that.
I can understand that some people would take it as a cheap game to guess
the character set of the next page they find, but even then there's no
need to preset the character set to play that game. And I swear I
neither like nor played that game.
May I recall that, just like it happened to ThiloPfennig and Alexander
Sack himself, the bug basically shows without doing...

Read more...

Revision history for this message
André Pirard (a.pirard) wrote : Re: Firefox can display a page with the wrong encoding

http://atilf.atilf.fr
Click 'Entrez dans le TLF'
Fill in search 'erreur', click 'Valider 1'
Select the word 'ERREUR', a window pops up
Click on 'le TLF1' in that window
The pop up window invariably displays ISO8859-1 as UTF-8 .
You can paste one of its frame's URL directly as a New Window's URL.
http://194.214.124.200/dendien/scripts/tlfiv5/displayp.exe?2;s=atilf_40_1002743130;i=ft-1-1.htm;;
When, after that, I came back to (redisplaying) the first and second page above, that is
http://atilf.atilf.fr/dendien/scripts/tlfiv4/showps.exe?p=combi.htm;java=no;
that page repeatedly displayed using UTF-8 again.
I forced it to redisplay using ISO8859-1.
After that, I could not reproduce what I had seen.

So, Mr Sack, you can see it works exactly the opposite of what you said.
Firefox failed without any "manual intervention".
On the other hand, it needed a "manual intervention" to restore correct working.

There is obviously something randomizing Firefoxe's behaviour.

Revision history for this message
Mante (mante) wrote :

Can confirm that firefox on intrepid (3.03) DOESN'T remember encoding. Tried with several sites. Sometimes works. Sometimes not. Moreover when utf8 is active, input text length is excessive. Don't know if it happens only to me, but this affects the aspect of several pages.
example google.it input text

Revision history for this message
André Pirard (a.pirard) wrote :

This seems to exhibit the problem repeatedly :
(Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008061015 Firefox/3.0)

display http://atilf.atilf.fr/tlf.htm
click "Entrez dans le TLF"
fill in "erreur" and click "Valider 1"
Select "ERREUR" (or any word), a window pops up.
Click on any link, e. g. "le TLFI" (*)
Now you have a window with a page containing two frames with incorrect encoding.
screen shot attached.
Open either frame in its own window.
Source: <head>...</head> no charset specified => iso-8859-1;
The file contains iso-8859-1 characters indeed but they are displayed using UTF-8.
THIS IS WRONG ENCODING DISPLAY WITHOUT MANUAL INTERVENTION.

(*) This "link" page displays correctly. It presumably contains iso-8859-1 like the rest.
BUT it's info says it was decoded with UTF-8.
However, when displaying its source (correctly), View|Encoding shows no selected encoding.
And when forcing any code (even UTF-8), the display gets totally wrong.
THIS IS WRONG ENCODING DISPLAY WITHOUT MANUAL INTERVENTION.
THIS IS TOTALLY WRONG.

I hope it's the same problem we all see randomly.

Revision history for this message
André Pirard (a.pirard) wrote :

Why, 10 months after having been introduced, does this bug remains "Incomplete"?
Much work has been spent to demonstrate the problem, and the last variation is the clearest and most repeatable report there is.
"This bug report was marked for expiration 55 days ago".
What does that mean, except frustration?
Should you need any further information, please do ask.
Does Ubuntu appreciate bug reports ???

Revision history for this message
John Vivirito (gnomefreak) wrote :

sorry didnt see the screenshot before i typed my reply but still stands what other sites do you see this on?
http://launchpadlibrarian.net/17344982/I_do_want_my_prize.png

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On 2009-03-21 19:13, John Vivirito wrote :
> sorry didnt see the screenshot before i typed my reply
 Which reply? Who are you talking to?

> but still stands what other sites do you see this on?
>
It does not depend on a site, presumably.
It depends on a Web page.
I myself have stopped analyzing each and every encoding mistake I see.
How many examples do you need?
Are you really keeping this report 'incomplete'?

Revision history for this message
André Pirard (a.pirard) wrote :

There it goes again.
I opened a new tab and I clicked a bookmark set to
http://atilf.atilf.fr/tlf.htm.
WiFi was down.
I don't know if I did anything in between.
But when I clicked "try again", I got what's in the attachment.
View/Character Encoding is UTF8 and displayed URL is this
http://atilf.atilf.fr/dendien/scripts/tlfiv4/showps.exe?p=combi.htm;java=no;

Why is this report still "incomplete" after one year?
I have identified a repeatable procedure.
Do you need more information?
Do you really want bug reports?
Am I losing my time?

Revision history for this message
papukaija (papukaija) wrote : Re: Firefox can display a page with the wrong encoding

Thank you for your bug report. To maintain a respectful atmosphere, please follow the code of conduct - http://www.ubuntu.com/community/conduct/ . Bug reports are handled by humans, the majority of whom are volunteers, so please bear this in mind.

Revision history for this message
papukaija (papukaija) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better. We are sorry that we do not always have the capacity to look at all reported bugs in a timely manner. There have been many changes in Ubuntu since that time you reported the bug and your problem may have been fixed with some of the updates. If you could test the current Ubuntu development version, this would help us a lot. If you can test it, and it is still an issue, we would appreciate if you could upload updated logs by running apport-collect 228988, and any other logs that are relevant for this particular issue.

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On 2010-08-22 16:08, papukaija wrote :
> Thank you for your bug report. To maintain a respectful atmosphere,
> please follow the code of conduct -
> http://www.ubuntu.com/community/conduct/ .
 Your useless comment is not maintaining a respectful atmosphere.
Please add to http://www.ubuntu.com/community/conduct/ that it is not
respectful to a bug reporter to change his bug title to mean something
else and to add a comment making that report somebody else's different
report.
Bug are almost exclusively reported by volunteers.
Please bear in mind that they stop reporting if they find it useless,
receive inappropriate answers or no answers at all, and if the sole
result of their work is that, as you say it yourself twice, "the bug and
your problem may have been fixed with some of the updates" in complete
disregard of what the reporter has done.
And please bear in mind that it is not "my" problem that I'm trying to
get fixed but Ubuntu's problem that I'm trying to cooperate with in a
least selfish manner.

Revision history for this message
papukaija (papukaija) wrote : Re: Firefox can display a page with the wrong encoding

I haven't edited this bug title nor its description. The CoC (code of conduct) notice was about your comments 11,18, 20, 26,27,29, 30 and now 33. So, back to comment 32; can you reproduce this bug in lucid or maverick? Do you have any other urls to test as your example url doesn't provide valid html at all (26 errors and 6 warnings!), it's certain that firefox has to be creative to render that page correctly and obviously fails. This bug is marked incomplete because the requested information hasn't been provided.

Revision history for this message
papukaija (papukaija) wrote :

And if I have edited one of your bug's title or description, it because the guideline available at https://wiki.ubuntu.com/Bugs/Improving

Revision history for this message
papukaija (papukaija) wrote :

We'd like to figure out what's causing this bug for you, but we haven't heard back from you in a while. Could you please provide the requested information? Thanks!

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On 2010-09-15 10:31, papukaija wrote :
> We'd like to figure out what's causing this bug for you, but we haven't
> heard back from you in a while. Could you please provide the requested
> information? Thanks!
>
 This bug has been discussed in great length and details and all
information was given.
Please read the log.

In particular, I explained that the character set should be described
within the HTML file exclusively.
If the character set of a HTML file is defined by MIME in a HTTP, SMTP
or whatever envelope, then Thunderbird must transfer that information
within the file when it stores that file to disk. Neither Thunderbird
nor Firefox do that.
The only reaction I had to these deep remarks is someone changing my bug
title to
"fuzzy/confusing firefox View -> Character encoding menu semantics"
which does not reflect at all what I wrote.

Revision history for this message
papukaija (papukaija) wrote : Re: Firefox can display a page with the wrong encoding

You were asked in comment 32 to reproduce this bug in the current development release of Ubuntu which is Maverick. I can't reproduce this bug in Lucid wit the given URL with UTF-8 encoding. That's why I asked you in comment 34 to give another URL to test with. Ideally that URL should produce valid HTML (the given URL's html has 26 errors and 6 warnings). I'm a bit confused, you wrote "In particular, I explained that the character set should be described within the HTML file exclusively." while your given URL doesn't have any meta tags in its header.

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding
Revision history for this message
André Pirard (a.pirard) wrote :
Download full text (3.5 KiB)

On 2010-09-15 22:38, papukaija wrote :
> You were asked in comment 32 to reproduce this bug in the current
> development release of Ubuntu which is Maverick. I can't reproduce this
> bug in Lucid wit the given URL with UTF-8 encoding. That's why I asked
> you in comment 34 to give another URL to test with. Ideally that URL
> should produce valid HTML (the given URL's html has 26 errors and 6
> warnings). I'm a bit confused, you wrote "In particular, I explained
> that the character set should be described within the HTML file
> exclusively." while your given URL doesn't have any meta tags in its
> header
 I'm sorry I don't run Maverick. Nor Lucid yet.
Please note that these problems are browser issue, not system issues.

I'm surprised you were not able to reproduce this problem.
I spent ½ h getting my hands on some Lucid, 2 min reproducing the bug as
I described it before and 1 more h making the same report for Lucid as I
did before. What I attached for today's test is the twin picture of the
former.

A character set issue is not affected by the number of errors in a
page. It's only affected by its number of character set related errors
(just like the number of bumps on your car cannot be the reason why you
hit another car).

If you look carefully at your W3 HTML validation for the two frames in
the attached picture, you will read as the first line (as if most
important) :
No Character Encoding Found! Falling back to |windows-1252|.
Having no character encoding statement is *NOT* an error. And as far as
I know, the fall-back character set is ISO8859-1, not |windows-1252|
which is a proprietary character set that the official standards are
avoiding, or trying to. But that doesn't change our discussion.

Now if you look carefully at the Firefox/View/Character Encoding, you
will see that what Firefox used as fall-back encoding is UTF-8.

Now, does that discrepancy between Firefox and the sacred HTML
validation (and what seems obvious to only me) prove enough what I've
been trying to explain during 2½ year to people relaying to say that
there is no bug, turning a very complete report to incomplete and making
it a mess. I did not appreciate to be accused of tweaking the browser in
order to exhibit bugs that don't exist.

NB: For exactly the same experiment, Internet Explorer (which is a Web
explorer actually, traceroute is an Internet explorer) correctly uses
Windows Latin-1 as fall-back (not 8859-1, of course). Is that enough proof?

People using 7-bit ASCII do not see these issues, of course, but they
should trust 8-bit users when they say they are making their days.

The phrase that confuses you relates mainly the the former bug and a
wider class of character set issues. There are indeed more bugs than
this, but I obviously hesitate spending 2½ more year per bug.

Now, as a side note, please look at the Screenshot-2 attachment.
Does anyone really think it's possible to work with that Windows panel?
Hem, I finally found that the Windows List shrunk for no reason of mine,
but the tooltip continues to hide the list and that's a 2½ year old
problem too.
I see that you have been tracking this Bug #222267 too, thank you, and
that you added a Har...

Read more...

Revision history for this message
André Pirard (a.pirard) wrote :

On 2010-09-15 22:38, papukaija wrote :
> I'm a bit confused, you wrote "In particular, I explained
> that the character set should be described within the HTML file
> exclusively." while your given URL doesn't have any meta tags in its
> header
 Please understand that "exclusively" in the phrase you quote means that
the character set should logically not be described anywhere else than
within the HTML file. Hence, what is important with regard to this
sentence is not what is inside the file but what is outside of it, or
rather that nothing is there, or rather that it is not used.
Your concern about a character set tag existing or not inside the HTML
file pertains to the earlier discussion of the fall-back character set.

Revision history for this message
papukaija (papukaija) wrote : Re: Firefox can display a page with the wrong encoding

You wrote "I'm sorry I don't run Maverick. Nor Lucid yet. Please note that these problems are browser issue, not system issues." Of course this bug is a browser issue but Lucid and Maverick have newer versions of Firefox than Hardy or Jaunty has. Anyway, I'm attaching a screenshot (taken in Lucid with FF 3.6.9) that doesn't have this issue.

tags: added: hardy jaunty
Revision history for this message
papukaija (papukaija) wrote :

"I described it before and 1 more h making the same report for Lucid as I did before"
-> Does that mean that you reported a new bug?

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On 2010-09-16 02:30, André Pirard wrote :
> ... but the tooltip continues to hide the list and that's a 2½ year old
> problem too. I see that you have been tracking this Bug #222267 ...
 Is it a result of this remark that the priority of this 2½ year old bug
has just been set to low?

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On 2010-09-16 13:39, papukaija wrote :
> "I described it before and 1 more h making the same report for Lucid
> as I did before"
> -> Does that mean that you reported a new bug?
 No, it means that comment 40 is the same as comment 24 but in (much)
greater detail.

Regarding the comment in 24 about "manual intervention", I had been
accused of doing something before displaying the page to cause the
problem. Obviously, as opposed to changing the character set *after* a
page is displayed, nothing you can do *before* displaying a Web page
must affect the way that page is displayed (unless, of course, purposely
designed to do so, like automatic character set recognition or being the
followup of another page).

Revision history for this message
André Pirard (a.pirard) wrote :

On 2010-09-16 13:24, papukaija wrote :
> You wrote "I'm sorry I don't run Maverick. Nor Lucid yet. Please note
> that these problems are browser issue, not system issues." Of course
> this bug is a browser issue but Lucid and Maverick have newer versions
> of Firefox than Hardy or Jaunty has. Anyway, I'm attaching a screenshot
> (taken in Lucid with FF 3.6.9) that doesn't have this issue.
 My Lucid is almost out of the box, runs FF 3.6.3, and does show the
problem as did all the FF versions I've used along 4 years.

And there we go again as I said, instead of trying to find the reason of
the problem, denying what I say in two lines sending me off to write two
pages to prove it until I get on my nerves.
(You say you attached a screenshot but there is none, the least you
could say is which View/Encoding your displayed page uses with but you
don't).

I supposed that the goal of bug reporting is to find how clicking on
"Install Ubuntu, French" can get you a working system, not proving that
other American ones meet no problem, but it turns out that I was
terribly wrong.

Yes, of course, you're perfectly right sir, the reports and screenshots
I sent you all along those 2½ years are made up to play a joke on Ubuntu.

Ubuntu asks me to report bugs. I've done my job. It's bye-bye from me.
And probably good bye because you all made me feel like not reporting
bugs any more.
After 2½ years seeing my many reports stranded I feel like having been
absolutely useless.
And I have other things to do.

papukaija (papukaija)
tags: added: lucid
Revision history for this message
papukaija (papukaija) wrote : Re: Firefox can display a page with the wrong encoding

The files (of the atil site) are stored on the server with ISO-8859-1 encoding (tested with wget) which is not recommended in Linux servers. I'm attaching the frontpage to this report. That file is saved with UTF-8 encoding and Firefox displays that page correctly with UTF-8 encoding but incorrectly with ISO-8859-1 encoding. What I see, is that Firefox has troubles showing pages if the encoding set in FF doesn't match with the file's encoding, but that, I think, is obvious and not a bug. As I and John in comment 28 asked, an other URL (to another site reproducing this issue) is necessary to fix this issue. By the way, you should install security updates to your Firefox. Thanks in advance!

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Windows test

I happened to launch VirtualBox to run Windows XP.
FYI, Here are the windowshots of the test case.

Windows_XP-Firefox_3.6.10
Windows_XP-Internet_Explorer_6.0.2900.5512.xpsp.080413-2111.png

As I know I will be accused to have tweaked those systems, I reply in
advance that it is not true. I use them very occasionally to help people
 or when cranks force me to use Windows.

Conclusion: update to Internet_Explorer_6.0.2900.5512.xpsp.080413-2111.

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Another URL

On 2010-09-16 18:43, papukaija wrote :
> As I and John in comment 28 asked, an other URL (to another site
> reproducing this issue) is necessary to fix this issue.
 That is why in comment 30 I was glad to mention a procedure I could
repeat at will.
It was not repeatable before and when I tried to find the fortuitous
conditions in which in occurred I was accused of having created those
conditions.
And now you're asking me yet another URL.
What is exactly the game? Is it 1 URL every 2 comments?
And why asking particularly me?
Lots of people come across that bug daily.
Why don't you ask *them* to stop shivering?

Revision history for this message
papukaija (papukaija) wrote : Re: Firefox can display a page with the wrong encoding

You are this bug reporter. In addition, this bug is only affecting one person. That's why you should give another URL to test. Thanks in advance!

Revision history for this message
papukaija (papukaija) wrote :

Those screenshots prove that this is an upstream (Mozilla) issue and it would be nice if somebody having it could send the bug to the developers of the software by following the instructions at https://wiki.ubuntu.com/Bugs/Upstream/Mozilla. If you have done so, please tell us the number of the upstream bug (or the link), so we can add a bugwatch that will inform us about its status. Thanks in advance.

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On 2010-09-16 18:43, papukaija wrote :
> The files (of the atil site) are stored on the server with ISO-8859-1
> encoding (tested with wget) which is not recommended in Linux servers.
 That is no reason for FF displaying the default character set
(ISO-8859-1) incorrectly.
FYI, the atlif's server is not a Linux but a Windows server.
> I'm attaching the frontpage to this report. That file is saved with
> UTF-8 encoding and Firefox displays that page correctly with UTF-8
> encoding but incorrectly with ISO-8859-1 encoding.
 What you say is fuzzy. AM I correctly guessing what you did?
You took a correct ISO-8859-1 page, transliterated it to UTF-8 and
stored it as an invalid page. Then you displayed that page with FF and
the display was incorrect because UTF-8 was displayed using default
ISO-8859-1. Then you forced FF to display the page with View/Character
encoding UTF-8 and UTF-8 was displayed correctly.

Invalid pages displaying incorrectly and correctly when tweaked is
perfectly expected results but is unrelated with the bug description.
Why do you explain that?
> What I see, is that
> Firefox has troubles showing pages if the encoding set in FF doesn't
> match with the file's encoding, but that, I think, is obvious and not a
> bug.
 There is no encoding set in FF that must match with the page's encoding
when it is loaded.
FF learns the encoding of the page through the declarations in it and
shows in View/Character Encoding (which is not a setting) the encoding
that it has used.
One may use View/Character Encoding *after* the page has displayed to
have FF redraw the page in another correct or incorrect character set
but this is totally unrelated with the bug.

The bug is that an ISO8859-1 encoded page that has to be displayed using
ISO8859-1 by default is *sometimes* (randomly) displayed using UTF-8. I
have shown that with much obviousness in the screenshots.

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Another case

Occurred while buying concert tickets, well, trying to.
https://www.rodrigue.fr/transact/venteenligne.asp?WCI=Panier_Reserver
Screenshot and page's source attached.
The codepage used MUST be ISO8859-1 and is UTF-8.
After I stored this page in a file, IT WAS ISO8859-1 correctly displayed
Such a stored page should display the same as its Web source.
It may be that FF depends on something having happened in the past.
But THAT MUST NOT HAPPEN.
The bug is OBVIOUS.
3.0.15, but it was demonstrated that 3.6 is codepage silly too.

And I told you, I know that people seeing this are just shrugging.

Revision history for this message
papukaija (papukaija) wrote : Re: Firefox can display a page with the wrong encoding

The URL goes to blank and black page... Did you use wget to download the file from the server? If yes, then it means that the file is encoded with iso-8859-1 and therefore should be displayed in FF with that encoding.

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

On 2010-10-01 12:06, papukaija wrote :
> The URL goes to blank and black page... Did you use wget to download the
> file from the server? If yes, then it means that the file is encoded
> with iso-8859-1 and therefore should be displayed in FF with that
> encoding.
 I saved the page, which is the best method.
Show source was showing ISO8859-1 data wrongly displayed in UTF-8.
Yes, you're perfectly right, FF displayed IT in UTF-8 instead of 8859-1.
This is called a bug and has been confirmed by many examples.
Why was the confirmed status removed?
It's wasting one's time reporting bugs if confirmed status is removed.

Revision history for this message
papukaija (papukaija) wrote : Re: Firefox can display a page with the wrong encoding

The URL in comment 53 goes to blank and black page which is useless to triage this bug.

Revision history for this message
papukaija (papukaija) wrote :

Is there any news about this bug? Has someone affected by this bug sent the report upstream? Could you tell us the bug number, so we can add a bugwatch that will inform us about its status? Thanks in advance.

papukaija (papukaija)
Changed in firefox:
status: New → Incomplete
Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Bad pages

 папская, if your hobby is checking Web pages, look at this one from Ubuntu:

http://manpages.ubuntu.com/manpages/dapper/man4/hunspell.4.html

Do you understand what's wrong?
I told you most people are shuddering about character set issues.

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding

 On 2010-10-01 21:01, papukaija wrote :
> The URL in comment 53 goes to blank and black page which is useless to
> triage this bug. https://bugs.launchpad.net/bugs/22898

The complete procedure to have a chance to reproduce this case is to
visit http://www.operaliege.be, start buying a ticket and have the
payment procedure fail.
That URL contains the message displayed in that case on return to ORW
from the financial site.

Revision history for this message
André Pirard (a.pirard) wrote :

> In addition, this bug is only affecting one person. That's why you
> should give another URL to test.

How do you know? How many persons did you ask?

I asked 10 friends to test that URL.
6 shuddered. I told you that's what people usually do (only those who
see characters to shudder at).
4 answered.
1 reported the same problem.
1 reported that the code page is sometimes 8859-1 and sometimes 1521,
which is the same bug, but invisibly.
Furthermore, Alexander Sack investigated and reported the same problem.
Furthermore I see ThiloPfennig and Mante reporting problems that
have not been investigated.
Furthermore, think: the bug does not affect a person but a system and I
have 4 different ones affected.

That makes 7 to 9 cases, not counting the shrugging.
Why do you say only one?

As I am a thinking person, I installed a fresh Ubuntu to make the test
and the bug was there.
And I made the test in a blank Firefox 3.6.11 configuration and the bug
was there.

Please stop keeping to ask more URLs and start thinking that it's
useless because there is a difference on your system that makes display
that page differently than on all of mine.
And remember that I said from the start that the bug is usually random
to the same person.
That's why I presented the atlif test which is not or less random.
One probably needs to look at the code to understand why.

I had made a new case because they made the old one a mess.
You made this one even more messy.

Please return the bug to confirmed status, there was no reason to change
that because there is is enough proof.

Revision history for this message
papukaija (papukaija) wrote : Re: Firefox can display a page with the wrong encoding

According to LP, this bug is affecting 1 person (as seen on the top of this bug report). That's one reason for this bug for not being marked as confirmed. Secondly, you were asked (first time in comment 28) to provide an other URL to test this bug. In comment 53 you gave one but that's not working and concerning operaliege.be, I am not going to give my bank details to them. Finally, you (or anyone else affected by this bug; but that's not the case according to LP) were asked in comment 51 to report this bug upstream. Reporting this bug upstream would give this bug "Triaged" as status and that status has higher importance than Confirmed, so you're strongly recommended to report this bug upstream. I can't report this bug upstream since I can't reproduce this bug. Btw, are you referring to Maverick by "fresh Ubuntu" ?

Revision history for this message
papukaija (papukaija) wrote :

Changed the main package to firefox as you told that you are affected by this bug in FF 3.6.x.

affects: firefox (Ubuntu) → ubuntu
affects: firefox-3.0 (Ubuntu) → firefox (Ubuntu)
Revision history for this message
papukaija (papukaija) wrote :

Is this upstream bug enough similar to this bug?:
https://bugzilla.mozilla.org/show_bug.cgi?id=253575

If, not there are few more encoding related bugs at https://bugzilla.mozilla.org/show_bug.cgi?id=254868 but none of those really is the same than this bug so it might be better to report this bug to the upstream as a new bug.

André Pirard (a.pirard)
summary: - Firefox can display a page with the wrong encoding
+ Firefox can display a page with the wrong default encoding
André Pirard (a.pirard)
description: updated
description: updated
Revision history for this message
papukaija (papukaija) wrote :

The new instructions in the update part of this bug's description gives a nice correctly shown page using ISO-8859-1. Please also answer the question in comment 63. Also, are you referring to Maverick by "fresh Ubuntu" ? Thanks in advance!

Revision history for this message
papukaija (papukaija) wrote :

Removing the bug-watches; they were automatically added by LP from my comment.

Revision history for this message
In , A-pirard-hotmail (a-pirard-hotmail) wrote :

User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.11) Gecko/20101013 Ubuntu/10.04 (lucid) Firefox/3.6.11
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.11) Gecko/20101013 Ubuntu/10.04 (lucid) Firefox/3.6.11

Full, detailed story: https://bugs.launchpad.net/ubuntu/+bug/228988

Since I'm using it, I see Firefox (2.X and 3.X) *sometimes* display pages using the wrong character encoding. Sometime a page will display correctly, later the same page won't, without any apparent reason, nor special action. However, I have found a procedure that's reproducible on all my systems, including freshly installed, e.g. Ubuntu Karmic as is, but unfortunately not on all other ones (mainly US ones).
display http://atilf.atilf.fr
Click 'Entrez dans le TLF'
Fill in search field: 'erreur', click 'Valider 1'
Select the word 'ERREUR', a window pops up
Click on 'le TLF1' in that window
The pop up window invariably displays ISO8859-1 as UTF-8 as shown in
http://launchpadlibrarian.net/55706907/Screenshot-Mozilla%20Firefox.png

See more details at the Ubuntu launchpad URLs.
Sorry if the questions I was asked there made that report a mess.

Side notes:

It is a bad idea to specify the character code of a Web page in the envelope (SMTP or HTTP MIME), because it puts the burden of modifying the page to someone who stores it to a file instead of to whoever wrote the page in the first place.

Any feature allowing the user to correctly display a bad page is not only inciting the Web authors to continue writing more bad pages but also towards the users displaying good pages the wrong way. The Web is full of horror stories by people not having understood character sets

Reproducible: Sometimes

Steps to Reproduce:
1. display http://atilf.atilf.fr
2. Click 'Entrez dans le TLF'
3. Fill in search field: 'erreur', click 'Valider 1'
4. Select the word 'ERREUR', a window pops up
5. Click on 'le TLF1' in that window
The pop up window invariably displays ISO8859-1 as UTF-8 as shown in
http://launchpadlibrarian.net/55706907/Screenshot-Mozilla%20Firefox.png

Actual Results:
http://launchpadlibrarian.net/55706907/Screenshot-Mozilla%20Firefox.png

Expected Results:
same after forcing ISO8859-1

Thanks.

Revision history for this message
André Pirard (a.pirard) wrote : Re: [Bug 228988] Re: Firefox can display a page with the wrong encoding
Download full text (4.5 KiB)

 Thanks for your response.

On 2010-10-31 12:26, papukaija wrote :
> According to LP, this bug is affecting 1 person (as seen on the top of
> this bug report). That's one reason for this bug for not being marked as
> confirmed.
 Please understand the meaning of the word shudder.
You seem not to have noticed that even those who comment bugs do not use
the counter.
Visit the 10 bug numbers next ton this one and you will find 10 bugs not
affecting anyone and yet confirmed or even better.
Why did Alexander Sack, and at least 3 persons meet the bug and not
count it?

You seem not to understand what ThiloPfennig wrote :
> To start with the first example mentioned in the bug report if I
> access the page: http://atilf.atilf.fr/tlf.htm Firefox sets encoding
> to UTF-8.
 Neither this
> I have another example where things do not work well:
> http://www.kodak.com/eknec/PageQuerier.jhtml?pq-path=12368&pq-locale=de_DE&CID=KOSBANNER&LOC=290808_CRM_F1_M863
> <http://www.kodak.com/eknec/PageQuerier.jhtml?pq-path=12368&pq-locale=de_DE&CID=KOSBANNER&LOC=290808_CRM_F1_M863>

As well as the next comment showing that Firefox had used !!!! ASCII
!!! for that page.
Why were those persons not asked to provide more URLs for the collection?

This bug reporting business is complete nonsense.
Bug #206884 reports exactly the same problem as this one.
Someone changed the title, confirmed it and linked it to something
totally different.
And, for the LP addicts, you can take one more count from there for this
one.

We're nearing 10 there and you count 1.
> Secondly, you were asked (first time in comment 28) to
> provide an other URL to test this bug. In comment 53 you gave one but
> that's not working and concerning operaliege.be, I am not going to give
> my bank details to them.
 I am not laying URLs.
Please try to understand that the best way to have a payment fail is not
giving any bank detail.
I'm sorry I can't choose the circumstances in which I meet this bug.
Could you please understand that after trying all the examples I and
other persons showed, it's no use to keep asking more URLs if there's a
reason why that bug will never show on your computer as obstinately as
it shows on all mine and that the matter is to understand why.
> Btw,
> are you referring to Maverick by "fresh Ubuntu" ?
 In VirtualBox, I have Ubuntu 10.04, 10.10, Windows 1.01, XP and 7 all
exhibiting the Bug with native Firefox or its latest Ubuntu update.
Maverick has 3.6.10 and my running Lucid 3.6.11.
> Finally, you (or anyone else affected by this
> bug; but that's not the case according to LP)
 Could you please let that LP alone?
> you were asked in comment 51
> to report this bug upstream. Reporting this bug upstream would give this
> bug "Triaged" as status and that status has higher importance than
> Confirmed, so you're strongly recommended to report this bug upstream. I
> can't report this bug upstream since I can't reproduce this bug.
 And you were asked not to remove confirmed status.

What I was actually asked is this:

"Please help us improve Ubuntu by reporting bugs" what I did 2½y ago,
and later without warning :
"Thank you for spending your time [and they don't imagine how much]
helpin...

Read more...

Revision history for this message
André Pirard (a.pirard) wrote :

 On 2010-10-31 12:58, papukaija wrote :
> Is this upstream bug enough similar to this bug?:
> https://bugzilla.mozilla.org/show_bug.cgi?id=253575
>
> If, not there are few more encoding related bugs at
> https://bugzilla.mozilla.org/show_bug.cgi?id=254868 but none of those
> really is the same than this bug so it might be better to report this
> bug to the upstream as a new bug.
 Thanks for this research.
No, obviously all these bugs are different.
What is a bug watch supposed to do? You can certainly ignore the first bug.
But the meta bug encouraged me to add a bug if it's included in that list.

Revision history for this message
papukaija (papukaija) wrote :

@members of bug-control team: This bug is now reported to the upstream, please mark it as triaged. Thanks in advance!

Changed in firefox:
status: Incomplete → New
importance: Undecided → Unknown
status: New → Unknown
Changed in firefox (Ubuntu):
status: Incomplete → New
Revision history for this message
papukaija (papukaija) wrote :

@André: Thank you for reporting this bug to the upstream. I've linked the bugwatch to this bug report; that way Ubuntu's developers know when the upstream has fixed the bug which can then be ported to Ubuntu (for example through Stable Release Updates). Ps. I'm not counting the number of people affected by this bug - LP does it.

Revision history for this message
In , Nickolay Ponomarev (asqueella) wrote :

Assuming that in step 4 by "select" you meant double-click the word in red, I can't reproduce on Mac, neither using 3.6.12, nor using Firefox 4 nightlies.

This is the page that opens in a frame in the pop-up: http://194.214.124.200/dendien/scripts/tlfiv5/displayp.exe?18;s=atilf_40_2841007530;i=ft-4-2.htm;;
It doesn't send the charset in HTTP headers at all and doesn't specify the charset in the HTML, as far as I can see.

Can you reproduce using a clean profile? <http://support.mozilla.com/en-US/kb/Basic+Troubleshooting> With a build from mozilla.org? What's selected in View -> Character Encoding -> Autodetect?

Bug Watch Updater (bug-watch-updater)
Changed in firefox:
importance: Unknown → Medium
status: Unknown → New
Revision history for this message
In , A-pirard-hotmail (a-pirard-hotmail) wrote :

Привет Николай?
> Assuming that in step 4 by "select" you meant double-click the word in red,
 No, by 'select' I mean 'select'.
> I can't reproduce on Mac, neither using 3.6.12, nor using Firefox 4 nightlies.
 That's the meaning of 'may', 'Reproducible: Sometimes' and comments on Launchpad. The occurrence seems to depend on the system used, on the page and on some circumstances. The choice of this particular example is because it always failed for me. But that's never for others. At least 7 people have confirmed to have seen something like this. But most people just shudder.
> This is the page that opens in a frame in the pop-up:
> http://194.214.124.200/dendien/scripts/tlfiv5/displayp.exe?18;s=atilf_40_2841007530;i=ft-4-2.htm;;
 The error is not in that page. I said "Click on 'le TLF1' in that window" and the error is in a page like this
http://194.214.124.200/dendien/scripts/tlfiv5/xxx.exe?mthk=1;forced=40;s=2807581110;mot=ERREUR;pot=40,53,41,42,43,44,46;from=atilf,51,2807581110;ru=no;
> It doesn't send the charset in HTTP headers at all and doesn't specify the
> charset in the HTML, as far as I can see.
 Correct. No tlfi page does it. And that why I said about the second page that it "displays ISO8859-1 as UTF-8". That is the problem. It may probably affect only pages without character set definitions.
BTW, specifying the character set in HTTP is a crazy invention because it means that a program storing a page must modify it (and Firefox fails to do that).
> Can you reproduce using a clean profile?
 Yes, it's now four years that I'm doing that on request. Somebody should conclude that the problem should be analyzed, not watched.

I said this. Isn't it what you mean?
> In VirtualBox[es], I have Ubuntu 10.04, 10.10, Windows 1.01, XP and 7 all [but 1.01 ;-)]
> exhibiting the Bug with native Firefox or its latest Ubuntu update.
> Maverick has 3.6.10 and my running Lucid [has] 3.6.11.
 These systems were just installed or very little used (just for some installations and tests), Ubuntu ran the Firefox version that came with it and the latest Firefox was installed in Windows.
> <http://support.mozilla.com/en-US/kb/Basic+Troubleshooting>
> With a build from mozilla.org?
 This troubleshooting article is indecent. Asking naive people to erase all the cookies that were recorded to ease their life and then to make tests in a blank Firefox profile sounds like a bad joke.

In my running system I installed 3.6.12 and I checked that the problem still occurs in a blank profile.
> What's selected in View -> Character Encoding -> Autodetect?
 That is a feature that is totally undocumented, that nobody understands [I have some good guesses], that everybody and his dog use to try to solve their problems the wrong way and that shouldn't exist.
Firefox is not supposed to autodetect anything and mu setting is most evidently (off).

Shall I say спасибо и пака?

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in firefox (Ubuntu):
status: New → Confirmed
Revision history for this message
In , A-pirard-hotmail (a-pirard-hotmail) wrote :

FYI

1) This bug, that admittedly does not hit everyone, has been confirmed by some 6 persons on Launchpad for Ubuntu. So, why 'unconfirmed'?

2) I reported it for release 3.x, I upgraded to release 15 and the bug is still there

3) Since then, I noticed a highly repeatable test: if I open a plain text file (not HTML), Firefox consistently displays with the Cyrillic codepage

4) Why Cyrillic?
a) because I have it configured in View/Character encoding selections?
b) because I sometimes use Cyrillic?
c) ... ?

5) That may be the reason why it is the codepage in which *some* HTML pages are erroneously displayed

4) there are at least 2 reason why I may report the bug more than others
a) not many Western characters users have Cyrillic configured and use it
b) I don't just shrug

5) I'm available to give more information to a developer or to conduct nondestructive tests if that's what solving a bug means

Revision history for this message
Henri Sivonen (hsivonen) wrote :

Pretty sure this was fixed as upstream bug https://bugzilla.mozilla.org/show_bug.cgi?id=844115

Bug Watch Updater (bug-watch-updater)
Changed in firefox:
status: New → Invalid
To post a comment you must log in.