Folder content disappeared in mozilla-thunderbird after upgrading from 1.5 to 2.0

[Karoly Molnar]

Binary package hint: mozilla-thunderbird

Hello,

After upgrading from Feisty to Gutsy the content of one folder and all its subfolders disappeared in thunderbird. Hopefully I could find the reason. The problem was that the folder had a "." in its name. The name was "Open Source Inc." so originally I had two files:
Open Source Inc.
Open Source Inc..msf
And one folder:
Open Source Inc..sbd

After the upgrade two new files were created:
Open Source Incc6bfcd54
Open Source Incc6bfcd54.msf

This lead me to a conclusion that the old thunderbird handled the "." as a dot and the new one replaced the "." with c6bfcd54 in the file name.

The solution was pretty simple, removing the "." in the old files and folder names and deleting the newly created files worked like a charm.

Thanks,
Karoly

[Ajschult (ajschult)]

This regressed between 1.8 branch builds of seamonkey between 2007-01-17 and 2007-01-25, so bug 229522

If I just create an A.B.C. folder with 1.1, I get the following in my Work.sbd directory:

A.B.C.
A.B.C..msf

If I then bring up a later build with the bug, I get:

A.B.C.
A.B.Ce15fb51d
A.B.Ce15fb51d.msf
A.B.C..msf

[M-wada (m-wada)]

(In reply to comment #1)
> A.B.Ce15fb51d
Caused by a part of patch for Bug 229522, a part for problem of Bug 117840(ending dot issue on Win).
"Ending dot" case should have been limited to MS Win.
Next is the quickest approach?
 - Remove "ending dot" and ship it with Tb 2.0.0.1, and re-open Bug 117840.
   RCS file: /cvsroot/mozilla/mailnews/base/util/nsMsgUtils.cpp
   -#define ILLEGAL_FOLDER_CHARS_AS_LAST_LETTER ".~"
   +#define ILLEGAL_FOLDER_CHARS_AS_LAST_LETTER "~"

Workaround :
 1) Delete garbage of A.B.Ce15fb51d / A.B.Ce15fb51d.msf
 2) Rename A.B.C. / A.B.C..msf to A.B.C_ / A.B.C_.msf, A.B.C / A.B.C.msf etc.
 3) Restart Thunderbird, and click the folder in order to display correct count

[M-wada (m-wada)]

To Andrew Schultz : Do you know result when Mac OS X? I guess that Mac OS creates file of "ending dot" normally too then this regression can occur on Mac OS too.

[Ajschult (ajschult)]

No, I don't have Mac OS X.

[Sugar-waffle (sugar-waffle)]

In Mac OS X, it is made as follows.
   Folder Name: A.B.C.
   File name made in Profile:
     A.B.Ce6bad49d
     A.B.Ce6bad49d.msf

Therefore, there is no problem.

Mac OS X 10.3.9
version 2.0.0.4pre (20070428)

[Ajschult (ajschult)]

Erm, that sounds exactly like linux... the problem is that old builds behaved differently.

[M-wada (m-wada)]

(In reply to comment #5)
> Mac OS X 10.3.9 version 2.0.0.4pre (20070428)
Thanks for test with Tb 2.0 on Mac OS X.
Hiro, can you test with Tb 1,5.x or before? I'd like to know behavior before this regression when Mac OS X.

[Sugar-waffle (sugar-waffle)]

(In reply to comment #7)
> Hiro, can you test with Tb 1,5.x or before? I'd like to know behavior before
> this regression when Mac OS X.
>
In Tb 1.5.10 Release, the file was made as follows.
(There is no problem in the access to this folder. )
  A.B.C.
  A.B.C..msf

And SeaMonkey1.1.1 results the same as comment#5.
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.2) Gecko/20070221 SeaMonkey/1.1.1

[M-wada (m-wada)]

(In reply to comment #8)
> A.B.C.
> A.B.C..msf
> Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.2)
Thanks for quick test.

To Andrew Schultz and hiro :
When mails are held in "A.B.C."(created by 1.5 and mails are saved), I think Tb 2.0 can use this file as folder of "A.B.C." normally. And I think subfolder (A.B.C..sbd/subfolder-N) access problem only, if "rebuild index" is not involved.
Is it true when Linux and Mac OS X? (sorry but I can test with Win only)

[Sugar-waffle (sugar-waffle)]

>Is it true when Linux and Mac OS X? (sorry but I can test with Win only)
It is a case with OS X.

It made it from Tb 1.5.10 as follows.
    |
   +-A.B.C.
         |
         +--sub1
And the message save to "A.B.C." and "sub1".

When this profile was used with 1.8branch Tb, the profile was as follows.
-rw------- 1 sek sek 750 5 5 06:47 A.B.C.
-rw-r--r-- 1 sek sek 2154 5 5 06:47 A.B.C..msf
drwxr-xr-x 4 sek sek 136 5 5 06:47 A.B.C..sbd/
-rw-r--r-- 1 sek sek 0 5 5 06:49 A.B.Ce6bad49d
-rw-r--r-- 1 sek sek 1095 5 5 06:49 A.B.Ce6bad49d.msf

So, the message is not seen though there is "A.B.C." in the folder list.
There is no change even if "Rebuild index" is executed.
Of course, it cannot access subfolder.

And two "A.B.C." appears when 1.8branch Tb is re-started.
In this case, it cannot access the message and the subfolder.

If the work around of comment#2 is not executed, the message cannot be seen with 1.8branch Tb.

[M-wada (m-wada)]

(In reply to comment #10)
Thanks for clear test result. I now understand difference between "|-A.B.C." and "|-A.B.C. (the folder is empty)" in comment #0.
"Rebuild Index" process looks to be invoked during restart. It seems that I expected too much...

I think there is two ways for ILLEGAL_FOLDER_CHARS_AS_LAST_LETTER "." issue:
 1)MS Win only "ending dot" for compatibility on Linux & Mac OS X.
   (When MS Win, no problem because folder of "ending dot" was impossible.)
   In this case, compatibilty between Linux/Mac and Win is lost,
   then several users will complain when they will move mail folders.
 2)Keep current patch for compatibility among OS's and for ease of maintenance.
   This means design change, then action for incompatibility is required.
   e.g. Explanation in release notes, automatic file name change logic.
And I feel 1) is better.

[M-wada (m-wada)]

To David Bienvenu : What do you think about best way?

[M-wada (m-wada)]

Bug 303729 has added "trailing space" case(fixed on 2007-03-05).
> -#define ILLEGAL_FOLDER_CHARS_AS_LAST_LETTER ".~"
> +#define ILLEGAL_FOLDER_CHARS_AS_LAST_LETTER ".~ "
(Sorry but I don't know whether included in tb 2.0.0.0 or not)

To Andrew Schultz and hiro :
I guess "trailing space" has to be "MS Win Only" too. Is it right?

[Ajschult (ajschult)]

Yes, space is a valid last character for linux (and I'd expect for Mac too).

[Sugar-waffle (sugar-waffle)]

(In reply to comment #14)
> Yes, space is a valid last character for linux (and I'd expect for Mac too).
>

Yes, it is the same also in Mac.

[Peter Trenholme (ptrenholme)]

I just upgraded from Tb 1.5.10.10 to Tb 2.0.0.0 on a Fedora Linux system, and one of my folders (named "Receipts, Invoices, etc.") was shown in the folder list, under that name, twice. (I.e., a duplicated folder name.) Both folders were shown as empty. Tihis was a sub-folder of the "Save" folder. When I when to the Mail subdirectory of the profile, in the Save.sbd directory, I saw two set of files: "Receipts, Invoices, etc7476509a.msf" and "Receipts, Invoices, etc..msf"

Deleting the "Receipts, Invoices, etc7476509a" files and renaming the "Receipts, Invoices, etc.." file and sbd directory to have a single dot fixed the problem. (Of course I lost the period at the end of the "etc.", but I can live with that.)

Still, the upgrade from 1.5 to 2.0 should do a better job of not loosing mail folders. Or, at a minimum, upgraders should be warned to rename their folders prior to the upgrade.

[M-wada (m-wada)]

*** Bug 382995 has been marked as a duplicate of this bug. ***

[Juha-krapinoja (juha-krapinoja)]

*** Bug 387862 has been marked as a duplicate of this bug. ***

[Hskupin (hskupin)]

David, shouldn't this be a blocker for bug 378635?

[Karoly Molnar (karoly-molnar)]

Binary package hint: mozilla-thunderbird

Hello,

After upgrading from Feisty to Gutsy the content of one folder and all its subfolders disappeared in thunderbird. Hopefully I could find the reason. The problem was that the folder had a "." in its name. The name was "Open Source Inc." so originally I had two files:
Open Source Inc.
Open Source Inc..msf
And one folder:
Open Source Inc..sbd

After the upgrade two new files were created:
Open Source Incc6bfcd54
Open Source Incc6bfcd54.msf

This lead me to a conclusion that the old thunderbird handled the "." as a dot and the new one replaced the "." with c6bfcd54 in the file name.

The solution was pretty simple, removing the "." in the old files and folder names and deleting the newly created files worked like a charm.

Thanks,
Karoly

[Saivann Carignan (oxmosys)]

This looks like a bug in Mozilla-Thunderbird itself. Maybe a folder compatibility issue between 1.5 and 2.0 ver. of Thunderbird?

Thanks for this detailed report.

[Saivann Carignan (oxmosys)]

I confirm this bug. I can reproduce it there.

[Mkmelin+mozilla (mkmelin+mozilla)]

*** Bug 417956 has been marked as a duplicate of this bug. ***

[Goldkind (goldkind)]

I've tried to rename the folders and managed to delete ONE of the two new duplicate gibberish-name folders. However, even after I have but ONE folder, and then RENAME it without any possible-offending characters, after I quit the program and then re-open it, the ORIGINAL bad folder returns, now with its ORIGINAL name: "!!Dana etc." Note that the name I gave it starts with two exclamation points, has a space between the words, and ends with a period. But, again, after I rename that folder to something without spaces or odd punctuation marks, it comes back after I reopen the program.

HELLLLLP!!!

[M-wada (m-wada)]

*** Bug 426144 has been marked as a duplicate of this bug. ***

[Hskupin (hskupin)]

(In reply to comment #21)
> But, again, after I rename that folder to something without spaces or odd
> punctuation marks, it comes back after I reopen the program.

This is not a user support forum but lets give a little help. You should rename this folder or MBOX file outside of Thunderbird. Close the application, rename the appropriate files/folders and delete the .msf file. After the next restart Thunderbird should come up with the new name and creates a new summary file.

[David-ascher (david-ascher)]

Given the age of 1.5, I think it's too late to consider this a blocking bug.

[M-wada (m-wada)]

*** Bug 475169 has been marked as a duplicate of this bug. ***

[Bozz (bozz)]

for what it's worth, I just experienced the "ending with period" subfolder problems in Mac OSX 10.4.11 whilst migrating from Netscape 7.2 to Thunderbird 2.0.0.19 (20081209). Ended up in a "duplicate report" [Bug 475169] because I wouldn't have thought to search the problem in anything prior to Tb 2.0.0.19...

[Saivann Carignan (oxmosys)]

Given that the bug report is reported upstream and that 1.5 is about to be unsupported on ubuntu (if not already), I'm closing this bug. Any future discussions should happen on the upstream bug : https://bugzilla.mozilla.org/show_bug.cgi?id=379101