Cannot delete file with bad filename

[Ofir Klinger]

Binary package hint: dolphin

I got stuck with a file which has a "bad" filename.

I have the following structure:
SomeDir
- ��� ���� �����!.txt

Trying to delete the file:
��� ���� �����!.txt
with Dolphin fails with an error message "File doesn't exists".

Trying to Shift+Del the entire SomeDir folder also gives the same error message.

I can't copy, move, rename or delete the file or parent folder (SomeDir).

I managed to delete the file by browsing with Dolphin to SomeDir and issuing the following command:
rm *.txt

[Ivo Anjo (knuckles)]

Version: (using KDE 4.0.83)
Installed from: Unlisted Binary Package
OS: Linux

Attached is a tar file with two folders with characters that aren't utf-8 encoded.

If I open it with ark, it seems to guess the right encoding, and shows the special characters, ("Relatório" and "Código" -- the problem is in the 'ó').

If I use dolpin tar:/... to navigate inside the file without extracting it, the special character just isn't shown (folders show up as Relatrio and Cdigo), but I can still navigate and see the files inside the folders.

Finally, if I extract the file (tar xvf <file>), and try to navigate inside the folders with the wacky names, I get an error saying The file or folder (...) does not exist, and It refuses to delete them too.

[Ivo Anjo (knuckles)]

Created attachment 25631
Tar file with broken encoding.

[Cgiboudeaux (cgiboudeaux)]

*** Bug 167523 has been marked as a duplicate of this bug. ***

[jortega (jorortega)]

This bug also affects full unicode names. I have a folder shared in windows xp with only unicode file names and files with only unicode characters (eg. spanish accented names or chinese/japanese/german named files). Dolphin behaves crashing sometimes (fully randomly) or just refuse to copy/change name the file, even with write permissions in the xp machine.

Also happens with local files, thus making dolphin/konqueror/kde 4.1 fully unusable for international users that need unicode compatibility.

[vsuarez (vsuarez)]

This bug forces me to install dolphin for KDE 3.5.x, which has not this problem. I have a Western Digital NAS that provides samba shares for accessing. For every file/folder with Spanish non-standard character, Dolphin shows this error message (translated from Spanish): "The file does not exist: smb://user@nas_ip/share_name/filename".

When I try to get the properties of a folder or file with this characteristics, Dolphin shows in the Permissions tab "forbidden" for user, group and others sets.

I also randomly get a Dolphin crash when browsing folders with this kind of filenames.

[Dario Andres (andresbajotierra)]

Is this related to bug 165400 ?

[jortega (jorortega)]

Probably.

Kde 4.1.1 comes with the bug included. Seems nobody cares.

J.

[ereslibre (ereslibre-kde)]

jorortega, I can perfectly use special characters (utf-8) system with Dolphin. Locally I mean, haven't tried with samba shares (it would be probably a related bug but on the smb:/ kioslave as well).

However, I can reproduce this issue when navigating with the tar:/ kioslave.

I also can say that trying to open this tar with ark from kde3 can't read those folders either.

[rabauke (sven-burmeister)]

Not being able to rename a local file, is a local problem. Hence this is not only related to some tar/smb ioslave.

[Ivo Anjo (knuckles)]

Yeah the problem is not about utf-8, it's about strange/broken encodings that make apps act strange (not able to access folder, delete it, ...).

[jortega (jorortega)]

I don't think so... kde 3.5.9 is able to manage all those files/folders/directoriers/shares/whatever nicely. Ark works well, konqueror is ok.

kde 4.0 and later have the problem. Seems that something is broken. Any file/directory/share/local/etc that contains:

- accents (spanish)
- Russian characters (Cyrilic)
- Janapense/chinese/korean/hebrew characters

just won't open in dolphin neither konqueror under kde 4 (An error message that indicates that "file "something%20blahblah%20.someext" doesn't exist") in some cases the folder listing will just ignore some files. I'm testing kde 4.1.1 in archlinux and kde 4.1 in kubuntu. Both distros have the same problem. Under Kde 3.5.9 in another machine, the same files works just OK. (Even the files attached to this error report just open ok in kde 3.5.9, not in 4.0)

Regards.

J.

[vsuarez (vsuarez)]

When I said "This bug forces me to install dolphin for KDE 3.5.x", I meant I installed dolphin 3.5.x on my KDE 4.1.1 instance. That is: I use KDE4 and Dolphin4, but when I need to access to a samba share, I must use Dolphin3 (running on KDE4).

I have no problems with local files and special (non-English) characters.

[Thiago Macieira (thiago-kde)]

We will no longer support broken encodings in KDE. We have had transition code for several years (at least since 2003) and I think 5 years is enough time for people to finish transitioning to UTF-8 environments.

This bug is about broken-encoded files. *Properly* encoded filenames should be
working and if they aren't, please open a new bug report on the subject.

You will hate me for this, but this bug is a WONTFIX. 5 years is enough time. If in 5 years you haven't renamed all your files, you should use the terminal to do it.

More info on why this is close to impossible to implement: http://lists.kde.org/?l=kde-core-devel&m=122025063320264&w=2

Do *not* reopen the bug.

[Thiago Macieira (thiago-kde)]

For information, WONTFIX here means more "CANNOTFIX" or "TOO_IMPRACTICAL_TO_FIX"

[rabauke (sven-burmeister)]

This is indeed working against users --> bad usability. No matter what the reason is, you screw the users, especially those with least experience and most likely files from others OSs.

Further, since renaming in konsole, i.e. mv "broken-name" "new name" works, makes your "close to impossible to fix" looks very silly.

[Thiago Macieira (thiago-kde)]

Read the link to kde-core-devel for the reasons.

[rabauke (sven-burmeister)]

Sorry, but you did not read my comment properly. The reasons given might make sense technically, but not usability-wise. As I said, new users, coming from other OSs, with files from those and no experience to work around the problem are lost. Huess what a user will do if he cannot work with his old music etc., blame it on Qt or Windows, or rather KDE and Linux?

If Qt only cares about the technical dimensions, fair enough, but this is KDE's bugzilla and hence WONTFIX not an option for this issue, IMHO.#

As I read in the mails, there are ways around this. So would it be possible to detect that something does not work with the file, although it exists and tell the user so, instead of just telling him that the file he sees does not exist? Further, one could offer the user to rename the file and use another than the "normal" function for renaming, nothing fancy, i.e. no conversion of e.g. é to e, just making the file work.

[ereslibre (ereslibre-kde)]

Despite I am not an encoding expert (Thiago is), I really believe we shouldn't drop our efforts on making legacy encodings working. We already have to do lots of things for caring about something that should have already been achieved (that is, filenames being in a properly encoding).

However, I also understand the user point of view. I wonder if other operating systems (as the newest from MS and Mac) handles this correctly.

[Thiago Macieira (thiago-kde)]

Windows and MacOS X are fully Unicode. There's no such thing as broken paths on Windows, since it uses UTF-16; on MacOS X, which is UTF-8, Finder shows as "C%F3digo" (i.e., URL encoding, which we do support), but simple applications like TextEdit crash while trying to use the file.

No, broken encodings are a thing of the past and should be confined to the terminal, where it belongs. Applications for the past 5 years have created files in UTF-8 encoding. Pen-drives use often VFAT, which is also Unicode clean.

It's not like a recent convert will get those broken-encoded files.

I think a 5-year transition period is more than enough. KDE 4 does not support non-UTF-8 locales anyways.

[vsuarez (vsuarez)]

Excuse me, then my problem is not related with wrong encodings, and I think also of the jorortega one. I will create a new ticket related to samba/cifs encodings and Dolphin4.

Jorortega, I think our problem occurs with file/folders named correctly from systems with full support of utf-8/unicode encoding (KDE 3.5.x/4 and Windows XP). Such created names are not well managed by Dolphin4 through a CIFS share.

[Ivo Anjo (knuckles)]

I understand that decoding that might be technically infeasible, because of Qt and QString. But can't we at least substitute it with ? so it appears like C?digo or something?

Not recovering the name properly, that I can understand. Giving strange and mysterious errors, and not letting you delete/move the files, that doesn't make any sense.

It reminds me of windows 98, where you created a folder on the command-line, using iso8859-15 characters, and then windows couldnt' access or delete it, so it was a (lame) way of keeping some files, or messing with users by creating something they couldn't delete. And that was very lame and bad.

[Thiago Macieira (thiago-kde)]

I am suggesting exactly that.

The question mark doesn't help because there's no way that the QString name (UTF-16) can be converted back to its system encoding. So, even if we showed with a ? in Dolphin, you wouldn't be able to edit or move it, rename it, etc.

[Bernt-lindhe (bernt-lindhe)]

Hi!

I have this problem to. I'm just a user so I don't know much about UTF and stuff.

All I can do is describe my problem.

I have a Ubuntu Server 8.04 with a samba share. On the share I have some files and folders with swedish characters. The files looks fine from the Ubuntu Server console. If i browse the samba share from Windows XP they also looks fine.

But if I browse the samba share from Kubuntu with KDE 4.1.1 with Dolphine it says that the file or folder don't exist if the file\folder contains swedish characters. Browse the same share with Krusader works just fine.

There is no problem with Dolphin and local files or folders with swedish characters.
If I copy a local file or folder with swedish characters using Dolhin to the samba share and then browse the share with Dolphin I get the same error.
And agin no proplem from Krusader, the server console or XP.

Looks like there is some problem with Dolphin.

Kind Regards
/Bernt

[Christopher Parker (cparker15)]

I think the WONTFIX resolution is pretty ridiculous. It's taking functionality away from a program and leaving an awfully broken mess behind.

I have a file server that I use to store all of my music that I access via SMB. I use KAudioCreator to rip CDs to a local directory ($HOME/Music) and then I move the music to the SMB share (ripping would take too long if I attempted to rip directly to the share).

I recently ripped a CD with KAudioCreator from a Finnish band that has letters with accents on them. I could access the files without a problem in the local directory. I then moved the files to the SMB share, and had issues with moving the files with accented characters in them (although the files appeared to have been successfully moved, as they no longer appeared in the local source directory).

Now, when I load the directory on the share that has these files in it, I'm immediately told the files don't exist, even though they clearly do, as they're showing up in the directory, although they show up as being 0B. I cannot rename them (although for some reason I'm unable to rename anything on the share). I cannot delete them. I cannot copy or move them back.

My question to you, Thiago, then, is: This is expected, normal behavior? I've somehow supposedly had five years to change my file-naming ways, even though another KDE application named the files to begin with? I think you need to re-evaluate how you're looking at this show-stopping bug. I now have no way of accessing these files, or other files that were pre-existing on the SMB shared drive that were perfectly accessible before the upgrade to KDE4. I experience this same issue when trying to access music from Björk, Green Jellÿ, Queensrÿche, Rinôçérôse, and more, which were all ripped under KDE3. If you're going to disable my access to my perfectly good files, you should at least be failing a lot more gracefully than this and providing an option for recovery while assuming I am an end user who doesn't know what a command line is.

[Thiago Macieira (thiago-kde)]

Yes, if your SMB server's setup is broken, that's the correct behaviour.

However, a properly setup server (including Windows machines as serves, for which no setup should be necessary) should work fine under Dolphin. If there's a problem with that, please file a new bug report.

This bug report here is about local files.

[jortega (jorortega)]

The can you at least take a look at bug 171819 please?

J.

[Helge Hielscher (hhielscher)]

And how is one to rename broken filenames? Is the solution to convert incoming files with another application first?

[Olidel-36 (olidel-36)]

Hello,

    I'm just wondering if that issue is a KDE issue or something which was changed in the packaging of my distribution (which is as of today october 15th, 2008 kubuntu 8.10 intrepid beta) because the issue described above looks to be solved. Yesterday, that was not yet working. Of course, I have updated my distribution between then and now, so I just know that something has been updated that corrected that issue but I have no idea of the package name which was responsible for the resolution. I just know that now the dolphin version is 1.1 using KDE version 4.1.2.

    There is also another issue which was solved at the same time it is the detail view which is working now properly when I browse a smb:/ address, Before today when I was clicking on the detail button I could be sure that dolphin would get frozen and the only thing that I could do it is to kill it.

Thanks.

O.D.

[Huynh Huu, Long (long-upcase)]

What about forcing an encoding like Nautilus does?

[Thiago Macieira (thiago-kde)]

Sorry, the problem is much deeper than that.

The *soonest* I could provide a solution would be in one year, then requiring a change in each KDE application (starting with Dolphin).

[FiNeX (finex)]

*** Bug 175764 has been marked as a duplicate of this bug. ***

[FiNeX (finex)]

@Thiago: I've reproduced this bug just now... shouldn't this report be reopened?

[FiNeX (finex)]

*** Bug 167097 has been marked as a duplicate of this bug. ***

[Thiago Macieira (thiago-kde)]

No, it shouldn't be reopened because we don't want to support wrong encoding. It's just far too much work, for very little gain. We don't have the infrastructure to handle this.

Also, note it's closed WONTFIX. That means "I won't fix". If you want to reopen, please reassign it to yourself. We'll be glad to review your patches and integrate them if they are good.

[Per Ångström (autark)]

If you don't have a "bad filename", this attached archive contains such a file.

[Per Ångström (autark)]

It's not just about deleting the file. Dolphin isn't able to pass the filename to the application it launches if I try to open the file. Nautilus handles this fine.

[Per Ångström (autark)]

The upstream bug is https://bugs.kde.org/show_bug.cgi?id=165044 . But don't expect it to be fixed - it is marked WONTFIX. There are plenty of duplicate bugs on the same issue.

[tdn (spam-thomasdamgaard)]

I also has this bug. This is really horrible. I cannot do anything with files from my mobile phone. My mobile phone records sound files that include a $'\370' char in the file name. Thus, when I have imported files, I cannot move, delete, rename or open them. I have to go to the shell and rename them with a shellscript.
Why is this WONTFIX in upstream?
Is there any chance Ubuntu/Kubuntu devs are going to address this themselves without upstream? I mean, it is really a big problem for people in non-english speaking contries.

[tdn (spam-thomasdamgaard)]

This is from upstream bug-report:
``We will no longer support broken encodings in KDE. We have had transition code
for several years (at least since 2003) and I think 5 years is enough time for
people to finish transitioning to UTF-8 environments.

This bug is about broken-encoded files. *Properly* encoded filenames should be
working and if they aren't, please open a new bug report on the subject.

You will hate me for this, but this bug is a WONTFIX. 5 years is enough time.
If in 5 years you haven't renamed all your files, you should use the terminal
to do it.

More info on why this is close to impossible to implement:
http://lists.kde.org/?l=kde-core-devel&m=122025063320264&w=2

Do *not* reopen the bug.''

---------

I think their reasons for not supporting broken encodings are valid. However, I think that at least they should provide a tool for fixing the broken encodings. E.g. allowing the user to rename the files to a correct encoding.

[rubengs (rgarcia-cucea)]

I use this scrpt to fix such bad filke names:
____________________________________________________________________________________
#!/bin/bash

#NTFS to UTF-8

if [ $# -gt 0 ];then

    convmv -f cp850 -t utf-8 -r --notest "$@"| zenity --progress --pulsate --text="conversion in progress" --auto-close

fi

exit 0

[RalfGesellensetter (rgx)]

Am Mittwoch, 20. Februar 2013 schrieb Hans-Peter Jansen
> Nobody reads this bug report, nobody cares. Thiago removed himself from CC list
> a loooong time ago, so you can target your comment to him, but he won't read
> it.

Hello,

we should make more people vote for this bug in order to draw more attention
to it. This bug is indeed annoying, but it is possible to drop dolphin without
dropping KDE.

What file manager is working with you? I use mc all the time in such cases,
and for copying anyway (because of the time stamp bug).

Kind Regards
Ralf

[Michal Hlavinka (mhlavink)]

(In reply to comment #138)
> we should make more people vote for this bug in order to draw more attention
> to it. This bug is indeed annoying, but it is possible to drop dolphin
> without
> dropping KDE.

Seems you don't understand this bug. It's not a bug in dolphin. It's not a bug in KDE. It's bug in Qt library. All Qt applications have this bug. Because KDE uses Qt library it means also all KDE applications are affected and because Dolphin is KDE application, that's why it is affected. This bug mentions just Dolphin, but all KDE and Qt apps have this bug. The file, you can't work with in Doplhin, can't be opened by any KDE nor Qt application.

[RalfGesellensetter (rgx)]

Am Donnerstag, 21. Februar 2013 schrieben Sie:
> All Qt applications have this bug. Because KDE
> uses Qt library it means also all KDE applications are affected and because
> Dolphin is KDE application, that's why it is affected. This bug mentions just
> Dolphin, but all KDE and Qt apps have this bug. The file, you can't work with
> in Doplhin, can't be opened by any KDE nor Qt application.

Hi and thanks for clarification.

1) My approach was to use another file manager in order to
fix broken file names, so just replace dolphin.

2) Yesterday, I had a malformed file name, and in Dolphin it
was not possible to copy it to another folder. However, I
could open that (PDF) file from Dolphin into Okular, then
I saved a copy as new file, and could also remove the
file with broken file name from within Dolphin.

dolphin --version
Qt: 4.8.2
KDE: 4.8.4 (4.8.4)
Dolphin: 2.0

But this was no Umlaut. Just double checked, and still malcoded
Umlauts are displayed like <?> and transscribed like this when
you drag&drop file names to non-KDE applications or Konsole.

Rather than sending plain (flatterned) file names to KDE apps,
it could be considered to sending direct inode references
within QT/KDE applications, couldn't it?

3) Next test: I installed xfe (+package xfe-i18n) as alternative
file manager. Interestingly, xfe displays "broken" umlauts
correctly (auto detection?), and passes the file name
string correctly to xpdf, so the file opens as it should.

But right you are: No chance to send it to okular.

Accordingly, file names have actually to be fixed before
they will work with KDE.

Is there a bug, "KDE can't handle file names beyond official encoding"?
There should probably be some legacy switch for this support, as
filenames containing binary code might compromise applications
when parsing them from environment.

Kind regards
Ralf

[Frank78ac (frank78ac)]

*** Bug 315715 has been marked as a duplicate of this bug. ***

[Szo (szo)]

Created attachment 78460
Working fix

This attacment is a patch against kde4libs-4.10.1.
It works around the problem like this: in KLocalePrivate::initFileNameEncoding() KDE sets the QFile's encoding/decoding function, to to/fromUTF8() in QString, which in turn calls QUtf8's converter function (QUtf8 is not exported to developers, so I had to use an inefficient method, I think it would be better if we could use the state parameter for error detection). I replaced this with the said functions' copy/pasted version and changed it, so when it encounters an invalid UTF8 string, it will encode it byte by byte, mapping the lower 128 their normal unicode place and the upper 128 to U+18000-U+1807F, and of course the decoder reverses it.
To make this actually work you have to define the KDE_UTF8_FILENAMES enviroment variable (otherwise we would need to patch at QT level).
So, do the following:
.kde/env/KDE_UTF8_FILENAMES.sh
with this content:
export KDE_UTF8_FILENAMES=yesplease

logout, login, try dolphin on faulty files. (instead of the usual boxed "?" you'll see just boxes)

HTH

[Ivo Anjo (knuckles)]

I built kdelibs with the patch, exported the KDE_UTF8_FILENAMES and it seems to work as advertised: strange chars are replaced by boxes on the filenames, but I can rename and delete them sucessfully.

I'll keep using the patch on one of my machines to see if there are any other issues.

Big thanks to Szokovacs Robert, and to all the people that complained loudly for all these years: this is your chance. Test the patch, use it, and if it works, bug your favorite KDE developer, or distribution to include it. Tell your friends to use it. Package and distribute it. Don't just complain and leave. I reported this bug in 2008, and I'm still here :)

P.s.: Applying the patch gave me a whitespace warning:
encoding.patch:173: trailing whitespace.

[Szo (szo)]

Maybe the fix would be more acceptable in the mainline if the new behaviour would only apply if the user sets KDE_UTF8_FILENAMES to a specific value, for example "broken_names".

[Szo (szo)]

Fixed in http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
and
https://projects.kde.org/projects/kde/kdelibs/repository/revisions/736d5237f822fc72736f75f379c4f86d6bf48098

[Ivo Anjo (knuckles)]

Thanks for your work!
It seems that for all the complaining, nobody bothered to test your patch, or even say thanks. It's a pity.

[Smartinds (smartinds)]

(In reply to comment #145)
> Fixed in
> http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
> and
> https://projects.kde.org/projects/kde/kdelibs/repository/revisions/
> 736d5237f822fc72736f75f379c4f86d6bf48098

YES, YES, YES.

Thanks!. I was waiting years for this. (^_^)

[Helder Meneses (geneiros)]

Thank you for your work...almost 5 years later...it's not your fault, off course...

but this and other problems made me quit using kde, more of the atitude of the developers in some bugs...

maybe one day i install kde only to see if this is fixed...it was a major bug to me, because i'm portuguese, and some files didn't open because of the wrong encoding...

keep alive and kicking... :)

[Cyberbeat-p (cyberbeat-p)]

I did not have such encoding problems since a long time, because I did not work with dual-boot -windows anymore, but nice to know that it works now :-) thanks

[Palkeo (palkeo)]

Nice to know that !
Thanks :)

[Szo (szo)]

(In reply to comment #146)
> Thanks for your work!
> It seems that for all the complaining, nobody bothered to test your patch,
> or even say thanks. It's a pity.

Actually there was some discussion on the reviewboard, and some delay was caused by me - I did not realize that I just need to apply for a developer account and not wait around for someone elso to commit for me :) Also, I forgot to include this bug in the commit, sorry about that.

[RalfGesellensetter (rgx)]

Thank you, Robert! You're a real hero!
Regards
Ralf

Am Donnerstag, 4. Juli 2013 schrieben Sie:
> https://bugs.kde.org/show_bug.cgi?id=165044
>
> --- Comment #145 from Szokovacs Robert <email address hidden> ---
> Fixed in
> http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
> and
> https://projects.kde.org/projects/kde/kdelibs/repository/revisions/736d5237f822fc72736f75f379c4f86d6bf48098
>
>

[Wyatt-epp (wyatt-epp)]

(In reply to comment #145)
> Fixed in
> http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
> and
> https://projects.kde.org/projects/kde/kdelibs/repository/revisions/
> 736d5237f822fc72736f75f379c4f86d6bf48098

Thank you so much! And, just to clarify, this works without setting KDE_UTF8_FILENAMES as suggested previously?

[vsuarez (vsuarez)]

Thank you, gracias!

[Hpj-u (hpj-u)]

This is a great day for KDE, and you made it, Robert.

Thanks for your patience and I really appreciate, that sanity finally won over arrogance.

BTW, KDEPIM is in deep need of people like you, Robert <wink> ;-)

[Szo (szo)]

(In reply to comment #154)
> (In reply to comment #145)
> > Fixed in
> > http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
> > and
> > https://projects.kde.org/projects/kde/kdelibs/repository/revisions/
> > 736d5237f822fc72736f75f379c4f86d6bf48098
>
> Thank you so much! And, just to clarify, this works without setting
> KDE_UTF8_FILENAMES as suggested previously?

Yes, if your locale is any kind of UTF8, the fix will kick in.

[Montosh-bisht (montosh-bisht)]

Thank you will check this as soon as I have a working
system (my HDD died) awaiting a new one.
Will be testing under Fedora 19 KDE :)
Right now online with Fedora 18 KDE live boot :)

Regards,
Monts

On 4 July 2013 16:33, Szokovacs Robert <email address hidden> wrote:

> https://bugs.kde.org/show_bug.cgi?id=165044
>
> --- Comment #157 from Szokovacs Robert <email address hidden> ---
> (In reply to comment #154)
> > (In reply to comment #145)
> > > Fixed in
> > >
> http://commits.kde.org/kdelibs/f4269ef3498581964e8a1a13cd0d6d7f19c88762
> > > and
> > > https://projects.kde.org/projects/kde/kdelibs/repository/revisions/
> > > 736d5237f822fc72736f75f379c4f86d6bf48098
> >
> > Thank you so much! And, just to clarify, this works without setting
> > KDE_UTF8_FILENAMES as suggested previously?
>
> Yes, if your locale is any kind of UTF8, the fix will kick in.
>
> --
> You are receiving this mail because:
> You are on the CC list for the bug.
>

[Christoph-maxiom (christoph-maxiom)]

In Qt 5 the filename encoding callbacks were removed, so we will face this issue again in a few years. Let's enjoy Róbert's work for the time being.

http://qt-project.org/doc/qt-5.1/qtcore/qfile-compat.html#setEncodingFunction

[Frank78ac (frank78ac)]

*** Bug 325114 has been marked as a duplicate of this bug. ***

[Christoph-maxiom (christoph-maxiom)]

*** Bug 254966 has been marked as a duplicate of this bug. ***

[Kde-2011-08 (kde-2011-08)]

Additional test files are available as attachements to Bug 254966.

[Cfeck (cfeck)]

*** Bug 237817 has been marked as a duplicate of this bug. ***

[Cfeck (cfeck)]

*** Bug 349898 has been marked as a duplicate of this bug. ***

[Victor Tran (vicr123)]

*** Bug 349996 has been marked as a duplicate of this bug. ***

[Kde-20091112 (kde-20091112)]

I just extracted an archive which contained a file that I could not copy/edit/move/rename/delete with dolphin. The file was simple readme file inside a nested subdirectory called L'$'\351''ame.html'

The error messages in dolphin were extremely vague and did not indicate in an way that KDE and QT are incapable by design of handling files with these encoding. They did not inform the user why the file was causing these problems or even hint that they needed to be renamed in a terminal because of a WONTFIX on this bug.

The only reason I knew what to do is because I encountered this bug years ago when operations on these files actually worked. Any normal user will have absolutely no idea what to do, and higher level systems will not handle these failures gracefully when these directory structures can't be modified because of a simple filename.

The file or folder L'$'\351''ame.html' does not exist.
 - This is _false_ it does exist!
Could not delete file L'$'\351''ame.html'.
 - No reason provided whatsoever. The user will not know how to solve this problem.
Could not remove folder "".
 - All other files deleted, but the nested file was ignored leaving the user confused.

This is ridiculous.

It might be a lot of work, but a solution to this problem would be to store these strings as a tuple, where there is a degraded and user-friendly version of the string and the original unmodified string. If no changes are made to the string, use the original unmodified string. If the user decides to change the string, then they are interacting with the degraded version and deciding to keep that new string.

[deja_vu (deja-vu)]

unsubscribe

> Gesendet: Donnerstag, 18. April 2019 um 00:05 Uhr
> Von: "Nathan Shearer" <email address hidden>
> An: <email address hidden>
> Betreff: [kdelibs] [Bug 165044] Dolphin can't handle well files/folders with wrong encoding
>
> https://bugs.kde.org/show_bug.cgi?id=165044
>
> Nathan Shearer <email address hidden> changed:
>
> What |Removed |Added
> ----------------------------------------------------------------------------
> CC| |kde-20091112@nathanshearer.
> | |ca
>
> --- Comment #166 from Nathan Shearer <email address hidden> ---
> I just extracted an archive which contained a file that I could not
> copy/edit/move/rename/delete with dolphin. The file was simple readme file
> inside a nested subdirectory called L'$'\351''ame.html'
>
> The error messages in dolphin were extremely vague and did not indicate in an
> way that KDE and QT are incapable by design of handling files with these
> encoding. They did not inform the user why the file was causing these problems
> or even hint that they needed to be renamed in a terminal because of a WONTFIX
> on this bug.
>
> The only reason I knew what to do is because I encountered this bug years ago
> when operations on these files actually worked. Any normal user will have
> absolutely no idea what to do, and higher level systems will not handle these
> failures gracefully when these directory structures can't be modified because
> of a simple filename.
>
> The file or folder L'$'\351''ame.html' does not exist.
> - This is _false_ it does exist!
> Could not delete file L'$'\351''ame.html'.
> - No reason provided whatsoever. The user will not know how to solve this
> problem.
> Could not remove folder "".
> - All other files deleted, but the nested file was ignored leaving the user
> confused.
>
> This is ridiculous.
>
> It might be a lot of work, but a solution to this problem would be to store
> these strings as a tuple, where there is a degraded and user-friendly version
> of the string and the original unmodified string. If no changes are made to the
> string, use the original unmodified string. If the user decides to change the
> string, then they are interacting with the degraded version and deciding to
> keep that new string.
>
> --
> You are receiving this mail because:
> You are on the CC list for the bug.

[Cfeck (cfeck)]

Removing complaining user.

[Ivo Anjo (knuckles)]

(In reply to Nathan Shearer from comment #166)
> I just extracted an archive which contained a file that I could not
> copy/edit/move/rename/delete with dolphin. The file was simple readme file
> inside a nested subdirectory called L'$'\351''ame.html'
>
> The error messages in dolphin were extremely vague and did not indicate in
> an way that KDE and QT are incapable by design of handling files with these
> encoding. They did not inform the user why the file was causing these
> problems or even hint that they needed to be renamed in a terminal because
> of a WONTFIX on this bug.
>
> The only reason I knew what to do is because I encountered this bug years
> ago when operations on these files actually worked. Any normal user will
> have absolutely no idea what to do, and higher level systems will not handle
> these failures gracefully when these directory structures can't be modified
> because of a simple filename.
>
> The file or folder L'$'\351''ame.html' does not exist.
> - This is _false_ it does exist!
> Could not delete file L'$'\351''ame.html'.
> - No reason provided whatsoever. The user will not know how to solve this
> problem.
> Could not remove folder "".
> - All other files deleted, but the nested file was ignored leaving the user
> confused.
>
> This is ridiculous.
>
> It might be a lot of work, but a solution to this problem would be to store
> these strings as a tuple, where there is a degraded and user-friendly
> version of the string and the original unmodified string. If no changes are
> made to the string, use the original unmodified string. If the user decides
> to change the string, then they are interacting with the degraded version
> and deciding to keep that new string.

Hello there -- I'm the original reporter of this bug.

Since this one is marked as fixed, my suggestion would be to open a new bug report, as the issue you saw may be a regression, rather than the same as this bug, and it helps keep the discussion focused.

Also, if possible, my suggestion is to also include an archive that triggers this issue as an attachment, so developers can quickly jump on this one.

Happy KDE'ing and happy easter everyone!

[Szo (szo)]

(In reply to Ivo Anjo from comment #169)
> (In reply to Nathan Shearer from comment #166)
> > I just extracted an archive which contained a file that I could not
> > copy/edit/move/rename/delete with dolphin. The file was simple readme file
> > inside a nested subdirectory called L'$'\351''ame.html'
> >
> > The error messages in dolphin were extremely vague and did not indicate in
> > an way that KDE and QT are incapable by design of handling files with these
> > encoding. They did not inform the user why the file was causing these
> > problems or even hint that they needed to be renamed in a terminal because
> > of a WONTFIX on this bug.
> >
> > The only reason I knew what to do is because I encountered this bug years
> > ago when operations on these files actually worked. Any normal user will
> > have absolutely no idea what to do, and higher level systems will not handle
> > these failures gracefully when these directory structures can't be modified
> > because of a simple filename.
> >
> > The file or folder L'$'\351''ame.html' does not exist.
> > - This is _false_ it does exist!
> > Could not delete file L'$'\351''ame.html'.
> > - No reason provided whatsoever. The user will not know how to solve this
> > problem.
> > Could not remove folder "".
> > - All other files deleted, but the nested file was ignored leaving the user
> > confused.
> >
> > This is ridiculous.
> >
> > It might be a lot of work, but a solution to this problem would be to store
> > these strings as a tuple, where there is a degraded and user-friendly
> > version of the string and the original unmodified string. If no changes are
> > made to the string, use the original unmodified string. If the user decides
> > to change the string, then they are interacting with the degraded version
> > and deciding to keep that new string.
>
> Hello there -- I'm the original reporter of this bug.
>
> Since this one is marked as fixed, my suggestion would be to open a new bug
> report, as the issue you saw may be a regression, rather than the same as
> this bug, and it helps keep the discussion focused.
>
> Also, if possible, my suggestion is to also include an archive that triggers
> this issue as an attachment, so developers can quickly jump on this one.
>
> Happy KDE'ing and happy easter everyone!

It is in deed a regression, the fix depended on a QT feature that was removed in 5.x, iirc.

[Strangiato Xanadu (strangiato)]

(In reply to Szokovacs Robert from comment #170)
> It is in deed a regression, the fix depended on a QT feature that was
> removed in 5.x, iirc.

Is there a report in Qt tracker asking for bring back this feature removed in 5.x?

[Cfeck (cfeck)]

https://bugreports.qt.io/browse/QTBUG-59402

[Cfeck (cfeck)]

Git commit 6738a8b2f71c527f30a624b0b560f79d992715d3 by Christoph Feck.
Committed on 21/05/2019 at 11:05.
Pushed by cfeck into branch 'master'.

[kioslave/file] Add a codec for legacy filenames

UNIX filenames can contain any bytes (except \0 and /).
Qt's QFile::decodeName() calls QString::fromLocal8Bit(), assuming that all
filesystems use the system's locale encoding. For filenames that have been
created with a different encoding, and have not yet been converted (e.g. using
convmv), this creates non-reversible U+FFFD (REPLACEMENT CHARACTER)
code points in the filenames.

For example, some old-style archives might not contain any information about
the encoding of the filenames, and even today archivers extract them without
trying to convert to the locale's encoding.

While full support for those filenames is not needed, Dolphin should at least
be able to delete, rename, and move those files. Since all actual (local) file
handling is done inside the file kioslave, patching Dolphin will not help.

This code is a near verbatim copy of the code we had in kdelibs, written by
Szókovács Róbert. Only minor adaptions to Qt5 were done. It decodes invalid
bytes as U+10FExx from Plane 16 (Supplementary Private Use Area-B) to be able
to encode them later.

Dolphin could detect filenames with those characters, and either mark them
(by color or overlay icon), or even automatically offer to rename them.
Related: bug 204768

TEST PLAN

touch "/tmp/test-"$'\377'".txt"
dolphin /tmp

Copying and deleting a test file worked with this code, failed without.

Reviewers: dfaure, Frameworks, Dolphin

Reviewed by: dfaure

Differential Revision: https://phabricator.kde.org/D18161

M +1 -1 src/ioslaves/file/CMakeLists.txt
M +8 -0 src/ioslaves/file/file.cpp
A +174 -0 src/ioslaves/file/legacycodec.cpp [License: LGPL (v2+)]
A +66 -0 src/ioslaves/file/legacycodec.h [License: LGPL (v2+)]

https://commits.kde.org/kio/6738a8b2f71c527f30a624b0b560f79d992715d3

[Strangiato Xanadu (strangiato)]

test plan from comment 173 fails on Arch Linux after upgrade to frameworks 5.59.
Dolphin says "The file or folder /tmp/test-�[U+10FE7F]�.txt does not exist."

[Cfeck (cfeck)]

That's because this file name is using the same private code points that are used to encode not correctly encoded UTF-8 file names. You cannot support both.

[Strangiato Xanadu (strangiato)]

Well, now at least I can rename and then delete the file. :)

[Nate-b (nate-b)]

*** Bug 481720 has been marked as a duplicate of this bug. ***