mc directory sizes not reliable

[Blue]

Binary package hint: mc

In lucid lynx mc reports erroneous directory sizes even between two consecutive invocations.
It also seems to try to cache the results somewhere, but it is not reliable at all.
How to reproduce :
F9, Command,Show directory sizes.
repeat (eventually sort by sizes in between)
Observe how the directory sizes change from one invocation to another.

[Yury V. Zaytsev (zyv)]

I can't reproduce this with my packages, which probably means that the issue has already been fixed and will make it into Ubuntu / Debian when our package will get uploaded.

https://launchpad.net/~zyv/+archive/ppa

[Blue (vali-dragnuta)]

I tried your package from your ppa and seems to do the same.
However I might have missed something : it seems that this unreliability manifests itself on nfs mounted directories with quite a large number of files (and megabytes ...)

[Blue (vali-dragnuta)]

...not only on nfs after all (see screenshot).

[Yury V. Zaytsev (zyv)]

Unfortunately I was able to reproduce this issue. I will submit this upstream.

[Yury V. Zaytsev (zyv)]

I had the same effect on

//xxx/yyy on /media/calculon type cifs (rw,mand,noexec,nosuid,nodev,user=zyv)

mc shows the directory size of 9 mb, but du -ch shows 231 mb. If I copy this folder to the local file system, the size matches. I think it's a du's problem, not mc. Try to compare with Nautilus or something else that can show you the file size.

I suspect this bug is invalid.

[Blue (vali-dragnuta)]

How could be the bug invalid if du's reporting is always ok ?
It can however be reproduced on the local filesystem. I am able to reproduce locally in many cases using the following scenario :
1). Go to a populated and loaded directory. For example a quite heavily used user home.
2). Enter one subdirectory - for example .gimp-2.6. In this directory call mc's directory sizes. Initially it will properly count everything.
3). Go back one directory in the user's home
4). Call directory sizes again.

When i do this, almost every time counting it is NOT made again,and it only puts a size on the previously counted directory (.gimp-2.6 in this case). All other directories remain with sizes of 4096 bytes or 0.

[Yury V. Zaytsev (zyv)]

How do you know that what du reports to you is always OK??? You didn't seem to have performed any independent validation for this. As I mentioned above du on a cifs-mounted filesystem is clearly bullshitting me, showing that a 9 mb directory takes 231 mb. mc's output, however, is consistent with the true directory size which I obtained by counting the files manually and the one shown by Nautilus.

In what concerns your other points, you obviously do not understand the way ls and directory size count features work.

1) Unless mc is explicitly asked to count the size of the directory it defaults to the output of ls, which is to report the amount of blocks currently used for the directory entry. That's why in most of the cases it shows 4096 for you, as your file system uses 4K blocks.

2) Count directory sizes is a CPU and time intensive operation and is never performed unless you ask mc to do this explicitly.

3) When you exit the directory, the results of the counting operation are not cached, so when you re-enter it again you have to recount the whole thing again.

I don't see any bug here.

[Blue (vali-dragnuta)]

Let me reissue :

we have the following structure :
A
|-- B
| |-- X
| |-- Y
| |-- Z
| |-- file1
| |-- file2
| `-- file3
|-- C
| |-- file1
| |-- file2
| `-- file3
|-- D
|-- E
`-- F

We enter directory B and request directory sizes. mc will count everything that resides in directory B and caches the result.
Now we are getting BACK to the parent directory A and request directory count again. Please keep in mind that untill this moment we only requested directory counts while we were in directory B.
Now when we request counting while in parent directory A we would expect correct counting for all directories B,C...F.
However, instead of that mc just fetches the total for directory B (in which we previously ran directory count). IT WILL NOT COUNT THE DIRECTORIES THAT ARE ON THE SAME LEVEL : C,D,E and F.
IT IS A BUG.

[Blue (vali-dragnuta)]

Please see the attached screen cast.
See how I descend in my home, in .gimp-2.4 and in .gimp-2.4 i run directory count.
Then, i go back to the parent directory (which is my home directory ) and run directory count there, too.
Instead of counting that directory, mc limits itself to displaying ONLY ONE size, that of the directory where i previously run directory count.
Then, for the argument's sake I also descend into gimp-2.6, where i run directory count and learn that the directory is not empty at all.
I descend back to the parent directory, run directory count again, and this time mc only gives me the correct size for the previously counted directory ONLY, instead for all the objects in the directory where I run directory count.

[Blue (vali-dragnuta)]

...another recording of the same issue, this time on a nfs mounted directory. It is clearly visible how mc itself forgets the correct size from one interrogation to another.

[Yury V. Zaytsev (zyv)]

This is not really a bug, but rather a change in the behavior of the "Show directory sizes" command. Previously it would calculate the sizes of all directories in the current directory if no selection was made, otherwise the size of the selected directories.

Now if the cursor is positioned on a file it will do nothing, on a directory --- calculate the size of the highlighted directory, in case if some directories were selected --- the size of these directories and if it's on "..", the size of the current tree.

Whether this is more intuitive or not is debatable, I will try to discuss this behavioral change with upstream.

[Blue (vali-dragnuta)]

Well, EVEN SO it fails, because it sometimes calculates (as previously expected) the sizes for all the content in the current directory and in other cases it does compute any sizes AT ALL or just limits itself to count only ONE directory from all the (sub)directories in the current directory.
In my screencasts i never selected any subset of files, and i still got three different results for the same operation :
- correct counting, sometimes, for all the objects in the current directory. This was the previous mode of operation, and the expected one.
- no counting at all
- only partial counting

[Yury V. Zaytsev (zyv)]

Will you read my previous message explaining the new behavior before posting more follow-ups to this bug? I explained the new behavior and my explanation is consistent with your screencasts. I will have a look at how to find a best compromise between the old mc behavior and the new FAR-like behavior, but either way, the current behavior is consistent with me explanation.

[Blue (vali-dragnuta)]

I understand what you say. Unfortunately the new behaviour is not intuitive at all IMHO.
Select ".." to count the CURRENT directory ?
Hope a smarter way is found.
I will also try to recheck the network fs mc vs du issue if i will be able to reproduce it.