Preview time is slow with many files to compare

[ROCKNROLLKID]

BleachBit crashes or hangs everytime it tries to clean out Steam Games, Steam Installers, and Steam More. Steam Big picture cache and Steam browser cache works fine. Three systems reported the same issue. This issue has existed for a long time now.

[Andrew Ziem (ahziem1)]

By "crash" do you mean BleachBit closes? Is there a log file saved? If so, please include the log.

If BleachBit doesn't close, do you mean an error log (like "access denied") shown in the application ?

Is any Steam process running?

Windows 7?

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

I am under a windows 7 64-bit system. The same issue was reported under a 32-bit system as well. There is no error at all. BleachBit will just hang and white out saying BleachBit is not responding. Only the 3 I listed are having the issues. It doesn't matter if it is running or not, it still hangs. There is nothing in the log file that replicates this issue. The entries seem to have came from Winapp2, I believe. They work fine in CCleaner.

[Andrew Ziem (ahziem1)]

Does the preview work? Are there thousands of files? Or file names with odd characters? Is overwrite file contents enabled?

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

Neither clean or preview work. It doesn't say any number of files just when one of them starts nothing happens. I know stuff like Direct X installation files and cabs files exist in one of them. Yes overwrite is enabled on mine, but it still does it even if it is disabled.

Can you please try this under Windows 7 64-bit SP1.
1. Install latest BleachBit (a.k.a 1.0)
2. Install latest winapp2 (either the modified or the one on winapp2)
3. Install latest steam.
4. Install a few games like Half-life 2, left 4 dead 2, and Team fortress 2.
5. Play the games alittle and change settings in games.
6. Once done, try cleaning out the selected 3 and see if it works for you.

[Andrew Ziem (ahziem1)]

I tried something similar using one free game, which wasn't working correctly. This didn't reproduce the problem, so I will try again later.

(If preview is affected, then overwrite doesn't matter)

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

I know steam did some updates on their directory paths and it was changed recently. Games were moved to the common foldier and the main path directory was changed to steamapps, I don't remember the old path. Make sure BleachBit cleaning rules were change to match the new directories and not the old one. I don't know if that helps, otherwise some free games would be Team Fortress 2, Planetside 2, APB Reloaded, etc.

[Andrew Ziem (ahziem1)]

Steam doesn't run very well on my PC. After installing a game, it doesn't play well. But still, I don't see anything freeze or crash.

Two questions

1. You wrote Steam Games, Steam Installers, and Steam More crashes. Did you try each individually? The first two have some complicated patterns, but Steam More looks simple.

2. Assuming Steam More crashes by itself, would you please run this command and provide the output file by either attaching it or directly emailing it?

dir %ProgramFiles%\Steam %ProgramFiles%\Valve > %UserProfile%\Desktop\SteamValveDir.txt

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

Unfortunetly, I don't have my pc up and running, yet, and probably won't be for awhile still. As soon as I get it up, I will test this and report back.

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

Hello. I am currently operating on a lowtech system, a friend of mine, who was having the same issue. I am not finding any file called SteamValveDir.txt. What exactly is it that you want me to do?

[Andrew Ziem (ahziem1)]

Running that command in a command prompt (CMD.EXE) window creates a listing of files called SteamValveDir.txt on the desktop. I'm wondering how many files there are, whether any have any strange names, and how they match up to the patterns.

Can the friend do a preview just for Steam More ?

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

Sorry for the long delay. I am now back on my PC. I tried searching that dir on my pc with CMD.exe, but it says The system cannot find the file specified, then I tried a CMD with just SteamValveDir.txt and it said "SteamValveDir.txt" is not recognized as an interal or external command, operable program or batch file. There doesn't seem to be a file in existence on my system.

[Andrew Ziem (ahziem1)]

Sorry, how about with quotation marks?

dir "%ProgramFiles%\Steam" "%ProgramFiles%\Valve" > "%UserProfile%\Desktop\SteamValveDir.txt"

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

I tried with quotes in it as you had it the cmd said file not found but it seemed to have created a txt file saying the following:
 Volume in drive C has no label.
 Volume Serial Number is DED9-195E

 Directory of C:\Program Files

 Directory of C:\Program Files

[Andrew Ziem (ahziem1)]

I am offering a bounty to fix this bug
https://www.freelancer.com/projects/5396617.html

as an experiment

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

Hi. As I came back to this issue and I tested it out. It seems the issue is that bleachbit takes a really long time to clean this out, whether it's previewing or cleaning. It doesn't crash, or hang, it just takes a long time. You can cancel the offer on freelancer, as I have found out the real issue. I ran a preview test and it took over 10minutes just to preview the steam content section. The cleaning took longer with pass wipe enabled. The files don't seem to be any bigger then 30mb, so I don't know why it is taking so long on mine.

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

Here is the result of the test I just recently took. This is running only a preview test.

Steam Big Picture Cache - Completed instantly but didn't find anything on my system. This is nothing to worry about because I have had this cleaned already and I don't use Big Picture.

Steam Browser Cache - 1 second, 40.1 MB, 259 files.

Steam Games - 5 minutes, 185MB, 300files.

Steam Installers - I did not run through the whole thing. I waited 10 minutes and it still wasn't done. During the 10 minutes, it didn't find anything at all. It didn't seem like it was doing anything.

Steam More - ~40 seconds, 142.7 KB, 5 files. The time for this one is still long for what little it found.

Steam Big Picture and Browser Cache are acting normal, but the other 3 are taking a really long time. I tried running the cleaner after it and it takes the same amount of time, even longer with pass wipe enabled, to clean them out.

[Theatre-X (theatrex)]

Yes, some of the cleaners take awhile to clear out files.
I opened a bug about BB taking awhile to clear out Firefox before.

By the way, which Steam were you using?
The one from Github or Winapp2.ini?

[Andrew Ziem (ahziem1)]

ROCKNROLLKID:
How many files do you have in %ProgramFiles%\Steam\Steamapps\common? (This is the Steam installers and Steam games.) I'm guessing there are a LOT of files, most of which do not match any pattern to delete.

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

@Gogeden: The issue is that BB is taking longer then it should, as if the cleaning rule for steam needs to be updated. I am using winapp2 from winapp2.com. I am using the latest steam version, as well.

@Andrew: I have 62,321 files, 1607 folders, and a total of 37.4 GB (40,328,077,312 bytes to be exact). I don't think the problem has anything to do with the number of files or how big they are. As I said, it is taking a really long time to clean even when pass wipe is disabled.

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

@Gogeden: Also, the steam version on github is for Linux. I am a windows user.

[Andrew Ziem (ahziem1)]

ROCKNROLLKID, preview time is proportional to the number of files that need to be scanned and compared to the pattern, so it has to scan 62,321 files to find about 300 files.

(Cleaning time is the same as preview time PLUS proportional to the size of the files, especially with overwrite enabled.)

So based on this, I am focus this bug ticket on speeding up preview time.

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

I don't think you guys understand what my issue is. The issue has nothing to do with preview speeds or cleaning speeds. The issue on my system is that, when I run the 3 selected entries, BleachBit hangs and takes a really long time. When I have all other entries selected and not the 3, it takes usually ~20seconds. Yes, I will admit BleachBit cleaning and preview times are a little slower then the average cleaner. When I have the time, I'll try to get a desktop recording to show you what I am talking about.

[Andrew Ziem (ahziem1)]

My understanding is

1. The preview hangs
2. The preview hangs because it's slow
3. The preview is slow because there are 62,321 files to compare to the pattern (which matches a small subset)

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

I see what you are getting at now. That could be the issue, but I am not sure. This doesn't happen with another entry that I know of, but I do see a drag with preview and cleaning when cleaning out entries. Maybe in 1.1 or 2.0, we can look into speeding up preview and cleaning times.

[Andrew Ziem (ahziem1)]

Other entries are probably not scanning 60,000 files. Also this case is unique because these Steam winapp2.ini entries are converted to a complicated regular expression. I'll check later

[Andrew Ziem (ahziem1)]

Posted a paid job https://www.odesk.com/jobs/Improve-CPU-speed-Python-program_~013255c9c368a69560

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

What is the status on this issue? I notice you change the milestones around quite a lot on here.

[Andrew Ziem (ahziem1)]

While it doesn't address the whole issue, I added an internal benchmark. Then I added a step to pre-compile the regular expressions, which boosted performance 5% on one system and 50% on another.

https://github.com/az0/bleachbit/commit/7ec7374051aea749cdd121e289684c4eecc4d6ab

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

When you did the 5 - 50% boost increaser in 1.10, it seemed to have sped this up to a tolerable speed now. There is still, however, a drag to this, especially if the entry has a lot of clutter to clean out. Currently, on my system, this hasn't been a big issue, unless someone else is having a bigger issue with this then me. I wouldn't close this, yet, until you can solve this out entirely, but I can say it is something I do not think has to be done right away in 1.12, if you cannot figure out what the other issue with this is.

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

Also, if you do own more games on Steam, this will cause the cleaner to slow down more. I wonder what the effects of this would be to, let's say, someone who owns like over 100 games on Steam.

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

So today I tried running a preview with those entries I listed above for Steam because I haven't ran those in so long, and it took a large amount of time to preview them (~10 minutes). As I said before, this will take longer when more games is installed.

The problem is I only have 2 games installed (I had a 3rd one but I uninstalled that). Both these games aren't very big either.

It would be nice if you could get this fixed in BleachBit 2.0 or 2.2, as Steam piles up with a lot of junk quickly and ignoring these is very inefficient for BleachBit. Not to mention that if someone had 10+ games installed on steam and ran BleachBit for the first time, they might have a problem with this, too.

[Andrew Ziem (ahziem1)]

The recent alpha versions of BleachBit use Python 2.7 which has many changes and improves. Do you still experience the slowness?

[ROCKNROLLKID (slinger1410-deactivatedaccount)]

Yes. Tested with 1.15.420. Actually I have installed even more games to steam since last check, so it is even slower then before.

[Andrew Ziem (ahziem1)]

Moved to https://github.com/bleachbit/bleachbit/issues/325