12.04 Unity Video Lense uses up all disk IO when indexing

[psypher]

This issue occurs upon boot and when attempting to open the Unity video lens.

The process: python /usr/lib/unity-lens-video/unity-lens-video uses all available disk IO causing full system performance decrease and application hangs (grey outs). 98-99% IO while monitoring with IOTOP. This happens for about 5 minutes on boot without clicking on the lens and once that has completed it starts to happen again when you click on the lens. The lens does not even work during this long process.

This is most likely exasperated by the widely experienced kernel disk IO and scheduling bug which has been ongoing for some time now. Some improvements have been made but I'm not sure if they have all been imported into the latest stable Precise kernels.

I suggest that the video lens should not be installed by default as normal users will not understand why their PC is freezing for no reason.

Can anyone else confirm this issue or is it specific to my PC and amount of files.

Takes about 5-10 minutes for the system to become usable again. Then the Video lens once again provides search results.

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: unity-lens-video 0.3.3-0ubuntu1
ProcVersionSignature: Ubuntu 3.2.0-17.27-generic 3.2.6
Uname: Linux 3.2.0-17-generic x86_64
ApportVersion: 1.94.1-0ubuntu2
Architecture: amd64
Date: Fri Mar 16 14:54:02 2012
EcryptfsInUse: Yes
InstallationMedia: Ubuntu 12.04 LTS "Precise Pangolin" - Alpha amd64 (20111129.1)
PackageArchitecture: all
SourcePackage: unity-lens-video
UpgradeStatus: Upgraded to precise on 2012-03-02 (14 days ago)

[David Callé (davidc3)]

Hello, thanks for the report.
Could you answer these questions to know it this is a fixed bug (but not released yet) or a whole new one?

1) Do you still have your default ~/Videos folder?

2) If yes, how many files do you have in it?

3) Could you please test the video lens package in this PPA:
https://code.launchpad.net/~davidc3/+archive/dev

Thank you very much.

[psypher (psypher246)]

Yes still have default videos.

890 items, totalling 17,1 GB

Upgrading now, will play around and see if there is an improvement

Thanks

[psypher (psypher246)]

I have been using it over the weekend, and the problem actually seems worse. process now runs continuously as soon as it's activated. not at 98% disk IO but at least 50%.

May I ask what it's doing? Shouldn't it take a few seconds to just scan all the filenames and index? Looking at how much disk IO it consumes, seems like it's reading the entire video file, over and over.

[David Callé (davidc3)]

It's trying to generate thumbnails for movies without Nautilus generated thumbnails (and caches them in ~/.cache/unity-lens-video when it's done so it doesn't have to generate them several times). But the lens doesn't allow more than three thumbnailers running at the same time, so some disk IO the first time the video lens encounters a file without thumbnail is normal, but what happens to you is a bit extreme.

Could you please test the following changes?

In a terminal:
gksu gedit /usr/lib/unity-lens-video/unity-lens-video
And on line 91 of this file:
replace "Q_MAX = 3" by "Q_MAX = 0".
It will prevent the thumbnailer to start, Q_MAX being the size of the thumbnailer queue.
Then delete the ~/.cache/unity-lens-video folder and log out.

The lens shouldn't try to generate thumbnails anymore and only shows thumbnails for videos having Nautilus generated thumbnails. If the intensive disk IO is gone, it means we need to be smarter about how the thumbnailers are handled and probably avoid the thumbnailing on some videos.

If the intensive IO is still here, it means it's another bug.

[psypher (psypher246)]

Hi David,

I had to remove the PPA after I installed updates today. Broke unity completely, would not load it. I logged a separate bug about it as I thought it was just Unity. But after removing your PPA Unity started working again. I didn't get much chance test the changes you requested. Although before I updated today I did check once and there seemed to be a slight improvement.

Here is the details of the other bug: https://bugs.launchpad.net/unity/+bug/962075

Thanks

[David Callé (davidc3)]

Hi,
I doubt that the PPA was responsible for your Unity issues. The version currently in distro is, since yesterday, the same one as in the PPA.

There should be a lot of new upgraders next week, they will help tune the thumbnailer and find out when he has issues. Thanks for your testing and if you find the time to do it, please try the Q_MAX = "0" change.

[Jonathon Fernyhough (jfernyhough)]

Changing Q_MAX appears at the moment to have helped. I don't understand why the lens makes its own thumbnails; surely it should use the Nautilus ones by default so work is not duplicated?

I noticed when I cleared out its cache that it was also thumbnailing everything possible in places like the Chromium web cache (including images) which suggests video-lens is doing a lot of unnecessary work.

[David Callé (davidc3)]

Jonathon,

1) the lens looks if a Nautilus thumbnail exists before trying to generate one.
2) for the lens to trigger the thumbnailing process, the file needs to be 1) a video 2) in the Videos folder. It wasn't the lens working in the Chromium web cache.