Comment 4 for bug 722168

Answers to your questions follow:

On 02/21/2011 12:08 AM, Phillip Lougher wrote:
> "As far as mksquashfs is concerned, it is being invoked in the usual way
> as an independent subprocess."
>
> What is the exact command line passed to Mksquashfs? (from looking at
> the Transparent Archivist tool's source code it doesn't seem to be
> passing any non-standard options).
Right. No options were being passed during these tests, just the source
directory and the target filepath. Then I tried -no-sparse which is, as
you noted, irrelevant. Then I tried -noappend, which may or may not
have solved the immediate problem.
> "Sometimes -- not very often, maybe once in 30-60 invocations --
> mksquashfs simply hangs."
>
> Is this 30-60 invocations of Mksquashfs (by the Transparent Archivist
> tool) or 30-60 invocations of the Transparent Archivist tool (which is
> expected to invoke Mksquashfs itself multiple times)?
Invocations of mksquashfs by TA. TA does successive approximations
until it's satisfied that it has put as much into a given squashfs
filesystem as will fit on a DVD or CD or whatever.
> "Having made some of the filesystem it's supposed to be making, It stops
> making the filesystem for no apparent reason. It continues to use a wee
> bit of processing power after that -- very occasionally rising to the
> top of the "top" display -- but it makes no observable progress on the
> filesystem that it's supposed to be making. Once I left it running for 4
> days; no progress, and no resumption of work occurred during all that
> time."
>
> With difficult to track down bugs (as this will be if it is a threading
> issue)

(ain't that the truth!)

> , even the smallest amount of clues can be useful. The fact that
> Mksquashfs when hung occasionally rises to the top of the top display
> suggests it is doing something (even though it is making no progress).
> This in turns suggests the hang is a "livelock" issue rather than a
> "deadlock" issue (in which case Mksquashfs will consume no CPU). When
> Mksquashfs rises to the top of the top display how much CPU is it using?
> and how often does it rise to the top of the top display?
Not much CPU, but enough to be noticed on the top display maybe once a
minute when the computer is otherwise idle. I don't know the % figure;
I watched it flash by a couple of times and wasn't impressed by it.
Maybe 5% or less for one top iteration, among many others in which it
was effectively 0%.
> As you've proved to be comfortable with modifying the Transparent
> Archivist tool's source code to alter the Mksquashfs invocation
> parameters, it would to useful to add the following options
>
> -no-progress -info
>
> This addresses two issues:
>
> First it gets rid of the progress bar (with the twirling progress
> indicator), which as this is handled by a separate thread could be
> completely responsible for Mksquashfs occasionally appearing in the top
> of top - this is unlikely unless your system is doing absolutely nothing
> else, but -no-progress eliminates that potential red herring.
Ah. Good idea. Thanks!
> Second it makes Mksquashfs display files as they are processed, in
> particular it will show the files processed immediately before
> Mksquashfs hangs. If over a number of hangs, the files processed before
> the hang are identical it will strongly suggest there is something about
> these files which are causing Mksquashfs to hang. This information will
> be extremely useful.
OK. I'll do this and send you a report. (Warning: each run takes over
24 hours, so don't expect immediate feedback.)