Execute parallel l-m-c actions where possible

[Mattias Backman]

Hi,

This was brought up in a session at the last Connect. The request is to perform actions in parallel in linaro-media-create to shorten execution time.

Thanks,

Mattias

[Guilherme Salgado (salgado)]

Hmm, my gut feeling is that the things we can do in parallel are fast
enough to not make a significant difference if we actually do run them
in parallel, but I'd like to be wrong there.

Either way, before investing any time on actually running things in
parallel we should probably find out what's the percentage of time spent
on each operation.

[Michael Hudson-Doyle (mwhudson)]

On Mon, 21 Nov 2011 13:44:43 -0000, Guilherme Salgado <email address hidden> wrote:
> Hmm, my gut feeling is that the things we can do in parallel are fast
> enough to not make a significant difference if we actually do run them
> in parallel, but I'd like to be wrong there.
>
> Either way, before investing any time on actually running things in
> parallel we should probably find out what's the percentage of time spent
> on each operation.

I remember Matt Waddell did some work where formatting and copying the
rootfs to the sd card and installing hwpacks in a chroot went in
parallel, with an rsync at the end to catch up -- it sounds like that
would still lead to a fair speed up in some use cases.

Cheers,
mwh

[Matt Waddel (mwaddel)]

mwh - do you mean the --no-rootfs/--no-bootfs options in l-m-c?

The vexpress had several system limitations, so I had to deploy the bootloader/kernel to an SD card and the rootfs was deloyed to a USB thumb drive. I implemented the --no-rootfs and --no-bootfs options so l-m-c would only deploy the part specified. Those options are still available, but I haven't tried them for a long time. It is an interesting thought to run them in parallel.

[Michael Hudson-Doyle (mwhudson)]

On Tue, 22 Nov 2011 00:20:46 -0000, Matt Waddel <email address hidden> wrote:
> mwh - do you mean the --no-rootfs/--no-bootfs options in l-m-c?

I don't remember :-) Maybe it wasn't you...

> The vexpress had several system limitations, so I had to deploy the
> bootloader/kernel to an SD card and the rootfs was deloyed to a USB
> thumb drive. I implemented the --no-rootfs and --no-bootfs options so
> l-m-c would only deploy the part specified. Those options are still
> available, but I haven't tried them for a long time. It is an
> interesting thought to run them in parallel.

Ah that does sound interesting, but it wasn't what I had in mind...

Cheers,
mwh

[Ricardo Salveti (rsalveti)]

On Mon, Nov 21, 2011 at 11:44 AM, Guilherme Salgado
<email address hidden> wrote:
> Hmm, my gut feeling is that the things we can do in parallel are fast
> enough to not make a significant difference if we actually do run them
> in parallel, but I'd like to be wrong there.

At least at my host system it takes a good amount of time to format
and create the ext4 filesystem at my sd cards. And that's directly
related with the size of the SD card that you're using, so with 8/16gb
card it can save a good amount of time.

I believe there are also some other areas to put in parallel, but
would need to look into more details. I know Tom Gall did some
optimizations in the past, maybe he knows how faster it was after
applying his changes.

[Milo Casagrande (milo)]

Due to the age of this issue, we are acknowledging that it will likely not be fixed, is no longer applicable, or was already fixed by an indirect change. If this issue is still important, please add details and reopen it.