--image-file and --image-size do not work in l-m-c

[Deepti B. Kalakeri]

The lava-dispatcher is failing to deploy the images. Here is the trace of the error:

root@master:~# [rc=0]: <LAVA_DISPATCHER>2011-10-25 07:30:02 AM INFO: About to handle with the build
<LAVA_DISPATCHER>2011-10-25 07:30:02 AM INFO: Downloading the http://ci.linaro.org/kernel_hwpack/linux-arm-soc-for-next/linux-arm-soc-for-next_panda-omap2plus/hwpack_linaro-panda_20111025-0725_b144_armel_supported.tar.gz file using cache
<LAVA_DISPATCHER>2011-10-25 07:44:02 AM INFO: Downloading the http://snapshots.linaro.org/11.05-daily/linaro-nano/20111024/0/images/tar/nano-n-tar-20111024-0.tar.gz file using cache
<LAVA_DISPATCHER>2011-10-25 07:44:02 AM INFO: Executing the linaro-media-create command
<LAVA_DISPATCHER>2011-10-25 07:44:02 AM INFO: sudo flock /var/lock/lava-lmc.lck linaro-media-create --hwpack-force-yes --dev panda --image-file /linaro/images/tmp/tmpeSFdyh/lava.img --binary /linaro/images/tmp/tmpeSFdyh/nano-n-tar-20111024-0.tar.gz --hwpack /linaro/images/tmp/tmpeSFdyh/hwpack_linaro-panda_20111025-0725_b144_armel_supported.tar.gz --image-size 3G
None
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/lava_dispatcher/actions/deploy.py", line 66, in run
    use_cache)
  File "/usr/lib/pymodules/python2.6/lava_dispatcher/actions/deploy.py", line 176, in generate_tarballs
    raise RuntimeError("linaro-media-create failed: %s" % output)
RuntimeError: linaro-media-create failed: /usr/lib/pymodules/python2.6/argparse.py:1576: DeprecationWarning: The "version" argument to ArgumentParser is deprecated. Please use "add_argument(..., action='version', version="N", ...)" instead
  """instead""", DeprecationWarning)
usage: linaro-media-create [-h] [-v] (--mmc DEVICE | --image_file DEVICE)
                           --dev
                           {overo,vexpress,mx53loco,snowball_emmc,panda,ux500,igep,mx51evk,smdkv310,snowball_sd,beagle,efikamx,origen,efikasb}
                           [--rootfs {ext2,ext3,ext4,btrfs}]
                           [--rfs_label RFS_LABEL] [--boot_label BOOT_LABEL]
                           [--swap_file SWAP_FILE] [--live | --live-256m]
                           [--console CONSOLES] --hwpack HWPACKS
                           [--hwpack-sig HWPACKSIGS] [--hwpack-force-yes]
                           [--image_size IMAGE_SIZE] --binary BINARY
                           [--binary-sig BINARYSIG] [--no-rootfs]
                           [--no-bootfs] [--no-part] [--align-boot-part]
                           [--nocheck-mmc]
linaro-media-create: error: one of the arguments --mmc --image_file is required
Lava failed at action deploy_linaro_image with error: Deployment tarballs preparation failed
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/lava_dispatcher/__init__.py", line 79, in run
    action.run(**params)
  File "/usr/lib/pymodules/python2.6/lava_dispatcher/actions/deploy.py", line 70, in run
    raise CriticalError("Deployment tarballs preparation failed")
CriticalError: Deployment tarballs preparation failed

You can refer to the job http://validation.linaro.org/lava-server/scheduler/job/2918 which failed as an example failure.
I guess the lava-dispatcher is not completely adopting the latest l-m-c changes.

Thanks!!!
Deepti.

[Fathi Boudra (fboudra)]

Have we updated l-i-t to version 2011.10?

[Fathi Boudra (fboudra)]

zyga confirmed that we're using l-i-t 2011.10 on maverick.

[Spring Zhang (qzhang)]

--image_file is changed to --image-file in dispatcher, http://bazaar.launchpad.net/~qzhang/lava-dispatcher/fix-863091/revision/141

[Fathi Boudra (fboudra)]

On 25 October 2011 11:50, Spring Zhang <email address hidden> wrote:
> --image_file is changed to --image-file in dispatcher,
> http://bazaar.launchpad.net/~qzhang/lava-
> dispatcher/fix-863091/revision/141

James W. told us that --image_file has been kept for compatibility.
--image_file is working for me, using l-i-t 2011.10.

[Fathi Boudra (fboudra)]

I can't reproduce the issue inside a maverick chroot using l-i-t 2011.10:
$ sudo linaro-media-create --dev panda --image_file lava.img --binary nano-n-tar-20111024-0.tar.gz --hwpack hwpack_linaro-panda_20111025-0725_b144_armel_supported.tar.gz

[...]

Done creating Linaro image on lava.img

Note: I didn't passed --image-size 3G as it's the default image size.

[Alexander Sack (asac)]

if this is really hat the old arguments are not there anymore, its a serious lmc regression. It MUST keep the old/depreacted arguments forever.

[Fathi Boudra (fboudra)]

On 25 October 2011 13:50, Alexander Sack <email address hidden> wrote:
> if this is really hat the old arguments are not there anymore, its a
> serious lmc regression. It MUST keep the old/depreacted arguments
> forever.

It can't be that, I used the deprecated argument --image_file (underscore).

[Mattias Backman (mabac)]

On Tue, Oct 25, 2011 at 12:50 PM, Alexander Sack <email address hidden> wrote:
> if this is really hat the old arguments are not there anymore, its a
> serious lmc regression. It MUST keep the old/depreacted arguments
> forever.

It's the other way around:
> usage: linaro-media-create [-h] [-v] (--mmc DEVICE | --image_file DEVICE)

Either --mmc or --image_file is not passed at all, or the new
parameter --image-file is used with an old l-m-c.

Can we have the full command line that was used to produce this error?

[Mattias Backman (mabac)]

On Tue, Oct 25, 2011 at 1:50 PM, Mattias Backman
<email address hidden> wrote:
> On Tue, Oct 25, 2011 at 12:50 PM, Alexander Sack <email address hidden> wrote:
>> if this is really hat the old arguments are not there anymore, its a
>> serious lmc regression. It MUST keep the old/depreacted arguments
>> forever.
>
> It's the other way around:
>>    usage: linaro-media-create [-h] [-v] (--mmc DEVICE | --image_file DEVICE)

I see that l-m-c is recent enough to have the -v option so someone
could do l-m-c -v and provide the version to us?

[Paul Larson (pwlars)]

I'm working around this temporarily by just using --image_file. If you use --image-file though, it complains. I've tried this on twi different machines:
$ sudo linaro-media-create --hwpack-force-yes --dev panda --hwpack hwpack_linaro-lt-panda_20111019-0_armel_supported.tar.gz --binary nano-n-tar-20111019-0.tar.gz --image-file foo.img
usage: linaro-media-create [-h] [-v] (--mmc DEVICE | --image_file DEVICE)
                           --dev
                           {overo,vexpress,mx53loco,snowball_emmc,panda,ux500,igep,mx51evk,smdkv310,snowball_sd,beagle,efikamx,origen,efikasb}
                           [--rootfs {ext2,ext3,ext4,btrfs}]
                           [--rfs_label RFS_LABEL] [--boot_label BOOT_LABEL]
                           [--swap_file SWAP_FILE] [--live | --live-256m]
                           [--console CONSOLES] --hwpack HWPACKS
                           [--hwpack-sig HWPACKSIGS] [--hwpack-force-yes]
                           [--image_size IMAGE_SIZE] --binary BINARY
                           [--binary-sig BINARYSIG] [--no-rootfs]
                           [--no-bootfs] [--no-part] [--align-boot-part]
                           [--nocheck-mmc]
linaro-media-create: error: one of the arguments --mmc --image_file is required

[Fathi Boudra (fboudra)]

After digging into the issue, the root cause is l-i-t. We're using 2011.10 and --image-file is broken.

[Mattias Backman (mabac)]

I can't reproduce this using 2011.10, I've tried on natty with python 2,7 and lucid with python 2.6.

steludxu1184:~/junk/linaro-image-tools-2011.10> ./linaro-media-create -h
/usr/lib/pymodules/python2.6/argparse.py:1576: DeprecationWarning: The "version" argument to ArgumentParser is deprecated. Please use "add_argument(..., action='version', version="N", ...)" instead
  """instead""", DeprecationWarning)
usage: linaro-media-create [-h] [-v] (--mmc DEVICE | --image-file DEVICE)

I see that the -h output in Paul's print does not list --image-file but rather --image_file and that was the old behaviour.

[Dave Pigott (dpigott)]

When we deprecate options it breaks automated systems. Maybe we should try to take care in the future to support the old behaviour with a "deprecated" warning being issued, and not remove the support until the next release?

[Mattias Backman (mabac)]

On Wed, Oct 26, 2011 at 10:11 AM, Dave Pigott <email address hidden> wrote:
> When we deprecate options it breaks automated systems. Maybe we should
> try to take care in the future to support the old behaviour with a
> "deprecated" warning being issued, and not remove the support until the
> next release?

Actually the old options are not touched at all, they still work and
there's no plan for removing them. The only reason there's no
deprecation warning is that adding the aliases was trivial and adding
deprecation messages would take a little (admittedly very little)
effort. We can make sure to add warnings a couple of releases before
the old options are removed, if they ever will be.

[Fathi Boudra (fboudra)]

Package in the PPA doesn't use the 2011.10 tarball and doesn't contain the commits to support the new and the deprecated format. Fixed in linaro-image-tools - 2011.10+1-0ubuntu1~linaro1 package.

[Dave Pigott (dpigott)]

Ah. OK. Misunderstood. Of course, the output of l-m-c -h told you that LAVA was using the old one.

Sorry about that.

On 26 Oct 2011, at 09:31, Mattias Backman wrote:

> On Wed, Oct 26, 2011 at 10:11 AM, Dave Pigott <email address hidden> wrote:
>> When we deprecate options it breaks automated systems. Maybe we should
>> try to take care in the future to support the old behaviour with a
>> "deprecated" warning being issued, and not remove the support until the
>> next release?
>
> Actually the old options are not touched at all, they still work and
> there's no plan for removing them. The only reason there's no
> deprecation warning is that adding the aliases was trivial and adding
> deprecation messages would take a little (admittedly very little)
> effort. We can make sure to add warnings a couple of releases before
> the old options are removed, if they ever will be.
>
> --
> You received this bug notification because you are a member of Linaro
> Validation Team, which is subscribed to LAVA Dispatcher.
> https://bugs.launchpad.net/bugs/881292
>
> Title:
> --image-file and --image-size do not work in l-m-c
>
> Status in LAVA Dispatcher:
> Confirmed
> Status in Linaro Image Tools:
> Fix Released
>
> Bug description:
> The lava-dispatcher is failing to deploy the images. Here is the trace
> of the error:
>
> root@master:~# [rc=0]: <LAVA_DISPATCHER>2011-10-25 07:30:02 AM INFO: About to handle with the build
> <LAVA_DISPATCHER>2011-10-25 07:30:02 AM INFO: Downloading the http://ci.linaro.org/kernel_hwpack/linux-arm-soc-for-next/linux-arm-soc-for-next_panda-omap2plus/hwpack_linaro-panda_20111025-0725_b144_armel_supported.tar.gz file using cache
> <LAVA_DISPATCHER>2011-10-25 07:44:02 AM INFO: Downloading the http://snapshots.linaro.org/11.05-daily/linaro-nano/20111024/0/images/tar/nano-n-tar-20111024-0.tar.gz file using cache
> <LAVA_DISPATCHER>2011-10-25 07:44:02 AM INFO: Executing the linaro-media-create command
> <LAVA_DISPATCHER>2011-10-25 07:44:02 AM INFO: sudo flock /var/lock/lava-lmc.lck linaro-media-create --hwpack-force-yes --dev panda --image-file /linaro/images/tmp/tmpeSFdyh/lava.img --binary /linaro/images/tmp/tmpeSFdyh/nano-n-tar-20111024-0.tar.gz --hwpack /linaro/images/tmp/tmpeSFdyh/hwpack_linaro-panda_20111025-0725_b144_armel_supported.tar.gz --image-size 3G
> None
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.6/lava_dispatcher/actions/deploy.py", line 66, in run
> use_cache)
> File "/usr/lib/pymodules/python2.6/lava_dispatcher/actions/deploy.py", line 176, in generate_tarballs
> raise RuntimeError("linaro-media-create failed: %s" % output)
> RuntimeError: linaro-media-create failed: /usr/lib/pymodules/python2.6/argparse.py:1576: DeprecationWarning: The "version" argument to ArgumentParser is deprecated. Please use "add_argument(..., action='version', version="N", ...)" instead
> """instead""", DeprecationWarning)
> usage: linaro-media-create [-h] [-v] (--mmc DEVICE | --image_file DEVICE)
> --dev
> {overo,vexpress,mx53loco,snowball_emmc,panda,ux500,igep,mx51evk,smdkv310,snowball_sd,beagle,efikamx,origen,...

[Fathi Boudra (fboudra)]

Set to fix-released:
- validation updated to latest package
- work around reverted from deploy.py
- smoke tested