Poor error text when rejecting an unknown source package format

[Glen Ditchfield]

A short time before 12 Apr 2010 02:05:23 +0000 I tried to upload a package that contained an orig.tar.bz2 file instead of an orig.tar.gz. If I understand bug #225151, that is now allowed. The e-mail I got has a rather terse error message:

====================
Rejected:
Further error processing not possible because of a critical previous error.

Format: 1.7
Date: Mon, 05 Apr 2010 22:32:14 -0500
Source: alephone
Binary: alephone
Architecture: source
Version: 20100218-0~ppa1
Distribution: hardy
Urgency: low
Maintainer: Glen Ditchfield <email address hidden>
Changed-By: Glen Ditchfield <email address hidden>
Description:
alephone - game engine based on Marathon 2
Changes:
alephone (20100218-0~ppa1) hardy; urgency=low

  * New upstream release
Files:
525eb7abe972c67a49df3d798b731edc 912 games optional alephone_20100218-0~ppa1.dsc
f23530cb278cd0066f15630db9527dea 3180567 games optional alephone_20100218.orig.tar.bz2
ba32c332b22d97c18f9c5b3baadaf058 9800 games optional alephone_20100218-0~ppa1.diff.gz
=============

I recompressed the orig.tar as a .gz and uploaded again at 03:10:26, and that upload was accepted.

Whatever caused the rejection, I'd like to see some error message before the "further error processing" line.

[Michael Nelson (michael.nelson)]

I'm not sure that this is related to the bz2 or just our ftp upload process needing a boot. Either way the error message should have been more descriptive. Checking the logs shows the following (lots) around that time:

{{{
2010-04-12 02:04:32 DEBUG Processing session complete in /tmp/tmpeH-uqE-poppy
2010-04-12 02:04:32 ERROR Exception during client done hook
Traceback (most recent call last):
  File "/srv/launchpad.net/codelines/ppa-rev-9186/lib/lp/poppy/server.py", line 140, in clientFinished
    self.client_done_hook(channel.fsroot, clienthost, clientport)
  File "/srv/launchpad.net/codelines/ppa-rev-9186/lib/lp/poppy/hooks.py", line 55, in client_done_hook
    shutil.rmtree(fsroot)
  File "/usr/lib/python2.5/shutil.py", line 161, in rmtree
    onerror(os.listdir, path, sys.exc_info())
  File "/usr/lib/python2.5/shutil.py", line 159, in rmtree
    names = os.listdir(path)
OSError: [Errno 2] No such file or directory: '/tmp/tmpeH-uqE-poppy'
}}}

which would indicate bug 414482.

[William Grant (wgrant)]

The bzip2 rejection code definitely sends a usable error message -- I tested it when I wrote it, and the code is still fine. Michael, were you able to find the process-upload.py log for this upload?

[Michael Bienia (geser)]

Does LP accept a v1 source package with .orig.tar.bz2? I assumed .orig.tar.bz2 is only to be used with v3 source packages?

[William Grant (wgrant)]

On Mon, 2010-04-12 at 08:33 +0000, Michael Bienia wrote:
> Does LP accept a v1 source package with .orig.tar.bz2? I assumed
> .orig.tar.bz2 is only to be used with v3 source packages?

Correct. The error message is "package_version.dsc: is format 1.0 but
uses bzip2 compression."

[Michael Nelson (michael.nelson)]

On Mon, Apr 12, 2010 at 10:19 AM, William Grant <email address hidden> wrote:
> The bzip2 rejection code definitely sends a usable error message -- I
> tested it when I wrote it, and the code is still fine. Michael, were you
> able to find the process-upload.py log for this upload?

I did, but there was no extra info there (see below).

AFAICS, we also reject with "Further error processing not possible
because of a critical previous error." (without further explanation)
if DSCFile.format is None (that is, it is an *unsupported* format, see
lp.archiveuploader.dscfile.DSCFile.__init__()).

Is it possible Glen that you used an unsupported format for your first
upload? If so, we can update this bug to instead reflect the need for
a better rejection (and log) message there?
-Michael

{{{
2010-04-12 02:05:22 DEBUG Initialising connection.
2010-04-12 02:05:22 DEBUG Beginning processing
2010-04-12 02:05:22 DEBUG Could not fix the lockfile permission:
[Errno 1] Operation not permitted:
'/srv/launchpad.net/ppa-queue/incoming/.lock'
2010-04-12 02:05:22 DEBUG Checked in
/srv/launchpad.net/ppa-queue/incoming, found
['upload-20100412-030520-004002']
2010-04-12 02:05:22 DEBUG Considering upload upload-20100412-030520-004002
2010-04-12 02:05:22 DEBUG Considering changefile
~gjditchfield/ubuntu/alephone_20100218-0~ppa1_source.changes
2010-04-12 02:05:22 DEBUG Finding fresh policy
2010-04-12 02:05:22 DEBUG Verifying signature on
alephone_20100218-0~ppa1_source.changes
2010-04-12 02:05:23 INFO Processing upload
alephone_20100218-0~ppa1_source.changes
2010-04-12 02:05:23 DEBUG Beginning processing.
2010-04-12 02:05:23 DEBUG Performing DSC verification.
2010-04-12 02:05:23 DEBUG Building recipients list.
2010-04-12 02:05:23 DEBUG Adding recipient: 'Glen Ditchfield <...>'
2010-04-12 02:05:23 DEBUG Sending rejection email.
2010-04-12 02:05:23 DEBUG Sent a mail:
2010-04-12 02:05:23 DEBUG Subject:
alephone_20100218-0~ppa1_source.changes rejected
2010-04-12 02:05:23 DEBUG Recipients: Glen Ditchfield <...>
2010-04-12 02:05:23 DEBUG Body:
2010-04-12 02:05:23 DEBUG Rejected:
2010-04-12 02:05:23 DEBUG Further error processing not possible
because of a critical previous error.
}}}

[Glen Ditchfield (gjditchfield)]

Michael Nelson: I destroyed the original .bz2 when recompressing it, but it was a renamed copy of the file available at http://sourceforge.net/projects/marathon/files/Aleph One/2010-02-18/AlephOne-20100218.tar.bz2/download, and bunzip2 had no complaints about it. The .dsc is gone, too, but it was generated by debuild on a Hardy AMD64 host.

Feel free to change the bug. A better rejection message would be good enough for me.

[Michael Nelson (michael.nelson)]

On Tue, Apr 13, 2010 at 12:49 AM, Glen Ditchfield <email address hidden> wrote:
> Michael Nelson: I destroyed the original .bz2 when recompressing it, but
> it was a renamed copy of the file available at
> http://sourceforge.net/projects/marathon/files/Aleph
> One/2010-02-18/AlephOne-20100218.tar.bz2/download, and bunzip2 had no
> complaints about it.  The .dsc is gone, too, but it was generated by
> debuild on a Hardy AMD64 host.

Hi Glen,

It's not the format of the zip file, but the debian format of the DSC
file. In the error below it seems that the DSC Format was set as 1.7,
and AFAICS that exception is raised in the code unless the format is
"1.0", or "3.0 (quilt)" or "3.0 (native)". If that is the case, then a
better error message would be ideal.

>
> Feel free to change the bug.  A better rejection message would be good
> enough for me.
>
> --
> upload of package with orig.tar.bz2 was rejected
> https://bugs.launchpad.net/bugs/561161
> You received this bug notification because you are subscribed to Soyuz.
>
> Status in Soyuz - The Launchpad Package Manager: Incomplete
>
> Bug description:
> A short time before 12 Apr 2010 02:05:23 +0000 I tried to upload a package that contained an orig.tar.bz2 file instead of an orig.tar.gz.  If I understand bug #225151, that is now allowed.  The e-mail I got has a rather terse error message:
>
> ====================
> Rejected:
> Further error processing not possible because of a critical previous error.
>
> Format: 1.7
> Date: Mon, 05 Apr 2010 22:32:14 -0500
> Source: alephone
> Binary: alephone
> Architecture: source
> Version: 20100218-0~ppa1
> Distribution: hardy
> Urgency: low
> Maintainer: Glen Ditchfield <email address hidden>
> Changed-By: Glen Ditchfield <email address hidden>
> Description:
> alephone   - game engine based on Marathon 2
> Changes:
> alephone (20100218-0~ppa1) hardy; urgency=low
>
>  * New upstream release
> Files:
> 525eb7abe972c67a49df3d798b731edc 912 games optional alephone_20100218-0~ppa1.dsc
> f23530cb278cd0066f15630db9527dea 3180567 games optional alephone_20100218.orig.tar.bz2
> ba32c332b22d97c18f9c5b3baadaf058 9800 games optional alephone_20100218-0~ppa1.diff.gz
> =============
>
> I recompressed the orig.tar as a .gz and uploaded again at 03:10:26, and that upload was accepted.
>
> Whatever caused the rejection, I'd like to see some error message before the "further error processing" line.
>
>
>