export doesn't handle files going missing from working tree

[Forest Bond]

Binary package hint: bzr-builddeb

debian/dirs need not exist for all packages, but bzr-builddeb will not build a package that does not have it.

[James Westby (james-w)]

On (06/12/07 21:20), Forest Bond wrote:
> Public bug reported:
>
> Binary package hint: bzr-builddeb
>
> debian/dirs need not exist for all packages, but bzr-builddeb will not
> build a package that does not have it.

Hi,

Thanks for the bug report. Unfortunately you did not provide enough
information about what symptoms you see. As far as I am aware there is
no requirement for debian/dirs, and if there was that would certainly
be a bug.

Which package are you seeing the problem with? What is the error that
you get?

Thanks,

James

--
  James Westby -- GPG Key ID: B577FE13 -- http://jameswestby.net/
  seccure key - (3+)k7|M*edCX/.A:n*N!>|&7U.L#9E)Tu)T0>AM - secp256r1/nistp256

[Forest Bond (forest-bond)]

Hi James,

On Mon, Dec 10, 2007 at 05:42:11PM -0000, James Westby wrote:
> On (06/12/07 21:20), Forest Bond wrote:
> > Public bug reported:
> >
> > Binary package hint: bzr-builddeb
> >
> > debian/dirs need not exist for all packages, but bzr-builddeb will not
> > build a package that does not have it.

> Thanks for the bug report. Unfortunately you did not provide enough
> information about what symptoms you see. As far as I am aware there is
> no requirement for debian/dirs, and if there was that would certainly
> be a bug.
>
> Which package are you seeing the problem with? What is the error that
> you get?

First, let me just say thanks for writing bzr-builddeb! Between that and CDBS,
packaging got a lot more pleasant for me!

The following should provide you with enough information to see the problem
(which appears to only occur when using -w):

--------------------------------------------------------------------------------
[~/work/debs/pytagsfs/pytagsfs]
13:10 fab@storm$ ls
debian
[~/work/debs/pytagsfs/pytagsfs]
13:10 fab@storm$ bzr stat
[~/work/debs/pytagsfs/pytagsfs]
13:10 fab@storm$ cat debian/control
Source: pytagsfs
Section: utils
Priority: optional
Maintainer: Forest Bond <email address hidden>
Build-Depends: debhelper (>= 5.0.37.2), python, python-support, cdbs, xsltproc, docbook-xsl
Standards-Version: 3.7.2
XS-Upstream-Vcs-Bzr: http://bazaar.launchpad.net/~forest-alittletooquiet/pytagsfs/dev
XS-Vcs-Bzr: http://bazaar.launchpad.net/~forest-alittletooquiet/pytagsfs/ubuntu

Package: pytagsfs
Architecture: all
Depends: ${python:Depends}, python-fuse, python-pyinotify, python-mutagen, python-sclapp
Provides: ${python:Provides}
Description: pytagsfs filesystem mapping media files to an arbitrary directory structure
 pytagsfs is a FUSE filesystem that was designed to present multiple views
 of tagged media files. For instance, a directory tree containing audio
 files could be mapped to a new directory structure organizing those same
 files by album, genre, release date, etc.
 .
 Homepage: http://www.alittletooquiet.net/software/pytagsfs/
[~/work/debs/pytagsfs/pytagsfs]
13:10 fab@storm$ bzr bd
Running in merge mode
Building branch from revision <email address hidden>
Preparing the build area: ../build-area
Exporting to ../build-area/pytagsfs-0.1.0 in merge mode
Looking for ../tarballs/pytagsfs_0.1.0.orig.tar.gz to use as upstream source
Exporting debian/ part to ../build-area/pytagsfs-0.1.0
Building the package in ../build-area/pytagsfs-0.1.0, using dpkg-buildpackage -uc -us -rfakeroot
dpkg-buildpackage: source package is pytagsfs
dpkg-buildpackage: source version is 0.1.0-0ubuntu4
dpkg-buildpackage: source changed by Forest Bond <email address hidden>
dpkg-buildpackage: host architecture i386
dpkg-buildpackage: source version without epoch 0.1.0-0ubuntu4

[...]

dpkg-deb: building package `pytagsfs' in `../pytagsfs_0.1.0-0ubuntu4_all.deb'.
 dpkg-genchanges
dpkg-genchanges: not including original source code in upload
dpkg-buildpackage: binary and diff upload (original source NOT included)
Cleaning build dir: ../build-area/pytagsfs-0.1.0
[~/work/debs/pytagsfs/pytagsfs]
13:11 fab@storm...

[Forest Bond (forest-bond)]

Hi,

On Mon, Dec 10, 2007 at 06:15:16PM -0000, Forest Bond wrote:
> On Mon, Dec 10, 2007 at 05:42:11PM -0000, James Westby wrote:
> > On (06/12/07 21:20), Forest Bond wrote:
> > > Public bug reported:
> > >
> > > Binary package hint: bzr-builddeb
> > >
> > > debian/dirs need not exist for all packages, but bzr-builddeb will not
> > > build a package that does not have it.

Wait a minute ...

This might be a bug in my version of bzr:

13:30 fab@storm$ bzr diff
=== removed file 'debian/dirs'
bzr: ERROR: No such file: u'/home/fab/work/debs/pytagsfs/pytagsfs/debian/dirs'

-Forest
--
Forest Bond
http://www.alittletooquiet.net

[Forest Bond (forest-bond)]

Actually a bug in bzr 1.0.0rc1, fixed in rc2.

[James Westby (james-w)]

On (10/12/07 18:57), Forest Bond wrote:
> Actually a bug in bzr 1.0.0rc1, fixed in rc2.
>

Thanks for tracking this down, and I am glad that it is fixed for you.

Thanks,

James

--
  James Westby -- GPG Key ID: B577FE13 -- http://jameswestby.net/
  seccure key - (3+)k7|M*edCX/.A:n*N!>|&7U.L#9E)Tu)T0>AM - secp256r1/nistp256

[Forest Bond (forest-bond)]

Now I'm on bzr 1.3.1, with the following plugins:

builddeb 0.92.0dev0
bzrtools 1.3.0
launchpad
rebase 0.3.0
svn 0.4.9

This is the same issue (it's back). I added debian/dirs, never committed, rm'd it, and now bzr bd -w is looking for it:

$ bzr bd -w
Running in native mode
Building using working tree
Preparing the build area: ../build-area
Exporting to ../build-area/ls-admin-utils-0.0.0dev01
bzr: ERROR: [Errno 2] No such file or directory: '/home/forest.bond/work/software/ls-admin-utils/ls-admin-utils/debian/dirs'

Of course, bzr rm complains that it doesn't exist, but executing that fixes the problem:

$ bzr rm debian/dirs
debian/dirs does not exist.
$ bzr bd -w
Running in native mode
Building using working tree
Preparing the build area: ../build-area
Exporting to ../build-area/ls-admin-utils-0.0.0dev01
Building the package in ../build-area/ls-admin-utils-0.0.0dev01, using dpkg-buildpackage -uc -us -rfakeroot
...

[Forest Bond (forest-bond)]

Re-opening, or so...

[James Westby (james-w)]

Hi,

I saw this the other day, it's in no way specific to debian/dirs.

The problem is when a versioned file goes missing from the working
tree. I'm not sure exactly what is at fault here, but the problem
is the the working tree still reports it as present, but then refuses
to provide the contents of the file. It's possibly a bug in bzrlib,
but I want to do some more investigation first.

I think the way that I worked around this problem was

  bzr commit -m 'dummy' && bzr uncommit

Thanks,

James

[John A Meinel (jameinel)]

Did you try something like 'bzr status' to see if it notices the file is removed. Or maybe explicitly informing it via "bzr rm foo" ?

[Forest Bond (forest-bond)]

Hi,

On Mon, Jun 02, 2008 at 02:14:47PM -0000, James Westby wrote:
> I saw this the other day, it's in no way specific to debian/dirs.

That's correct.

> The problem is when a versioned file goes missing from the working
> tree. I'm not sure exactly what is at fault here, but the problem
> is the the working tree still reports it as present, but then refuses
> to provide the contents of the file. It's possibly a bug in bzrlib,
> but I want to do some more investigation first.
>
> I think the way that I worked around this problem was
>
> bzr commit -m 'dummy' && bzr uncommit

`bzr rm foo' fixes the situation. This has to do with bzr's noticing that the
file was removed, and considering that a removal from the working tree, rather
than a missing file.

Thanks,
Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org

[James Westby (james-w)]

On Mon, 2008-06-02 at 14:44 +0000, John A Meinel wrote:
> Did you try something like 'bzr status' to see if it notices the file is
> removed. Or maybe explicitly informing it via "bzr rm foo" ?
>

Hi,

I've just looked in to this a little more.

The problem is running "rm foo" but not "bzr rm foo". "bzr st" then
shows the file as removed, but the WorkingTree inventory still has
the file.

The current failure is calling _read_tree_state() with the path, which
is done from inside builddeb. I can fix this to check
tree.has_filename() first, which does an lstat. However, the code
fails then inside the export, as that assumes that if the tree lists
the file it should be present (as it only works on revision trees in
bzr).

So, I have a few questions:

  1. Should the WorkingTree list the file if it has been removed?
  2. Should I change the export code to not assume a file is present?
  3. Am I missing something that would make this just work?

Thanks,

James

[Robert Collins (lifeless)]

On Fri, 2008-06-13 at 10:22 +0000, James Westby wrote:
> On Mon, 2008-06-02 at 14:44 +0000, John A Meinel wrote:
> > Did you try something like 'bzr status' to see if it notices the file is
> > removed. Or maybe explicitly informing it via "bzr rm foo" ?
> >
>
> Hi,
>
> I've just looked in to this a little more.
>
> The problem is running "rm foo" but not "bzr rm foo". "bzr st" then
> shows the file as removed, but the WorkingTree inventory still has
> the file.
>
> The current failure is calling _read_tree_state() with the path, which
> is done from inside builddeb. I can fix this to check
> tree.has_filename() first, which does an lstat. However, the code
> fails then inside the export, as that assumes that if the tree lists
> the file it should be present (as it only works on revision trees in
> bzr).
>
> So, I have a few questions:
>
> 1. Should the WorkingTree list the file if it has been removed?

Yes.

> 2. Should I change the export code to not assume a file is present?

exporting a working tree should handle a missing file like commit does -
that is to elide it from the output.

-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

[James Westby (james-w)]

This does need partially fixing in bzr, so I'm re-opening that task.

Thanks,

James

[James Westby (james-w)]

Hi,

I should try and come up with a fix for this sooner
rather than later.

Thanks,

James

[James Westby (james-w)]

bzr patch sent to the mailing list.

Thanks,

James

[James Westby (james-w)]

Marking as invalid, as this is now fixed in bzr, and no changes need to be made
in bzr-builddeb to use the fix.

Thanks,

James

[Robert Collins (lifeless)]

On Thu, 2009-02-26 at 19:34 +0000, James Westby wrote:
> Marking as invalid, as this is now fixed in bzr, and no changes need to be made
> in bzr-builddeb to use the fix.

You could consider a versioned dependency to ensure the fix is
available :)

-Rob