Ftp servers with bogus APPE are not supported

[Heinrich Flugelhorn]

For more than one year, we've had a fully functionnal Bazaar
repository, centralized at a FTP server.

Suddenly, we all got the following error when we try to commit:

bzr: ERROR: Unrecognised container format: 'B246'

If I try to recreate another setup by scratch now, I get the same error:

$ bzr init-repo --verbose --no-trees ftp://<email address hidden>/repository2
Shared repository (format: 2a)
Location:
 shared repository: ftp://<email address hidden>/repository2/
$ bzr init ftp://<email address hidden>/repository2/site
Created a repository branch (format: 2a)
Using shared repository: ftp://<email address hidden>/repository2/
$ cd .bzr checkout ftp://<email address hidden>/repository2/branch branch
$ cd branch
$ bzr mkdir www
added www
$ bzr commit -m " "
Committing to: ftp://<email address hidden>/repository2/branch/
added www
bzr: ERROR: Unrecognised container format: 'B246'

The FTP is hosted in a mutualized hosting (at OVH.com), so it is
unlikely that something has changed in its configuration (or is it?). Nothing has been changed in our configurations.

[Martin Pool (mbp)]

On 7 February 2012 09:20, Fbe546 <email address hidden> wrote:
> Thanks a lot for the answer.
>
> We posted a bug report at https://bugs.launchpad.net/bzr/+bug/927929
> with the .bzr.log file.
>
> The pack files of the old repository did indeed begin with "Bazaar
> pack format 1 (introduced in 0.18)\n". There are no pack files inside
> the newly created repository (with the commands above), but I suppose
> it's normal.

In the bug you create a new repository and then try to commit to it.
In this case bzr is creating and then reading back a pack file. If
the data we wrote in to the pack is immediately missing this is seems
very likely a bug in the ftp server implementation.

--
Martin

[Heinrich Flugelhorn (fbe546)]

You mean that the problem would be on the side of the FTP server ? It
would be quite strange since it is a mutualized hosting at the largest
hosting company in Europe and everything had been working for a year.

Anyway, do you have any idea of how we could get a more precise idea
of where the problem lies ?

Looking at the FTP log below (found in .bzr.log), the problem would be
that wb3yq8rbpvyrz1zlhsbm.pack isn't read correctly at 4.107 ?

Again thanks for the help and Bazaar :)

3.363 FTP stat: /repository2/.bzr
3.485 FTP put: /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.733 FTP has check: wb3yq8rbpvyrz1zlhsbm.pack =>
/repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.765 FTP has: /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.795 FTP appe to /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.795 FTP appe (try 0) to
/repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.925 FTP has check: wb3yq8rbpvyrz1zlhsbm.pack =>
/repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.955 FTP has: /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.985 FTP appe to /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
3.985 FTP appe (try 0) to
/repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
4.107 FTP get: /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
4.229 FTP rm: /repository2/.bzr/repository/upload/wb3yq8rbpvyrz1zlhsbm.pack
4.265 FTP get: /repository2/site/.bzr/branch/lock/held/info
4.389 FTP rename: /repository2/site/.bzr/branch/lock/held =>
/repository2/site/.bzr/branch/lock/releasing.obv9us6ree5bjyztw5ue.tmp
4.453 FTP rm: /repository2/site/.bzr/branch/lock/releasing.obv9us6ree5bjyztw5ue.tmp/info
4.489 FTP rmd:
/repository2/site/.bzr/branch/lock/releasing.obv9us6ree5bjyztw5ue.tmp
4.521 Transferred: 0kB (0.0kB/s r:0kB w:0kB)
4.536 Traceback (most recent call last): (...)

[Robert Collins (lifeless)]

On Wed, Feb 8, 2012 at 12:49 AM, Fbe546 <email address hidden> wrote:
> You mean that the problem would be on the side of the FTP server ? It
> would be quite strange since it is a mutualized hosting at the largest
> hosting company in Europe and everything had been working for a year.

Unusual, yes. Unprecedented no. We had another user using FTP to a
hosting site that was build on Microsoft Windows with a cluster file
system that took 1-2 *seconds* to allow newly written content to be
read. (The details are in an old bug report, I don't remember the
number).

> Anyway, do you have any idea of how we could get a more precise idea
> of where the problem lies ?
>
> Looking at the FTP log below (found in .bzr.log), the problem would be
> that wb3yq8rbpvyrz1zlhsbm.pack isn't read correctly at 4.107 ?

Yes, exactly that spot. The file is written with APPE, then when it is
read-back it is not returning the expected content.

Cheers,
Rob

[Heinrich Flugelhorn (fbe546)]

Maybe someone familiar with the source code of Bazaar could answer
this question before I begin to dig into it:

How hard would it be to replace the use of APPE with a GET and a PUT ?

It would surely be less efficient, but at least it would work...

[Martin Pool (mbp)]

On 8 February 2012 06:56, Fbe546 <email address hidden> wrote:
> Maybe someone familiar with the source code of Bazaar could answer
> this question before I begin to dig into it:
>
> How hard would it be to replace the use of APPE with a GET and a PUT ?
>
> It would surely be less efficient, but at least it would work...

There is already support for doing this in transport/ftp/__init__.py,
the has_append attribute. If the server doesn't support APPE at all
we fall back to this. Apparently we have not yet encountered a server
that has an append but where it doesn't work properly. You could
fairly easily check the location config stack for a new variable
ftp.disable_append and then turn that on. It would be at least worth
trying.

If it's true, as it seems, that the problem is the server is not
read-after-write coherent with itself on a single file, I don't know
why PUT would be any more coherent than APPE....

--
Martin

[Heinrich Flugelhorn (fbe546)]

Yay, it works again if I set "self._has_append" to false in
transport/ftp/__init__.py!

So it means, as you said, that the servers implements APPE, but it
doesn't work correctly... We reported that to our host, let's see what
they will say...

Thanks for your help.

[Martin Pool (mbp)]

On 8 February 2012 23:00, Fbe546 <email address hidden> wrote:
> Yay, it works again if I set "self._has_append" to false in
> transport/ftp/__init__.py!
>
> So it means, as you said, that the servers implements APPE, but it
> doesn't work correctly... We reported that to our host, let's see what
> they will say...
>
> Thanks for your help.

OK, that's interesting. Would you like to try a patch to add a
configuration variable for avoiding append?

It may get pretty slow to read everything back as your repository gets larger.

When we write a file out in a single stream we could possibly avoid
APPE with some code changes...

--
Martin

[Heinrich Flugelhorn (fbe546)]

The host told use they "reported the issue", but I don't hope much.

I can try what you want. If I have some free time soon, I might try
too to understand where exactly the server bug with APPE lies. Maybe
all the data is returned, but with something bad in addition...

[Scott Aubrey (scott-aubrey-deactivatedaccount)]

Dear,

There is something really amazing I've found a couple of days ago, check it out http://hncg.com.cn/quarter.php?0302

Rushing, scott

[Scott Aubrey (scott-aubrey-deactivatedaccount)]

Hi friend!

I'm sure you'll like the stuff I've just found for you, here, take a look http://raynerecording.com/implication.php?3435

Sincerely yours, scott

[Scott Aubrey (scott-aubrey-deactivatedaccount)]

Yo!

This info may be very useful for you, you won't regret, just take a look at http://www.solheim.nl/intimate.php?f8f9

Yours, scott

[Scott Aubrey (scott-aubrey-deactivatedaccount)]

Hello friend,

I've been writing an article and I've come across that strange stuff, do you have any ideas what could it be? Just take a look http://hazazontamar.badgleybankruptcysolutions.com

Best wishes, scott

Sent from Mail for Windows 10

[Scott Aubrey (scott-aubrey-deactivatedaccount)]

Hello,

I've recently read some useful information, I guess it might be useful for you too, please read at http://pallab.andrewtam.us

Best regards, scott

From: 927929 [mailto:<email address hidden>]
Sent: Sunday, May 14, 2017 2:42 PM
To: <email address hidden>
Subject: I think so.

I'm certain that at the very, very least, the PSU was powerful enough for the new equipment at one point. My new mobo/CPU worked just fine for about 1.5 days before they abruptly stopped working. Also, I'm typing this from the old mobo/CPU combo right now, so I'm pretty confident that the PSU isn't the problem. I was just hoping that there would be some way to tell whether or not it's the CPU or the mobo that is broken. Looks like I may just have to flip and coin on returning one of them.

Sent from Mail for Windows 10