FTP upload causes squid hang

Bug #1423498 reported by Stanislav German-Evtushenko on 2015-02-19
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Squid
Unknown
Unknown
squid3 (Ubuntu)
Undecided
Unassigned
Trusty
Medium
Andreas Hasenack
Utopic
Undecided
Unassigned
Vivid
Undecided
Unassigned

Bug Description

[Impact]
Uploading an empty file to an ftp server through the squid3 proxy hits an assertion error and the proxy crashes. It recovers after a few seconds.

The patch is applied upstream and basically adds to FTP the same checks it already has for HTTP for this type of empty content.

[Test Case]
* deploy trusty on an lxd
* install packages:
$ sudo apt install squid3 proftpd-basic curl

* give the ubuntu user a password:
$ echo -e "ubuntu\nubuntu" | sudo passwd ubuntu

* create an empty file:
$ truncate -s 0 /tmp/zero

* try to upload it to the ubuntu user's home directory:
$ ftp_proxy=http://localhost:3128/ curl --upload-file /tmp/zero ftp://ubuntu:ubuntu@localhost/

On affected versions, that will return an error from curl:
curl: (52) Empty reply from server

The proxy cache.log file will show a crash and later restart on its own:

==> /var/log/squid3/cache.log <==
2017/09/28 18:01:47| assertion failed: Server.cc:244: "r->body_pipe != NULL"

[Regression Potential]
The patch is focused, applied upstream for quite some time (since Dec 2013) and still present in trunk. I don't have other comments about the patch.

[Other Info]
I couldn't find a debian bug about this, and also didn't file one because debian no longer ships this old version of squid according to rmadison.

--------------------------------------------

Hello,

Ubuntu 14.04 LTS with recent updates.
squid3 version 3.3.8-1ubuntu6.2

If we try to upload file by FTP squid hangs and not responding anymore. Logs show the following:
2015/02/19 10:52:19| assertion failed: Server.cc:244: "r->body_pipe != NULL"

I have found that this bug was resolved in the recent versions but not yet applied for squid3 in Ubuntu repository:
http://bugs.squid-cache.org/show_bug.cgi?id=3498
http://squid-web-proxy-cache.1019090.n4.nabble.com/assertion-failed-Server-cc-245-quot-r-gt-body-pipe-NULL-quot-td4663925.html

Could you apply the mentioned patch?

Best regards,
Stanislav

Related branches

no longer affects: squid (Ubuntu)
Robie Basak (racb) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better.

Is this bug fixed in upstream release 3.4.8? If so, then I will be fixing the development release of Ubuntu shortly.

If someone can prepare backports, please follow the steps at https://wiki.ubuntu.com/StableReleaseUpdates#Procedure to have Trusty, Utopic and Vivid updated.

All the steps documented there need to be followed. In particular, I'm concerned that we have a working test case that can be followed by someone not familiar with the package for SRU verification purposes.

I my opinion this change is simple (but critical, any user can break the service) and the patch should be applied for squid in the main repository of Ubuntu LTS.

Robie Basak (racb) wrote :

That's fine, but we need to take care to avoid regressing existing users not affected by this bug, and we need a test case to exercise the bug to verify that it is fixed before issuing the update. Please see my link above if you can assist with this.

Kick In (kick-d) on 2015-08-04
Changed in squid3 (Ubuntu):
importance: Undecided → Critical
importance: Critical → Undecided
assignee: nobody → Kick In (kick-d)
Tiago Stürmer Daitx (tdaitx) wrote :

Thank you for reporting this bug to Ubuntu. Utopic reached EOL on July 23, 2015.
See this document for currently supported Ubuntu releases: https://wiki.ubuntu.com/Releases

Changed in squid3 (Ubuntu):
status: New → Confirmed
Changed in squid3 (Ubuntu Trusty):
status: New → Confirmed
Changed in squid3 (Ubuntu Vivid):
status: New → Confirmed
Changed in squid3 (Ubuntu Utopic):
status: New → Invalid
Robie Basak (racb) on 2016-05-24
Changed in squid3 (Ubuntu):
assignee: Kick In (kick-d) → Nish Aravamudan (nacc)
Nish Aravamudan (nacc) wrote :

Vivid has also reached EOL, but I have provided a debdiff for Precise. Please test the package (once built) at the PPA: https://launchpad.net/~nacc/+archive/ubuntu/lp1423498.

Thanks,
Nish

Changed in squid3 (Ubuntu Vivid):
status: Confirmed → Invalid
Nish Aravamudan (nacc) wrote :

@Stanislav, if you could provide a testcase still, that would be ideal.

Changed in squid3 (Ubuntu Trusty):
status: Confirmed → In Progress
assignee: nobody → Nish Aravamudan (nacc)
status: In Progress → Confirmed
status: Confirmed → In Progress
Nish Aravamudan (nacc) wrote :

Err, sorry, wires crossed on bugs, I'll update my debdiff for Trusty.

@Robie, do we also want to fix in Wily, then?

Robie Basak (racb) wrote :

@Nish

If applicable and someone is affected then yes please. If nobody is reported as affected then I'd say no point though (so speak up now!) so maybe not? If we don't think anyone will benefit from an SRU then I'd say no point. We'll probably stop doing lower priority SRUs for Wily soon anyway as it's EOL in July.

Nish Aravamudan (nacc) wrote :
Nish Aravamudan (nacc) wrote :

I have updated the debdiff to the Trusty version and submitted it to the same PPA. @Stanislav, if you could test squid3 version 3.3.8-1ubuntu6.8 from the same, I'd appreciate it.

Nish Aravamudan (nacc) wrote :

Yakkety and Xenial are up to 3.5.12-1ubuntu8 / 3.5.12-1ubuntu7 respectively.

@Stanislav, I've unassigned myself while we wait on testing results, but will watch the bug for updates.

Changed in squid3 (Ubuntu):
assignee: Nish Aravamudan (nacc) → nobody
Changed in squid3 (Ubuntu Trusty):
assignee: Nish Aravamudan (nacc) → nobody
Changed in squid3 (Ubuntu):
status: Confirmed → Fix Released

I'm so sorry for not being here for a while. Now I have time again to work on this a little bit.

I've tested 3.3.8-1ubuntu6.8 and it is still crashes.

Here is how to test:

sudo apt-get install squid3 lftp
echo test > /tmp/file1
touch /tmp/file2
lftp -e "set ftp:proxy http://localhost:3128; set net:max-retries 1; open $USER@localhost; put /tmp/file1; exit"

sudo cat /var/log/squid3/cache.log

Oops, once again:

sudo apt-get install squid3 lftp
echo test > /tmp/file1
touch /tmp/file2
lftp -e "set ftp:proxy http://localhost:3128; set net:max-retries 1; open $USER@localhost; put /tmp/file1; exit"
lftp -e "set ftp:proxy http://localhost:3128; set net:max-retries 1; open $USER@localhost; put /tmp/file2; exit"
sudo cat /var/log/squid3/cache.log

And once again (missed proftpd):

sudo apt-get install squid3 lftp proftpd-basic
echo test > /tmp/file1
touch /tmp/file2
lftp -e "set ftp:proxy http://localhost:3128; set net:max-retries 1; open $USER@localhost; put /tmp/file1; exit"
lftp -e "set ftp:proxy http://localhost:3128; set net:max-retries 1; open $USER@localhost; put /tmp/file2; exit"
sudo cat /var/log/squid3/cache.log

Andreas Hasenack (ahasenack) wrote :

Confirmed on trusty, happens with zero sized files. Checking the rest

Changed in squid3 (Ubuntu Trusty):
status: In Progress → Triaged
importance: Undecided → Medium
Andreas Hasenack (ahasenack) wrote :

Indeed it only happens with trusty at this point.

Updated test case:

sudo apt install squid3 proftpd-basic curl
echo -e "ubuntu\nubuntu" | sudo passwd ubuntu
touch /tmp/zero
ftp_proxy=http://localhost:3128/ curl --upload-file /tmp/zero ftp://ubuntu:ubuntu@localhost/

On affected versions, that will return an error from curl:
curl: (52) Empty reply from server

Andreas Hasenack (ahasenack) wrote :

Triage note:

https://github.com/squid-cache/squid/commit/ccc927f9204159637bc77820fbb3f36521a54c86

commit ccc927f9204159637bc77820fbb3f36521a54c86
Author: Alex Rousskov <email address hidden>
Date: Sun Dec 29 07:56:02 2013 -0800

    Bug 3498: FTP PUT assertion Server.cc:246: "r->body_pipe != NULL"

diff --git a/src/ftp.cc b/src/ftp.cc
index f4b108080..71f3d3035 100644
--- a/src/ftp.cc
+++ b/src/ftp.cc
@@ -3051,6 +3051,13 @@ void FtpStateData::readStor()
     debugs(9, 3, HERE);

     if (code == 125 || (code == 150 && Comm::IsConnOpen(data.conn))) {
+ if (!originalRequest()->body_pipe) {
+ debugs(9, 3, "zero-size STOR?");
+ state = WRITING_DATA; // make ftpWriteTransferDone() responsible
+ dataComplete(); // XXX: keep in sync with doneSendingRequestBody()
+ return;
+ }
+
         if (!startRequestBodyFlow()) { // register to receive body data
             ftpFail(this);
             return;

tags: added: bite-size server-next
Changed in squid3 (Ubuntu Trusty):
assignee: nobody → Andreas Hasenack (ahasenack)
status: Triaged → In Progress
description: updated

Hello Stanislav, or anyone else affected,

Accepted squid3 into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/squid3/3.3.8-1ubuntu6.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in squid3 (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-trusty
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.