cat of long files breaks dropbear ssh connection

[Madper Xie]

I'm connecting via ssh to our platform running dropbear. If I cat /var/log/syslog, I'm seeing the connection break with the following:
Write failed: Broken pipe

The syslog is about 2MB size.

I can reproduce the issue in ubuntu 14.04.3.
The version of dropbear is 2013.60-1ubuntu2.1

`xxd /dev/zero` can reproduce the issue 100%.

[Impact]

 * In fact it is really an annoying bug due to I can't visit my syslog or other large file via cat.

[Test Case]
The minimum reproducer should be:
 * system A running a dropbear as a ssh server.
 * ssh username@systemA from system B. (A and B should be different computer.)
 * xxd /dev/zero
 * the connection will be broken.

[Regression Potential]
 * I find the patch from upstream. And it already been merged. So I think it's safe to merge it into ubuntu.

[Other Info]
 * The latest dropbear-2015.68 can't reproduce the issue.
 * After a bisect I find this patch fixed the issue: https://github.com/mkj/dropbear/commit/db688e3
 * There is a bit conflict if you apply above commit. The fixed version is attached.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "test.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Madper Xie (bbboson)]

I just test ubuntu 15.10 nightly. can't reproduce the same issue.

ii dropbear 2014.65-1ubuntu2 amd64 lightweight SSH2 server and client

u@u-ThinkPad-T450:~$ cat /etc/issue
Ubuntu Wily Werewolf (development branch) \n \l

[Matt Johnston (matt-ucc)]

I can't immediately see how that patch would fix it (I wrote it). When dropbear fails does it log anything in auth.log?

Cheers,
Matt

On 21 September 2015 4:03:24 pm AWST, Madper Xie <email address hidden> wrote:
>Public bug reported:
>
>I'm connecting via ssh to our platform running dropbear. If I cat
>/var/log/syslog, I'm seeing the connection break with the following:
>Write failed: Broken pipe
>
>The syslog is about 2MB size.
>
>I can reproduce the issue in ubuntu 14.04.3.
>The version of dropbear is 2013.60-1ubuntu2.1
>
>`xxd /dev/zero` can reproduce the issue 100%.
>
>[Impact]
>
> * In fact it is really an annoying bug due to I can't visit my syslog
>or other large file via cat.
>
>[Test Case]
>The minimum reproducer should be:
> * system A running a dropbear as a ssh server.
> * ssh username@systemA from system B. (A and B should be different
>computer.)
> * xxd /dev/zero
> * the connection will be broken.
>
>[Regression Potential]
> * I find the patch from upstream. And it already been merged. So I
>think it's safe to merge it into ubuntu.
>
>[Other Info]
> * The latest dropbear-2015.68 can't reproduce the issue.
> * After a bisect I find this patch fixed the issue:
>https://github.com/mkj/dropbear/commit/db688e3
> * There is a bit conflict if you apply above commit. The fixed version
>is attached.
>
>** Affects: dropbear (Ubuntu)
> Importance: Undecided
> Status: New
>
>** Patch added: "test.patch"
>https://bugs.launchpad.net/bugs/1497883/+attachment/4470114/+files/test.patch
>
>** Description changed:
>
>I'm connecting via ssh to our platform running dropbear. If I cat
>/var/log/syslog, I'm seeing the connection break with the following:
> Write failed: Broken pipe
>
> The syslog is about 2MB size.
>+
>+ I can reproduce the issue in ubuntu 14.04.3.
>+ The version of dropbear is 2013.60-1ubuntu2.1
>+
> `xxd /dev/zero` can reproduce the issue 100%.
>- [Impact]
>
>- * In fact it is really an annoying bug due to I can't visit my
>syslog
>+ [Impact]
>+
>+  * In fact it is really an annoying bug due to I can't visit my
>syslog
> or other large file via cat.
>
> [Test Case]
> The minimum reproducer should be:
>- * system A running a dropbear as a ssh server.
>- * ssh username@systemA from system B. (A and B should be different
>computer.)
>- * xxd /dev/zero
>- * the connection will be broken.
>+  * system A running a dropbear as a ssh server.
>+  * ssh username@systemA from system B. (A and B should be different
>computer.)
>+  * xxd /dev/zero
>+  * the connection will be broken.
>
>- [Regression Potential]
>- * I find the patch from upstream. And it already been merged. So I
>think it's safe to merge it into ubuntu.
>+ [Regression Potential]
>+  * I find the patch from upstream. And it already been merged. So I
>think it's safe to merge it into ubuntu.
>
> [Other Info]
>- * The latest dropbear-2015.68 can't reproduce the issue.
>- * After a bisect I find this patch fixed the issue:
>https://github.com/mkj/dropbear/commit/db688e3
>- * There is a bit conflict if you apply above commit. The fixed
>version is attached.
>+  * The latest dropbear-2015.68 can't reproduce the issue.
>+  * After a bi...

[Madper Xie (bbboson)]

Here is what I got from auth.log:

Sep 21 21:58:32 madper-HP-ProBook-4230s dropbear[13147]: Not backgrounding
Sep 21 21:58:35 madper-HP-ProBook-4230s dropbear[13178]: Child connection from 192.168.1.217:43352
Sep 21 21:58:37 madper-HP-ProBook-4230s dropbear[13178]: Password auth succeeded for 'madper' from 192.168.1.217:43352
Sep 21 21:58:40 madper-HP-ProBook-4230s dropbear[13178]: Exit (madper): Error writing
Sep 21 21:58:46 madper-HP-ProBook-4230s dropbear[13147]: Premature exit: Terminated by signal

[Matt Johnston (matt-ucc)]

I wonder if https://secure.ucc.asn.au/hg/dropbear/rev/c4f138dae2fd might help, if the problem is that write() is returning EAGAIN.

If you apply the patch https://secure.ucc.asn.au/hg/dropbear/rev/3873b39c4de6 the auth.log output should show why write() is failing.

[Madper Xie (bbboson)]

Hmmmm, I just got a time slice to test the first patch in #6.
However after apply it I still got:

root@54631e34cc9c:~/dropbear-2013.60# ./dropbear -F -E -B
[6909] Jun 02 13:52:45 Not backgrounding
[6910] Jun 02 13:52:48 Child connection from 172.17.0.2:33058
[6910] Jun 02 13:52:49 Password auth succeeded for 'madper' from 172.17.0.2:33058
[6910] Jun 02 13:52:52 Exit (madper): Error writing

Let me apply the 2nd patch to find why the write() failed.

[Madper Xie (bbboson)]

Okay, after I apply 2nd patch I got:

root@54631e34cc9c:~/dropbear-2013.60# ./dropbear -F -E -B
[8066] Jun 02 13:56:30 Not backgrounding

[8067] Jun 02 13:56:34 Child connection from 172.17.0.2:33108
[8067] Jun 02 13:56:35 Password auth succeeded for 'madper' from 172.17.0.2:33108
[8067] Jun 02 13:56:37 Exit (madper): Error writing: Invalid argument

[Madper Xie (bbboson)]

This time I print line number in debug info:

root@54631e34cc9c:~/dropbear-2013.60# ./dropbear -F -E -B
[8131] Jun 02 14:07:21 Not backgrounding
[8132] Jun 02 14:07:22 Child connection from 172.17.0.2:33726
[8132] Jun 02 14:07:23 Password auth succeeded for 'madper' from 172.17.0.2:33726
[8132] Jun 02 14:07:30 Exit (madper): Error writing: Invalid argument line: 89

So HAVE_WRITEV is true in my case.

[Matt Johnston (matt-ucc)]

Does this attached patch work for you? It fixes when I test here.

Combination of
https://secure.ucc.asn.au/hg/dropbear/rev/c919dbb39395
https://secure.ucc.asn.au/hg/dropbear/rev/68723d66dec6
https://secure.ucc.asn.au/hg/dropbear/rev/88043f9d40bd

[Madper Xie (bbboson)]

@Matt,
Sorry for the late.
I already tested, the patch in comment #10 fixes the issue. Do you plan to backport the patch into 14.04?

[Matt Johnston (matt-ucc)]

I guess that's a question for the Ubuntu maintainer :)

Matt

> On Mon 8/8/2016, at 9:31 pm, Madper Xie <email address hidden> wrote:
>
> @Matt,
> Sorry for the late.
> I already tested, the patch in comment #10 fixes the issue. Do you plan to backport the patch into 14.04?
>
> --
> You received this bug notification because you are subscribed to
> dropbear in Ubuntu.
> https://bugs.launchpad.net/bugs/1497883
>
> Title:
> cat of long files breaks dropbear ssh connection
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/oem-priority/+bug/1497883/+subscriptions

[gerald.yang (gerald-yang-tw)]

This debdiff comes from comment #10 to fix the bug