sftp in 12.04 has tty attributes setting problem

[wzis]

In Ubuntu 12.04 LTS, when doing following:
1. stty -echo
2. sftp user@server
3. key in pass as required.
4. cd /tmp
5. bye
6. stty -a

By right, the contents entered in step 4-6 shouldn't be visible, but they do on 12.04.
Found the same issue on RHEL 6.x. But sftp on Solaris/AIX/HP-UX and RHEL 5.x do not have this issue.

[Robie Basak (racb)]

Thank you for reporting this bug and helping to make Ubuntu better.

I don't think that it's clear that the behaviour should be as you expect. I think it's normal for programs to disable echo when requesting a password from the terminal, and then to enable echo again, without any regard for whether echo was already disabled to start with.

This is the sort of issue that it would be best for upstream to consider and fix if they consider it appropriate. I think it would be best for distributions to defer to upstream's opinion on this, rather than fragment behaviour by fixing things on their own.

Could you please test the latest upstream release, and if this still affects the upstream release without distribution patches, file a bug in the upstream bug tracker to get upstream's opinion?

Setting this bug to Incomplete, as it needs an opinion from upstream as to whether this is really a bug at all. If you find that Ubuntu's behaviour differs from upstream, or if upstream agree that this is a bug, then please change the bug status back to New. If upstream consider the current behaviour to be the correct behaviour, the please change the bug status to Invalid. Thanks!

[wzis (wzis)]

The proper behavior should be when prompt for password/passphrase, turn off the echo, and after that restore the original setting. This is extremely important especially when exit from sftp, the tty setting should be at its original state just before entering sftp!

[Robie Basak (racb)]

I understand that this is your point of view. But I don't think it's appropriate to make a change in Ubuntu unless upstream agree with you. Please can you consult them for their view on this?

[wzis (wzis)]

And we run "stty -echo" before sftp is for security reason as we don't want the user to know what we are doing.

[wzis (wzis)]

the sftp program in RHEL 6.1/6.2/6.3/6.4 has a bug in tty settings.

The 5.3p1 sftp program used in RHEL 6.1/6.2/6.3/6.4, and Ubuntu 12.04 LTS has tty attributes setting problem.

Steps to Reproduce:
1. stty -echo
2. sftp user@server
3. key in pass as prompted.
4. cd /tmp
5. ls
6. bye
7. stty -a

Actual results:
The entered sftp sub-commands in step 4 to step 6 are visible, and the OS command entered after sftp in step 7 is also visible.

Expected results:
Those inputs entered in step 4 to step 7 shouldn't be visible.
This is for security reason we want to hide the sftp subcommands when presented in the menu for user to get files.

[Darren Tucker (dtucker)]

I can't reproduce this on a fedora 18 box with openssh-5.3p1, built with or without libedit ("libedit-3.0-7.20120601cvs").

My guess is that it's actually a bug in libedit.

have you reported this to ubuntu? can you reproduce it with the vanilla source from openssh.com? with or without libedit?

[wzis (wzis)]

I reported the issue to ubuntu, but they said I need to report it in upstream.

[Darren Tucker (dtucker)]

who from ubuntu said that? is there an ubuntu or redhat bugreport?

IMO that attitude from a vendor is laziness on their part. They're shipping an old, modified version of openssh, so it's their responsibility to investigate problems to a) make sure it's not due to their changes, b) not already fixed and c) the problems are reported against the right upstream.

In this case, I happen to have an ubuntu 12.04 VM, and I can't reproduce the problem without libedit, but *can* reproduce it --with-libedit (2.11-20080614-3ubuntu2) but not on Fedora (which has libedit 3.0-7.20120601).

[wzis (wzis)]

Agree with you: here is the bug reported to ubuntu:
https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1225327

[wzis (wzis)]

So, based on your test, it's libedit caused the problem for ssh/sftp.
This is the first time I reported bug to ubuntu, and found them not so helpful.

[wzis (wzis)]

I did a test on Fedora 19: found it also has a problem: the sub-commands in sftp are still visible, but after exits from sftp, the setting recovered. Could you try on the Fedora 18 you have, using the Fedora's sftp?

[wzis (wzis)]

An openssh developer has helped to debug the issue and found out the issue is not openssh 5.3p1 caused but the libedit version 2.11. Please fix the problem.

[Darren Tucker (dtucker)]

fedora 18's sftp does do it, but the one I built doesn't (although it's linked against a locally built libedit in /usr/local, which I had not noticed):

$ stty -echo; /usr/bin/sftp -o preferredauthentications=password localhost; stty sane
dtucker@localhost's password:
Connected to localhost.
sftp> cd /tmp
sftp> exit

$ ldd /tmp/openssh-5.3p1/bin/sftp | grep libed
 libedit.so.0 => /usr/local/lib/libedit.so.0 (0xb7744000)

$ stty -echo; /tmp/openssh-5.3p1/bin/sftp -o preferredauthentications=password localhost; stty sane
Connecting to localhost...
dtucker@localhost's password:
sftp> sftp> Remote working directory: /tmp
sftp>

$ rpm -q libedit
libedit-3.0-7.20120601cvs.fc18.i686

the local one is libedit-20100424-3.0

[Darren Tucker (dtucker)]

repeated the test with libedit-20130712-3.1 and it fails. so:

20100424: works
20120601: fails
20130712: fails

I'd be taking a pretty close look at this change, which was just before the 20120601 release:L

http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libedit/tty.c?rev=1.42&content-type=text/x-cvsweb-markup&sortby=date

save and restore the tty settings on entry and exit respectively.
cleanup debugging printfs

[Robie Basak (racb)]

Thanks for following this up. The same questions come up though. Is this problem fixed in the latest upstream libedit? If not, it should be taken upstream for a fix. If it is fixed in the latest upstream, then we need to figure out which release it was fixed in, and then make sure Ubuntu will release with a new enough release, or identify and cherry-pick a patch.

[wzis (wzis)]

The openssh developer has questioned why Ubuntu bug fixer didn't do any checking and ask upstream to check? It's Ubuntu taking the package and may have made change to the package, so Ubuntu should do something when user reports bug to them.

[Robie Basak (racb)]

Indeed, it is inappropriate to take bugs upstream in case the distribution made a change to the package. This is why I said "Could you please test the latest upstream release, and if this still affects the upstream release without distribution patches..." and have asked the same for libedit.

I am not an "Ubuntu bug fixer" and am not stepping up to fix this bug. I am merely helping to triage this bug, so that we can confirm if and what needs to be fixed in Ubuntu. If you are unwilling to help, then you're welcome to keep this bug open in the bug tracker and wait to see if someone else will step up to fix it.

[Robie Basak (racb)]

> IMO that attitude from a vendor is laziness on their part. They're
> shipping an old, modified version of openssh, so it's their
> responsibility to investigate problems to a) make sure it's not due to
> their changes, b) not already fixed and c) the problems are reported
> against the right upstream.

Hi. I triaged this bug in Ubuntu. I feel that I have been mispresented here.

I appreciate that I shouldn't be sending bugs upstream without checking them first, being reasonably confident that they affect the latest upstream release and have not been introduced by the distribution.

I did NOT say that this bug was ready to be reported upstream. I specifically said that the bug needed to be tested against the latest upstream release without distribution patches, and only in that case should the bug be reported upstream.

I said: "Could you please test the latest upstream release, and if this still affects the upstream release without distribution patches, file a bug in the upstream bug tracker to get upstream's opinion?"

In the case of this particular bug I wasn't even sure if this was expected behaviour by design. I do not wish to diverge from upstream on behaviour such as this. So if verified that this affected the latest upstream release without distribution patches, I then wanted to check with you if you considered this behaviour a bug at all.

I'm sorry that it didn't occur to me that you were depending on another library for this functionality. As you can see from the bug, I didn't get that far.

Rather than laziness, we are in a situation where far too many bugs are reported, and there aren't enough bug triagers to keep up. So rather than have bugs languish forever, I prefer to note in the bug what actions need to be taken next to empower the reporter to do them to at leave the option of making progress. Bugs affecting only one person after over a year of release tend not to get looked at any other way.

Again, I did not intend to get you involved without a reporter having done basic checks to ensure that your latest release was affected, and I apologise for the noise.

[Robie Basak (racb)]

This comment might help identify the issue in libedit: https://bugzilla.mindrot.org/show_bug.cgi?id=2155#c8

[wzis (wzis)]

I compiled the 5.3p1 taken from openssh.com and tested, it doesn't show the problem. So it's very clear it's Ubuntu complied sftp program that has the issue.
Though by default, the openssh configure doesn't include "--with-libedit", it's Ubuntu added that option.

[wzis (wzis)]

It seems to me that the proper solution to make ssh/sftp work as on other UNIX platforms is compile it without using the "--with-libedit": the functions provided by libedit are not necessary to run ssh/sftp and not that useful in use ssh/sftp also: the openssh is used on all other UNIX platforms without the libedit, and I don't think those UNIX vendors have received any meaningful number of complaints for not having libedit. With libedit, openssh lost control to the tty settings which caused end user not able to control whether they want to hide the sftp sub-commands.

[wzis (wzis)]

Based on Dtucker's debugging and tests, it seems to me when libedit is used, openssh is unable to control the tty settings based on its own needs, so with that in mind, and also the fact sftp on other UNIX or RHEL 5.x has no such problem and no other critical issue caused by not using libedit, the openssh should be packaged without using libedit.
Wondering why in the first place, the libedit is included in the configure.

[Darren Tucker (dtucker)]

(In reply to robie.basak from comment #9)
> I feel that I have been mispresented here.

I get that impression after having read the ubuntu bug, so my apologies.

anyway, having looked at this some more I'm not convinced it's a bug, or at least that that fact that it behaves that way without libedit is anything more than an implementation detail. Even in batch mode, sftp makes a point of echoing the commands:

    if (!interactive) { /* Echo command */
            printf("sftp> %s", cmd);

I also tried disabling libedit ("sftp:edit off" in ~/.editrc) however that didn't change the behaviour.

@wzis what exactly are you trying to do here? ie what's the objective of the exercise?

[wzis (wzis)]

By right, a well behaved program should handle the echo only at the necessary point: for example, to read password/passphrase, the echo needs be turned off, but after that, echo should be recovered to what user set. If the program wants to make sure a string be visible, it should explicitly output it, not by turn on the echo when user wants to turn it off. Only that way, both program can work properly and user's requirement can be met.
We use that feature to give user a menu to download files, but not let user see which files.

[wzis (wzis)]

By the way, if the program outputs the command, our program can receive the outputs and filters them out. So, that's still ok. But turn the echo on is not what other program can revert it back, that's a bug.