curl and pycurl is not compiled with sftp support

Bug #311029 reported by Alex on 2008-12-23
108
This bug affects 21 people
Affects Status Importance Assigned to Milestone
curl (Debian)
Fix Released
Unknown
curl (Ubuntu)
Low
Unassigned
Declined for Lucid by Sebastien Bacher
Declined for Maverick by Sebastien Bacher

Bug Description

Binary package hint: curl

Running curl --version, I see:
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps
SFTP is obviously missing, although it is supported by curl upstream
The SFTP is also missing from python-pycurl, probably because libcurl doesn't have it.

Martin Lindhe (martinlindhe) wrote :

I second that.
Installed php5-curl to use libcurl with php scripts to read/write to a sftp, only to find that the packaged curl is built without sftp support.
Please enable sftp protocol support in curl!

Brian Murray (brian-murray) wrote :

sftp and scp support are unavailable with curl version 7.19.5-1ubuntu2 in Karmic Koala.

Changed in curl (Ubuntu):
importance: Undecided → Low
status: New → Triaged
Martin Lindhe (martinlindhe) wrote :

To generate a curl with ssh support on Ubuntu 9.04:

sudo apt-get install build-essential debhelper libssh2-1-dev
sudo apt-get source libcurl3
sudo apt-get build-dep libcurl3

cd curl-7.18.2/debian

gedit rules

   find and replace "--without-libssh2" with "--with-libssh2"

cd ..

sudo dpkg-buildpackage

cd ..

sudo dpkg -i curl_7.18.2-8ubuntu4.1_amd64.deb
sudo dpkg -i libcurl3_7.18.2-8ubuntu4.1_amd64.deb
sudo dpkg -i libcurl3-gnutls_7.18.2-8ubuntu4.1_amd64.deb

Martin Lindhe (martinlindhe) wrote :

Upon further inspection, it appears that the debian package is configured with "--without-libssh2"

Can someone shed some light as to why libssh2 is disabled?
It makes curl unable to handle sftp or scp protocols.

It sure would be nice if the package could be built with "--with-libssh2" for Lucid's release. Any chance of that happening?

goraxe (goraxe) wrote :

I have attached a patch for this. One of our applications depends on this, we currently are building our own version of this package with patch applied, I have just had to rebuild for lucid.

tags: added: patch
Martin Lindhe (martinlindhe) wrote :

goraxe: your patch contains unrelated changes.
All that is needed is in curl/debian/rules

- cd debian/build-gnutls && ./configure ${CONFIGURE_ARGS} --with-ca-bundle=/etc/ssl/certs/ca-certificates.crt --without-ssl --with-gnutls --without-libssh2
+ cd debian/build-gnutls && ./configure ${CONFIGURE_ARGS} --with-ca-bundle=/etc/ssl/certs/ca-certificates.crt --without-ssl --with-gnutls --with-libssh2

/martin

Martin Lindhe (martinlindhe) wrote :

created a minimal patch to debian/rules to fix the issue

As a side note I have been using this to enable ssh in curl on a production server since September 2009 and it has been working perfectly.

Nigel Babu (nigelbabu) wrote :

This patch has been reviewed as part of operation cleansweep. Thanks for your patch. This particular change is inherited from debian, could you please open a bug in Debian and attach the patch? It would be much better to hear upstream Debian thoughts on this.

goraxe (goraxe) wrote :

I have opened a bug with the debian bug tracking system

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=587735

tags: added: patch-forwarded-debian
removed: patch
Changed in curl (Debian):
status: Unknown → New
Cameron Smith (cameron-smith) wrote :

Here are Martin Lindhe's instructions from post #3 above, updated for 10.04, again assuming you are using 64-bit Ubuntu. The resulting curl installation works just as the curl mapage says it should, against an OpenSSL server.

==========
#the line below will create a temporary directory and move you to it - so that you can easily clean up the build files later
# you are short on disk space
mkdir -p ~/temp/curl
cd ~/temp/curl

sudo apt-get install build-essential debhelper libssh2-1-dev
sudo apt-get source libcurl3
sudo apt-get build-dep libcurl3

cd curl-7.19.7/debian

gedit rules

   find and replace "--without-libssh2" with "--with-libssh2"

cd ..

sudo dpkg-buildpackage

cd ..

sudo dpkg -i curl_7.19.7-1ubuntu1_amd64.deb
sudo dpkg -i libcurl3_7.19.7-1ubuntu1_amd64.deb
sudo dpkg -i libcurl3-gnutls_7.19.7-1ubuntu1_amd64.deb

#at this point you may move away from, and then remove the entire ~/temp/curl directory if you wish
# alternately, you may choose just to keep the 3 .deb files which were the end product of the building,
# in case you have to use them again in the future.

Sebastien Bacher (seb128) wrote :

is that still an issue with the current version?

Martin Lindhe (martinlindhe) wrote :

according to http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=587735#15
the latest unstable debian packages (7.21.0-1) has sftp enabled

Andreas Olsson (andol) wrote :

If I understand the debian/rules file correctly, the --without-libssh2 option should only affect libcurl3-gnutls and not libcurl3? Futher it seems like the command line curl uses the (openssl) libcurl3?

The problem with curl and libcurl3 seems more related to a specific Ubuntu delta. That is that Ubuntu drops the build-dependency on libssh2-1-dev. When I rebuilt curl 7.21.0-1ubuntu1 (current maverick) with libssh2-1-dev readded as a build dependency sftp support worked just fine.

Martin Lindhe (martinlindhe) wrote :

Andreas: I think you are confusing ssh and ssl/tls support.

Andreas Olsson (andol) wrote :

Martin: How so?

Martin Lindhe (martinlindhe) wrote :

Andreas wrote "If I understand the debian/rules file correctly, the --without-libssh2 option should only affect libcurl3-gnutls and not libcurl3?"

gnutls is "Transport Layer Security, a network protocol and successor to Secure Sockets Layer (SSL)", it is unrelated to ssh.

Andreas wrote "Futher it seems like the command line curl uses the (openssl) libcurl3?"

Correct. however this bug is about missing sftp (ssh) support and not about missing ssl support (it is indeed there).

Andreas wrote "The problem with curl and libcurl3 seems more related to a specific Ubuntu delta. That is that Ubuntu drops the build-dependency on libssh2-1-dev."

According to mine and others previously analysis the rules/debian file are from Debian repo who also have disabled ssh support. See related upstream bug http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=587735

Andreas wrote "When I rebuilt curl 7.21.0-1ubuntu1 (current maverick) with libssh2-1-dev readded as a build dependency sftp support worked just fine."

How so? the rules/debian file shipped has a "--without-libssh2" configure paramater which seems to be the cause of the error.

Andreas Olsson (andol) wrote :

No, I am not confusing ssh with ssl/tls. Yet, it is very possible that I am not expressing myself very clearly.

The reason I am mentioning gnutls and openssl is to differentiate between the libraries/packages libcurl3 and libcurl3-gnutls.

The option --without-libssh2 is only used when building the libcurl3-gnutls package. Hence it has nothing to do with the ability to use sftp:// with anything which uses/depends on libcurl3, such as curl or php5-curl.

Whatever libcurl3 can provide sftp support or not depends on whatever libssh2-1-dev is present during compile time.

I am not saying that --with(out)-libssh2 doesn't matter. There are a multitude of tools (including python-pycurl) depending on libcurl3-gnutls. My point is that that compile option isn't the whole stole.

Martin Lindhe (martinlindhe) wrote :

Andreas:

You are indeed correct.

I've myself recompiled and only installed the resulting libcurl to get SSH support (both command line and in php5-curl).
Before doing this, i also installed the libssh2-1-dev.

This would result in a working setup but my conclusions as to why were obviously flawed (ps. thanks for pointing this out!)

Attached is a corrected patch against curl-7.21.0 (maverick).

I added the dependency against the "curl" package. However I'm not sure if it should rather be against "libcurl3".

Can someone help confirm this solution so we can try to push a patch upstream.

Andreas Olsson (andol) wrote :

I am afraid it is not as simple as re-adding the build-dependency on libssh2-1-dev. It was not removed by mistake, but as an explicit decision, per bug #175891.

For curl to be able to depend on libssh2 it too would have to be in Main. See https://wiki.ubuntu.com/UbuntuMainInclusionRequirements for more information.

Martin Lindhe (martinlindhe) wrote :

Andreas, thanks for researching this.

I don't know how to proceed with resolving this issue since I dont have enough experience with these things.

As far as I can see there are two options

a) open a new bug report requesting libssh2 to be moved to main

or

b) create a separate package in universe which would enable ssh support in curl if installed.

I think method A would be easier to do, however it may be a more politicial issue than a technical one to resolve.

As for method B, i wouldn't know how we could do that

Andreas Olsson (andol) wrote :

Being more of a triager than a developer myself I am not sure myself what the best route is. Assuming you are comfortable with IRC you might want to try #ubuntu-devel or #ubuntu-motu on freenode.

Martin Lindhe (martinlindhe) wrote :

I have opened a MIR request in lp bug #681423

Steel-Cat (panther-toppoint) wrote :

Hello Martin,

your MIR was closed for inactivity... what a pity...

Is the maintainer of libssh2 not able to resolve the comments from Kees Cook?

How we can advance?

Steel-Cat

Changed in curl (Debian):
status: New → Fix Released
KennethOnah (onah-kenneth) wrote :

This is 2015 and still curl does not support sftp out of the box. Why so?

Martin (ub71-martin) wrote :

Yes, why ? Please ?

Uqbar (uqbar) wrote :

This is actually 2016.
15.10 curl is:

    [Uqbar@Feynman ~] curl --version
    curl 7.43.0 (x86_64-pc-linux-gnu) libcurl/7.43.0 GnuTLS/3.3.15 zlib/1.2.8 libidn/1.28 librtmp/2.3
    Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
   Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets

Moreover the upstream bug got fixed long ago: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=587735 .

What else?

mdyn (tamerlaha-gmail) wrote :

7 years old bug... niiice.

Uqbar (uqbar) wrote :

As of 16.04 we're still lacking behind: NO SFTP!!!

[Uqbar@Feynman ~] curl --version
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets

C'mon, guys! Is it that difficult to pull from Debian?
May I say it's a shame?

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.