RFE: enable cookies for data sources

[Peter Levi]

From PheniX:

Some data sources allow a wider selection of wallpapers if the user is logged in.
I have tried a very fast hack to add cookies at URL fetch by adding to the Util.urllib function something like:

    if url.find('wallhaven') > 0:
        cookies = "name0=value0;" # put here the session cookies that identify an open session/logon
        cookies += "name1=value1;"
        request.add_header('Cookie', cookies )

just before the line

    return urllib2.urlopen(request, data=urllib.urlencode(data) if data else None, timeout=20)

This works enough for me, but a better way could be:
- the user saves the datasource cookies with a browser to a text file after logon (plenty of addons help doing it),
- enabling cookie handling in variety with cookielib by reading from the saved cookie file

Or even better:
- enable cookie handling with cookielib
- add a Logon dialog for the specific datasource to do logon straight from variety

[Peter Levi (peterlevi)]

@PheniX:

I would rather NOT have the logon dialog in Variety - asking people to provide credentials for external services within an application like Variety teaches everyone bad security practices. There is OAuth for this exact reason and it should be used when available. I had something like this for Facebook sharing, but in an embedded Webkit, still passing via OAuth, and I'm glad I got rid of it and use OAuth properly in the browser now for FB.

Flickr supports OAuth, but I'm not sure the user gets much benefit from being logged in to Flickr, the effort to implement OAuth there just doesn't seem worth it.

For Wallhaven OAuth is not an option.

I guess what we can have is to add support for a folder ~/.config/variety/login-cookies where users will be able to put cookie text files and Variety will use them, like in the first solution you mention, then we document this on the website and in Tips-and-tricks. This means only more advanced and interested users will be able to use it.

PheniX, would like to provide a patch for this?

[PheniX (a-sterbini)]

I agree with you that Oauth is the correct way o handle external logon and in view of that that the cookie's directory is the nicest option.
I am not proficient in using cookielib to handle a cookies file but I can give it a try as soon I find some time

[PheniX (a-sterbini)]

Hi Peter
I have done some doc search and it seems that something like the following lines should do the trick

       # To enable sessions only in wallhaven
        if url.find('wallhaven') > 0:
            cj = cookielib.MozillaCookieJar()
            cj.load('~/.config/variety/cookies/wallhaven.txt') # I normally use here the full path of the file
            opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
            urllib2.install_opener(opener)

to be placed in Utils just before the line
        request = urllib2.Request(url) if not head_request else HeadRequest(url)

Yet, the cookie file saved (for example) by Firebug (which is really nice because it exports only the cookies of the selected website)
requires some massaging to be loaded:
- first add the lines (the first one is the really important one)

# Netscape HTTP Cookie File
# http://www.netscape.com/newsref/std/cookie_spec.html
# This is a generated file! Do not edit.

- then you must add to each cookie line (if the expiration field is missing) a tab just before the cookie name field

... I must test all this some more ... sometime the server complains with a 400 code ...

Ciao
A

[PheniX (a-sterbini)]

I hadn't time to pursue the cookielib road and managed the following (too) simple hack in Util.py:

    @staticmethod
    def urlopen(url, data=None, head_request=False):
        if url.startswith('//'):
            url = 'http:' + url
        request = urllib2.Request(url) if not head_request else HeadRequest(url)
        request.add_header('User-Agent', USER_AGENT)
        request.add_header('Cache-Control', 'max-age=0')

        # session cookies for wallhaven
        cookiefile = '~/.config/variety/cookies/cookies.txt'
        if url.find('wallhaven')>0 and os.path.exists(cookiefile):
            cookies = ''
            with open(cookiefile) as CF:
                for line in CF:
                    _, _, _, _, name, value = line.strip().split('\t')
                    cookies += "%s=%s;" % (name,value)
            request.add_header('Cookie', cookies )

        return urllib2.urlopen(request, data=urllib.urlencode(data) if data else None, timeout=20)

Once Util.py is "fixed" as above, to enable session cookies for wallhaven you just log-on with Firefox to wallhaven and save the session cookies to ~/.config/variety/cookies/cookies.txt by using the Firebug extension