lens searches can be unmasked by local network sniffing

Bug #1074780 reported by mike davis
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
unity-lens-shopping (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

first i want to say that the default nature of the amazon spam plugin really is a violation of the community trust, and I highly advocate the EFF's position on this plugin.. the user should have the choice *before* their information is reported to some entity on the internet..

issue:
while its true that the lens encrypts search queries to the productsearch.ubuntu.com server, the subsequent fetch of the image links within the search results and the algorithmicly generated nature of the results on the server allow a local network user to sniff the network for HTTP get requests to the ubuntu server to unmask either the exact search term, or a closely related terms of an ubuntu user.

how this works in the real world:
an "eve" precaches the search results using a word list and parses the json results and notes which and how many image results were provided for a particular word of interest.. "eve" then sniffs the network looking for bursts of image requests, the attacker then compares the block of image requests to the results that were cached earlier and and scores the results.

the search term (or closely related search term) is then revealed

an attacker can also choose to build the dictionary after the initial packet sniffing so long as the server cached contents havnt shifted significantly .. though it is likely the results would still me similar enough to score the results for a best fit.

an example:
"eve" has a database filled by requesting a list of interesting search terms, below is the query for "diapers":

phar@thing:~/ubuntu curl https://productsearch.ubuntu.com/v1/search?q=diapers 2> /dev/null | grep ecx.images-amazon.com | grep SL160
          "http://ecx.images-amazon.com/images/I/41w92ZKCHBL._SL160_.jpg"
          "http://ecx.images-amazon.com/images/I/51xRI9n2puL._SL160_.jpg"
          "http://ecx.images-amazon.com/images/I/516o3TWAOBL._SL160_.jpg",
          "http://ecx.images-amazon.com/images/I/5197vs3wtvL._SL160_.jpg",
          "http://ecx.images-amazon.com/images/I/51UEzvC7X9L._SL160_.jpg",
          "http://ecx.images-amazon.com/images/I/51ZFlIGw0DL._SL160_.jpg",
          "http://ecx.images-amazon.com/images/I/51b3JCCi6RL._SL160_.jpg",
          "http://ecx.images-amazon.com/images/I/51p7qujvx2L._SL160_.jpg"
          "http://ecx.images-amazon.com/images/I/51tV-ZBj2aL._SL160_.jpg"
          "http://ecx.images-amazon.com/images/I/41T4yIgZzNL._SL160_.jpg",
          "http://ecx.images-amazon.com/images/I/41gmpjcLEuL._SL160_.jpg",
          "http://ecx.images-amazon.com/images/I/41lX0WGGOrL._SL160_.jpg",
          "http://ecx.images-amazon.com/images/I/41qoOh5-jqL._SL160_.jpg",
          "http://ecx.images-amazon.com/images/I/5167DrJVUEL._SL160_.jpg",
          "http://ecx.images-amazon.com/images/I/51iitgcf%2BvL._SL160_.jpg"
          "http://ecx.images-amazon.com/images/I/51LCvCjDnOL._SL160_.jpg"
          "http://ecx.images-amazon.com/images/I/51M7z0dUXDL._SL160_.jpg"
          "http://ecx.images-amazon.com/images/I/41QtRL2VlXL._SL160_.jpg",
          "http://ecx.images-amazon.com/images/I/51gD2PgaJ9L._SL160_.jpg"
          "http://ecx.images-amazon.com/images/I/51MS7z8oHhL._SL160_.jpg"
          "http://ecx.images-amazon.com/images/I/51eO4S5QRiL._SL160_.jpg"

now, eve sniffs the network looking for a closly related burst of image queries:

phar@thing:~/ubuntu sudo ngrep GET -S 50 -d eth1 -q -t
interface: eth1 (192.168.1.0/255.255.255.0)
match: GET

T 2012/11/03 16:52:57.664091 192.168.1.7:53387 -> 54.240.188.195:80 [AP]
  GET /images/I/410xVwYbA9L._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:52:57.668615 192.168.1.7:46213 -> 54.240.188.34:80 [AP]
  GET /images/I/21Ke7hDgllL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:52:57.669380 192.168.1.7:46985 -> 54.240.188.248:80 [AP]
  GET /images/I/51lACGaNvpL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:52:57.693032 192.168.1.7:46922 -> 205.128.91.126:80 [AP]
  GET /images/I/31Agova21UL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:53:18.938638 192.168.1.7:57036 -> 54.240.188.68:80 [AP]
  GET /images/I/41w92ZKCHBL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:53:19.043135 192.168.1.7:44472 -> 98.142.98.180:80 [AP]
  GET /static/img/sleeveart/00/012/360/0001236002_17

T 2012/11/03 16:53:19.047354 192.168.1.7:44474 -> 98.142.98.180:80 [AP]
  GET /static/img/sleeveart/00/016/006/0001600688_17

T 2012/11/03 16:53:19.050731 192.168.1.7:59410 -> 54.240.188.137:80 [AP]
  GET /images/I/51tV-ZBj2aL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:53:19.125583 192.168.1.7:44475 -> 98.142.98.180:80 [AP]
  GET /static/img/sleeveart/00/000/914/0000091491_17

T 2012/11/03 16:53:19.127287 192.168.1.7:46998 -> 54.240.188.248:80 [AP]
  GET /images/I/516o3TWAOBL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:53:19.135532 192.168.1.7:36150 -> 54.240.188.53:80 [AP]
  GET /images/I/41T4yIgZzNL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:53:19.137307 192.168.1.7:50431 -> 54.240.188.26:80 [AP]
  GET /images/I/51LCvCjDnOL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:53:19.138487 192.168.1.7:39225 -> 54.240.188.129:80 [AP]
  GET /images/I/51M7z0dUXDL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:53:19.140637 192.168.1.7:39971 -> 54.240.188.69:80 [AP]
  GET /images/I/51xRI9n2puL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:53:19.200223 192.168.1.7:56033 -> 216.137.35.219:80 [AP]
  GET /images/I/41QtRL2VlXL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:53:19.215688 192.168.1.7:44482 -> 98.142.98.180:80 [AP]
  GET /static/img/sleeveart/00/012/282/0001228244_17

T 2012/11/03 16:53:19.308043 192.168.1.7:34426 -> 216.137.35.57:80 [AP]
  GET /images/I/51MS7z8oHhL._SL160_.jpg HTTP/1.1..Ho

T 2012/11/03 16:53:19.313324 192.168.1.7:46245 -> 54.240.188.131:80 [AP]
  GET /images/I/51eO4S5QRiL._SL160_.jpg HTTP/1.1..Ho

i leave it to the reader to do the comparison, you'll see that there are /some/ differences between the two.. it might be due to my client string.. or some mixing function on the server, but you can see how scoring would quickly give you one or two candidate terms depending on how many matches you requre before calling it a candidate.. you can see how amazons algorithm for generating search results works for eve here... its was pretty easy to whip up a python tool for doing this using googles bad word list as a dictionary..

other side channel leakage:
since the search requests are "live" partial search results are provided - sometimes keystroke to keystroke for those that type slowly - an attacker who has a large enough database can use these intermediate results to narrow down subsequent result possibilities for increased accuracy

ProblemType: Bug
DistroRelease: Ubuntu 12.10
Package: unity-lens-shopping 6.8.0-0ubuntu1
ProcVersionSignature: Ubuntu 3.5.0-17.28-generic 3.5.5
Uname: Linux 3.5.0-17-generic x86_64
ApportVersion: 2.6.1-0ubuntu6
Architecture: amd64
Date: Sat Nov 3 16:21:42 2012
InstallationDate: Installed on 2012-11-01 (2 days ago)
InstallationMedia:

MarkForUpload: True
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: unity-lens-shopping
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
mike davis (mdavis-r) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in unity-lens-shopping (Ubuntu):
status: New → Confirmed
Revision history for this message
mike davis (mdavis-r) wrote : Re: [Bug 1074780] Re: lens searches can be unmasked by local network sniffing
Download full text (10.6 KiB)

This bug has been mislabeled as a dupe of 1074780 which calls out privacy settings. My bug is an implementation bug in which the lens fails to protect searches in the way it /attempts/ to by utilizing ssl for the request.

One bug is about you not respecting my privacy. The other is how you failed at implementing a feature.

-mike

On Nov 14, 2012, at 6:15 PM, "Benjamin Kerensa" <email address hidden> wrote:

> *** This bug is a duplicate of bug 1073114 ***
> https://bugs.launchpad.net/bugs/1073114
>
> ** This bug has been marked a duplicate of bug 1073114
> Shopping Lens Does Not Respect User Privacy
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1074780
>
> Title:
> lens searches can be unmasked by local network sniffing
>
> Status in “unity-lens-shopping” package in Ubuntu:
> Confirmed
>
> Bug description:
> first i want to say that the default nature of the amazon spam plugin
> really is a violation of the community trust, and I highly advocate
> the EFF's position on this plugin.. the user should have the choice
> *before* their information is reported to some entity on the
> internet..
>
> issue:
> while its true that the lens encrypts search queries to the productsearch.ubuntu.com server, the subsequent fetch of the image links within the search results and the algorithmicly generated nature of the results on the server allow a local network user to sniff the network for HTTP get requests to the ubuntu server to unmask either the exact search term, or a closely related terms of an ubuntu user.
>
>
> how this works in the real world:
> an "eve" precaches the search results using a word list and parses the json results and notes which and how many image results were provided for a particular word of interest.. "eve" then sniffs the network looking for bursts of image requests, the attacker then compares the block of image requests to the results that were cached earlier and and scores the results.
>
> the search term (or closely related search term) is then revealed
>
> an attacker can also choose to build the dictionary after the initial
> packet sniffing so long as the server cached contents havnt shifted
> significantly .. though it is likely the results would still me
> similar enough to score the results for a best fit.
>
>
> an example:
> "eve" has a database filled by requesting a list of interesting search terms, below is the query for "diapers":
>
> phar@thing:~/ubuntu curl https://productsearch.ubuntu.com/v1/search?q=diapers 2> /dev/null | grep ecx.images-amazon.com | grep SL160
> "http://ecx.images-amazon.com/images/I/41w92ZKCHBL._SL160_.jpg"
> "http://ecx.images-amazon.com/images/I/51xRI9n2puL._SL160_.jpg"
> "http://ecx.images-amazon.com/images/I/516o3TWAOBL._SL160_.jpg",
> "http://ecx.images-amazon.com/images/I/5197vs3wtvL._SL160_.jpg",
> "http://ecx.images-amazon.com/images/I/51UEzvC7X9L._SL160_.jpg",
> "http://ecx.images-amazon.com/images/I/51ZFlIGw0DL._SL160_.jpg",
> ...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.