remove duplicate images

[sophia233]

Provide a way to dedupe image across different sources.
E.g. if an image has been downloaded from http://www.wallpapers.net
then when it will cancel fetching the same image from wallhaven.cc

One solution is to store image md5. And if new download image same as pre-existed, delete it. it could be an optional option if use wants to enable it.

[Iain (igray78756)]

It would also be nice to permanently store these md5 signatures so that if you delete a photo, Variety can use the md5 to cancel fetching those as well.

I find myself deleting photos and then they re-download about a week later.

[Peter Levi (peterlevi)]

It is not as easy as it sounds. "Same" images are almost never exactly the same, so strict hashes like md5 are always different. Even if the image itself is exactly 100% the same (very very rare in practice), often there are differences in the metadata, again resulting in different file md5. Even Variety itself adds to the image metadata to know where it was downloaded from.

Special image hashes (phash, dhash, etc.) are used for finding images that are "similar", but these come with their own set of problems, e.g. ignoring colors, ignoring differences in "minimal" images with a flat background and just some minor icon in the center, etc. Calibrating these to work properly is a major pain.

[Advisor (master-of-shadows)]

Hi!

I create tag "duplicate" for this image: https://vrty.org/tag/duplicate

It allow remove/merge image for "hand".