Face detection / recognition

[Javier Jardón]

This feature is already present in other photo management software, like iPhoto or Picasa

It would be very cool if some free software get such a feature. Maybe it could be implemented as a general approach (so other project could use it)

[Christian Weiske (cweiske)]

Version: (using KDE KDE 3.5.5)
Installed from: Gentoo Packages

Reaching my personal 10.000 photos barrier in the next weeks, I face a problem:
Tagging photos by hand is not doable anymore since it takes too much time. While setting tags for a whole album (france, wedding, 2007) is easy and fast to do, tagging each of the 200+ photos in an album with the people on it is impossible.

When searching for photos, you often try to find a person - how it looked at a time, or generally need all photos of a certain person. Tags help here, but not all photos are tagged through the reason above. What I would like to see is a plugin that automatically detects faces and allows one to drag a tag from the tag tree on, or just add a tag to the detected face.

There are some face detection and recognition libraries out there that could be used, one well-known is the Intel's OpenCV lib [1]. libFace [2] is another one that specializes in face rec/det and uses intel's lib internally.

Gnome's F-Spot will get this feature trough the summer of code project [3, 4]. If it gets that, it'll be my personal killer feature to use fspot.

It would be very cool if digkam would get such a feature. And by the way, thanks for making digikam. My girlfriend and I use it since years, and it's been unvaluable for managing our photo collection.

References:
1 http://sourceforge.net/projects/opencvlibrary
2 http://libface.sourceforge.net/
3 http://idea.zanestate.edu/archives/2007/04/face-recognition-coming-to-f-spot/
4 http://bugzilla.gnome.org/show_bug.cgi?id=324391

Mailing list thread:
http://mail.kde.org/pipermail/digikam-users/2007-June/003642.html

[Stephan (stephanolbrich)]

*** This bug has been confirmed by popular vote. ***

[Caulier-gilles-9 (caulier-gilles-9)]

Hey, i have never seen a wish confirmed after 2 days than it have been open...

Marcel, what do you think about Face detection. For me it sound a killer feature...

Gilles Caulier

[Marcel-wiesweg (marcel-wiesweg)]

Yes it sounds very interesting.
We would need to find out what the workflow is (training of the face recognition?), and how well the library works.
Then it's about writing an easy to use wrapper class for the library.
And finally integrating it to digikam, perhaps in the image search or in some yet to be written tagging workflow?

[Caulier-gilles-9 (caulier-gilles-9)]

Marcel,

... or during downloading from camera...

Gilles

[Arnd-baecker (arnd-baecker)]

Well, this will be a long road until success:
I had a quick look at installing opencv (which fortunately exists
in debian etch as version 0.9.7-4).
Then libface needs some tweaking in the header files so that opencv is found,
but compiles fine after that. (I can post more details on that if needed).
Then one has to look in the examples directory of libface-0.1,
and compile
constructmat.cpp facedetect.cpp recognize.cpp train.cpp
However, it is not clear from the (essentially lacking, unless
I missed something) documentation how to use these:
Presumably it is train followed by recognize or facedetect.
For train one has to supply a file which contains names and
images of faces to be trained.
So far I only managed to get a segmentation fault...
(note to self: don't use my own pictures, but those of models,
the code seems to be picky ... ;-)

Then there is ellip-mask.jpg which provides a mask, maybe(?) used
for the input images. Whether the algorithm is able to detect
the region of interest with a face to be recognized is not clear to me.
Presumably all this will require some more digging in the original opencv
documentation.

Also note that the libface project does not appear that active,
after an initial check-in 6 weeks ago nothing happened...

OK, I will stop at this point, maybe someone else has more luck ...

[Caulier-gilles-9 (caulier-gilles-9)]

Arnd,

I'm not surprised by this feedback (:=))). I know very well the world of experimental algorithms study (i'm work in a French scientific research center)

With this experience, i'm always aware about it. It's always an instable world witch need a long time to be stabilized.

Also, if the project of this shared library is not really active, we need to wait is something will be done during Google Summer Code to improve this situation.

My viewpoint is this : Face Detection is an interressing feature of course, but we have more important task to do before KDE4/Qt4 or DB update port for ex. (:=)))

So we wait and see...

Gilles

[Arnd-baecker (arnd-baecker)]

> So we wait and see...

Yes, there is a lot more concrete stuff...
Apart from the technical side of the KDE4 transition,
there are also non-trivial workflow aspects
to be addressed wrt tagging (e.g
http://bugs.kde.org/show_bug.cgi?id=114465)
and improving searching/tag-filtering ...

[Christian Weiske (cweiske)]

What I understood so far about opencv:
- There are two things: Face detection and face recognition. The first is to actually detect the position and region of a face, or all the faces on the photo. The second is to recognize the face at the detected region and say who it is. This needs a training set.

The simple / first thing I think should be implemented is face detection and face outline display in digikam. The user OR a program would say where the the faces are (e.g. user could select a frame as done for selection now), and digikam would store this information somewhere in the database. Further, a tag shall be associatable to that selected region (that would not only allow faces, but other details to be selected - as flickr allows it). This way, you can use a person name/tag and tag photo regions.

The second is face recognition. With the things implemented in the first point, it should be possible to extract the faces/regions from already tagged photos and train the face recognitioner. Untagged regions could then be sent to the recognition engine/script and be tagged automatically after.

So in short: Digikam first needs a way to tag/describe regions of an image. I think rectangles should be fine, no need to make it complicated and support circles or even AND OR and whatnot. After this, even outside scripts can be used to facedetect and recognition.

[Arnd-baecker (arnd-baecker)]

My impression is that first the algorithmic side of both steps
- face detection
- face recognition
has to be working first (well, it did not for me in the first try...),
before the integration into digikam can be done.
All can be tested in detail separately
and this should then also clarify the needs on the digikam side.

Your suggestion to describe regions of an image and
associate those with a tag is indeed something of use
independent of the face detection.
This should have many applications (--> separate B.K.O entry?)

Best, Arnd

[Christian Weiske (cweiske)]

Arndt, I don't think so. OpenCV already provides a working algorithm set, so digikam needs the infrastructure to use the results of that algorithms - that is, mark a region of a photo as "face", and to say that this region/face is someone - tag it. Only then detection/recog are useful. You don't need fuel until your car has wheels.

[Arnd-baecker (arnd-baecker)]

So obviously I missed this then - there is a fully working code based on OpenCV
to which you supply images (eg. just containing the face part) to train
and then do the recognition for new images? Do you have a pointer?
But what is the role of libface then? Is it needed? Did you manage to
get it working?

[Christian Weiske (cweiske)]

Arnd, it's my fault. OpenCV is a lib that can be used if one wants to do face detect/recognition, and libface would be the one that could be used by digikam. But if it crashes regularly, it doesn't help. Are there other libs out there that could be used, or even a program via cmdline?

[Christian Weiske (cweiske)]

http://openbio.sourceforge.net/ is pretty old, but it has the cmdline programs

[Colin Guthrie (launchpad-colin)]

In Reply to Comment #9:
> Your suggestion to describe regions of an image and
> associate those with a tag is indeed something of use
> independent of the face detection.
> This should have many applications (--> separate B.K.O entry?)

Yes this would be very good. I think a new BKO is needed. From my point of view this would help for my Sync plugin (yeah I know I've made 0 progress lately!) when integrating with e.g. Facebook - Tags of regions to represent your friends etc. I'd imagine an extension of the current tags concept to store always a region (defaults to the whole image) of an item. Smaller regions can be defined that represent e.g. faces or other special notes of interest.

I can see how Face Detection could automate the assignment of these people tags.

I wonder however if such a capability could be included as a kipi plugin? What more would the API need to do to enable this. We are always discussing when we'll break the API anyway so perhaps the requirements of face detection should help mould the API changes?

Col

[Christian Weiske (cweiske)]

I opened a new bug #146337 for region tagging

[Colin Guthrie (launchpad-colin)]

To clarify: when I said:
> I wonder however if such a capability could be included as
> a kipi plugin? What more would the API need to do to enable
> this. We are always discussing when we'll break the API anyway
> so perhaps the requirements of face detection should help mould
> the API changes?

I was referring to the face detection stuff, rather than the region tagging stuff. Should be obvious if you think about it, but wanted to clarify :)

[Christian Weiske (cweiske)]

Two interesting links for face recognition:
http://www.iis.fraunhofer.de/bf/bv/kognitiv/biom/dd.jsp
(Fraunhofer Institut provides a demo application to detect faces. Perhaps someone could talk to them and convince them to open-source the algorithm? Page is in German)

http://www.riya.com
Is a visual search engine where you can upload your pictures, unfortunately they do not allow/provide offline use.

[Christian Weiske (cweiske)]

It seems the f-spot code is ready:
http://apart-dev.blogspot.com/

[Caulier-gilles-9 (caulier-gilles-9)]

ok, face detection is fun...

Christian, we will be busy on KDE4 for few months (XMP support + new Database), but i'm interressed to take a look later about this code...

So i will be happy to have more users feedback about this code especially statistics to detect the same face on a collection of picture (to apply a tag automaticly)...

Thanks in advance

Gilles

[Arnd-baecker (arnd-baecker)]

Just to add two more links about face-recognition (discussed in the German
magazine c't 13, pp 168, 2008):
http://ivt.sourceforge.net/
http://www.face-rec.org/

Ivt is a GPL library which employs a USB webcam (etc.)
for face detection and face recognition.
Internally it uses OpenCV, however I don't know whether anything
additional is contained which might be useful for our situation with still images.

[Caulier-gilles-9 (caulier-gilles-9)]

Arnd,

Andi, know OpenCV library. he develop a new kipi-plugin to fix automatically red-eyes. He can give more info about if necessary.

Gilles

[Michael Skiba (opensource-michael-skiba)]

Here's a collection of additional links to programs which deals with this topics - note however that this page is dated 2005. But maybe there's something valuable..
http://pages.cpsc.ucalgary.ca/~hanlen/vision/facelinks.html

[S-dem (s-dem)]

Version: (using KDE 3.5.9)
OS: Linux
Installed from: Gentoo Packages

It would be very nice if photo's could be automatically tagged by using face recognition techniques. All your photo's could be scanned for people, then grouped by similarity and tagged all at once. If already tagged people are found, they could be tagged without user intervention.

Would be very nice!

[Jan Kundrát (jktjkt)]

Got a library for face recognition?

[S-dem (s-dem)]

Maybe OpenCV: http://opencv.willowgarage.com/wiki/ ?

[S-dem (s-dem)]

I believe digikam already uses this library to perform automatic red-eye correction...

[Caulier-gilles-9 (caulier-gilles-9)]

This file is definitively for KDE4

Gilles Caulier

[Julien (julien-narboux)]

A page which explains how to use opencv for face detection: http://opencv.willowgarage.com/wiki/FaceDetection

[Andi-clemens (andi-clemens)]

Face Detection is not that hard, and there are ready-to-use classifiers as well in the OpenCV package.
But unfortunately those classifiers are only working very precisely when using for frontal faces. If the head is turned to the left or right or if hair hides some parts of the face, detection fails.

The much harder part is face recognition and I don't know how to make it learn the classifier that fast. As I trained the classifier for eyes (RemoveRedEyes plugin), it took two weeks to extract the features. The classifier isn't even that good, I might need to train another.

I really don't know how Picasa is going to do this.

Andi

[Andi-clemens (andi-clemens)]

(In reply to comment #25)
> I really don't know how Picasa is going to do this.

iPhoto I mean...

[Gaetan Semet (gaetan-xeberon)]

I do agree this could be a great addition to digikam software.
Currently, iPhoto and Picasa (web) are doing this.

- Go or you picasa web album and click on "Person" tab.
- all faces are detected (I suspect a script is run periodically to update the cache).
- you access to a serie of batch of photo of the same person and you just enter his/her name.

I would suggest, for this feature to be as killing as possible:
- First, perform a face detection on each photo in the entier library or a smaller part. Then, sort and display first the photos on the person on the most shots. This is psychologically important and have a strong impact since the user will think "whow! it is able to find all the photo with my mum, whoaa!!!". There is also lot of single photo (faces too different), but displayed last it have less impact and this can be entered latter
- of course if the scan already have been performed, the result should be cached/saved somewhere so we don't have to perform a new scan each time.
- In each batch of photo, ask to enter his/her name and tag the photo accordingly (please, use metadata tag written in the file itself, accordingly to a rule like "Persons/Name of the person")
- When adding photo, automatically performs face detection and display the most relevant name, or allow to enter his/her name

[8pp-u26-gj5 (8pp-u26-gj5)]

This is a real killer feature and I urge including it into digikam.

[Alexander-balzer (alexander-balzer)]

Fspot has it, Picasa has it already, iPhoto has it now.
When digikam is last in implementing users will switch.
I think this is a feature people would switch for!

[Stéphane List (slist)]

Hi,

You can find an example of howto integrate opencv mechanism into a QT application on my new webpage :
http://slist.lilotux.net/linux/qmotion/

I'm currently working on a similar project for face detection and face recognition.

Don't hesitate to get in touch with me for more informations.

Stephane

[Andi-clemens (andi-clemens)]

We are already using OpenCV in one of the KIPI plugins.
Face detection in general should not be the problem, but recognition seems to be hard.
What do you plan to solve this problem?

If you don't want to post here, you can sent my a private mail. I used OpenCV for my diploma thesis, it's been a while but I think at the moment I'm one of the devs with the most experience in OpenCV.

Andi

[Caulier-gilles-9 (caulier-gilles-9)]

To all,

This is my vision for Faces auto tagging...

If you use Facebook, you can mark people face over images. this is simple and great. I think it's the first step to do: added a small to tool to mark face manually in digiKam. For this stage, i know somebody in Mandriva who have work on this subject...

The second step, is to implement a tool to use current faces database done manually to be abble to find automatically similar faces over the whole collection of image. This can be done also during image downloading from camera...

For this second step, OpenCv can be used to parse image contents and detect face. Of course we need another algorthim somewhere to create faces fingerprints and compare faces one by one. For me it's sond similar than Haar stuff, but using more precision and details.

Perhaps, OpenCv provide something like this. Here i don't know.

What do you think about ?

Gilles

[Leshiy (alexjironkin)]

Face Detection is easy, OpenCV does this using Haar filter. Face recognition on the other hand is very very very hard. I am pretty positive OpenCV doesn't have this feature, of face recognition, yet.

Although if anybody know of any other non proprietary library that provides that support, would be interesting to have a look at that.

[Leshiy (alexjironkin)]

I had a look around the internet for something suitable, first of all apparently IPhoto 9, I think, has this face recognition technology. But I guess Apple just bought the company that developed it originally, because I just can't find anything about it on the internet.

With regards to open source equivalents, there is libFace, http://libface.sourceforge.net/index.html, but that is in preAlpha and doesn't look it has been active for a while. Other then that, might have to write own code for it. One of the ideas is Eigen Face.

[Alexander-balzer (alexander-balzer)]

There is also a feature request for digikam
https://bugs.kde.org/show_bug.cgi?id=146288

Maybe it could be implemented as a general approach (so other project could use it)

[Leshiy (alexjironkin)]

A quick update on the status of things. I finally got KDE SVN account but i can't check anything out with my username :(, but working on it.

Trying to get some integration with digikam now. I am not sure where to put it in though. I have looked through the code and if I understand it right, then I want to put it some where before the image preview is displayed but after it is loaded. Are all images after they have been loaded with custom loader converted to QImage and then displayed?

Alex

[Leshiy (alexjironkin)]

Also a quick question, does digikam use some QCanvas to display images?

The reason is that to annotate image with tags, one doesn't want to draw rectangles and so o on the image itself, but have another layer of widgets on top, for example for face recognition draw a box around and have a label below, which can be edited etc. on a layer above. I don't know if canvas has an ability to do this, unless it can be done differently.

The idea is somewhat similar to panes in Java:
http://java.sun.com/docs/books/tutorial/uiswing/components/rootpane.html

Alex

[Marcel-wiesweg (marcel-wiesweg)]

Alex, we dont have the infrastructure yet you are talking about here.
Michael has written a proposal how to store tags on image regions, but that is not yet implemented. We also dont have any UI, not at all.

We currently do not use a canvas like QGraphicsView. There is related work to do that in the image editor.
We also dont use QImage, but a custom image container...

What part exactly do you intend to test? Is the workflow fully implemented?
> digikam library -> use haarfeatures to detect faces on image -> use face
> recognition on face identified by classifier -> output to digikam to confirm

I imagine there must be a widget to show boxes (autodetected) and to allow to add boxes (manually), then associate a box with a tag (with a tag, a person, and a face identifier).
Should this for starting be done in a separate window? It would be easier then to play with the UI.

Where and how do you store the results of learning?

[Leshiy (alexjironkin)]

I have left the storing of the data to the people that want to use it as far as the library is concerned. In this case, I think it would be appropriate to store in some flat file somewhere, is it is constantly used and changed. Purely because some concerns over storing it in database (see above). The data itself are matrices and vectors of doubles.

Why I have brought it up, pretty much any of the face recognition has to be supervised (decided by a person whether it is match or not), that is why there is a need for a feedback in the process.

I am going to write a small example where library is used and hopefully outputted for feedback. I think then the overall process will become a lot more clear.

P.S. I think I will try and merge with the existing libface library, last I have looked at it on sourceforge, it was unstable and not maintained anymore for a couple of years. This was the reason why I went to write one in the first place.

Alex

[Caulier-gilles-9 (caulier-gilles-9)]

Created attachment 39728
people tag widget screenshot.

At this place from KDE repository:

http://websvn.kde.org/trunk/playground/base/nepomuk-kde/peopletag/

You will find nepomuk people tags widget. This is exactly what's you need as widget, we don't care about the rest of nepomuk to implement face detection GUI.

Gilles Caulier

[Nicolas Réau (kolia)]

@Javier:

Some people have started working on it: https://bugs.kde.org/show_bug.cgi?id=146288 (related to Digikam)

[Mathieu Comandon (strycore)]

Thank you for taking the time to make Ubuntu better. Since what you submitted is not really a bug, or a problem, but rather an idea to improve Ubuntu, you are invited to post your idea in Ubuntu Brainstorm at http://brainstorm.ubuntu.com/ where it can be discussed, voted by the community and reviewed by developers. Thanks for taking the time to share your opinion!

[Javier Jardón (jjardon)]

Hello Mathieu,

Maybe this is not a bug for Ubuntu packages, but It's already a bug for upstream projects, like digikam or F-Spot.
They also have some work done related to this.
See the upstream bugs.

Best regards

[Nicolas Réau (kolia)]

@Javier: This is not a bug neither upstream. if you check the report I was talking about you will see:

Severity: wishlist

This is a popular wish, and indeed people are working on it, but this is not a bug since it doesn't exist yet :)

[Javier Jardón (jjardon)]

Nicolas,

Yeah, It's a bug with wishlist severity ;)

[Leshiy (alexjironkin)]

UPDATE:

I was looking at peopletag project but had problem with it compiling altogether.
I am going to try and get in touch with the developer to add to the people tagger ability to recognise faces through the libface library.

I also will be merging what there is to the libface project on sourceforge, as I think it is the most appropriate place for it.

Progress has been slow as I have a little too much work on my hands lately :( Sorry

Alex

[Caulier-gilles-9 (caulier-gilles-9)]

ALex,

If you need help...

Gilles Caulier

[Alexander-balzer (alexander-balzer)]

@Alex
I have tags of 250 people in ~ 5000 images in digikam. So if you need somebody to test your work. Feel free to contact me.

[7-jm-b (7-jm-b)]

Have you thought about tag storage format ? I have not read any mention of that in this thread, so I suggest using the Fotonotes XML format [1] which is as close as a common image location tagging format as we have. Flickr implementing a derivative of Photonotes [2] is endorsement enough that Fotonotes has the potential.

[1] http://www.fotonotes.net/fotonotexml.html
[2] http://www.fotonotes.net/#installations

The best thing place to store that data would be in the JPEG header. That way, the embedded location tagging metadata would follow the image wherever it goes, whatever the tool used. I can very well imagine the Gallery Tagface module using that information.

[Mfpl (mfpl-deactivatedaccount)]

(In reply to comment #92)
> The best thing place to store that data would be in the JPEG header.

Are there any default places defined for other image formats like PNG, TIFF, etc?

Michael

[7-jm-b (7-jm-b)]

(In reply to comment #93)
> Are there any default places defined for other image formats like PNG, TIFF,
> etc?

A great starting point is the part 3 of the XMP Specifications - "Storage in Files" : http://www.adobe.com/devnet/xmp/pdfs/XMPSpecificationPart3.pdf pages 15 to 25. It provides information about how serialized XMP metadata is packaged into XMP Packets and embedded in different file formats.

Like Fotonotes, XMP is formatted with XML so embedding it into image files is exactly the same problem. But no Fotonotes library is in sight.

Sorry for the lack of proofreading in my preceding message - I shouldn't have posted while I was in a hurry to leave...

[Aditya Bhatt (adityabhatt1991)]

Hi Alex,
I've been playing with openCV for a few months now. I'm quite familiar with Qt too. I'd like to help and get involved with digikam. I'd like to work on the face recognition thingy. I'm still new to bugzilla and svn, but I'll learn soon enough.

[1] My opinion is that the haar cascades bundled with openCV seem quite good, and using multiple types of cascades for different poses might be a good idea. I think one can call the face detector function with very permissive arguments so that almost all faces and some nonfaces are detected, and then apply some of one's own tricks to discard the nonfaces.

[2] Another thing I'd like to mention is - a guy named Rohan Anil has made a PAM face authentication module in the previous two GSOC's for KDE. I've checked his app and the detection part is quite good. In fact, close to perfect.

The catch is that Rohan's app is supposed to detect only ONE face in an image. I think that the previous thing I mentioned in [1] can be used to grab anything that looks remotely like a face and feed it to [2].

His face recognition is something I haven't tested much, but I think it would be, well, *security-level* stuff.

The PAM Module's link is here : http://code.google.com/p/pam-face-authentication/

My bitbucket page : http://bitbucket.org/aditya_bhatt/

Please contact me at : adityabhatt1991 at gmail dot com

Regards,
Aditya

[Leshiy (alexjironkin)]

@Aditya,
I had a look at the "paper" they have published recently and the module itself and not particularly convinced. For example, the system doesn't learn, which means it is limited to the original raining data. As I understand it, this modules uses eyes for face tracking and authentication. There is no guarantee what the eyes will be like in a photo, e.g. an eye is covered tilted rotated and occluded.

The module has high rejection rate for security reasons, and that is something is opposite to what we want here. In practical aspects it builds dxd matrix somewhere inside where d is number of pixels in an image. Take an average 7M pixels image and make that a square. What you get is something unfeasable. Or at the very least needs a lot to optimise. Where as working off a 320x220 image from webcam is a different story. It makes no mention on the limitations of the system, rotation in both x-y plane and z axis is not mentioned as well as the image intensity invariance. What is the limit on how big a face it can detect? In the demo they have on website it only detected image when tester moved closer.

Based on what I have just said and more I am very skeptical about this module. However, having said this I think there are some worth while ideas there like MACE filter and Local Binary Pattern. Perhaps interesting things can also be done with eye detection.

I don't think there is a single system that will do the trick, but instead a number of systems for face detection and recognition put together, working together to reinforce one another will be the best thing in the end.

I don't want to discourage you from trying this approach, so if you write the class based on what they have used, one for face detection and/or one for face recognition feel free to send it to me and I will try to integrate into the library.

By the way since the last post I managed to upload the library to libface (on sourceforge) and this is where it is going to be hosted permanently now. Feel free to download the library and compile it, but be warned the work is still very much in progress :)

Alex

[rohananil (rohan-anil)]

I dont think MACE Filter would be the way to go. The module i am working on is pam-face-authentication is for authentication that is 1 vs 1 matching. For the design - i had to choose between security vs speed (real-time) vs usability. So adapting it for recognition is out of question. So I dont think its fair to compare it for use in recognition :). And question about face detection - its an implementation available on OpenCV based on a method by Viola and Jones. If you want the performance numbers - http://www.lienhart.de/. He has a paper on it. Also again pam-face-authentication is for realtime authentication - So you have to provide your frontal face to the webcam! :) . I have bigger plans for the authentication system once i perfect my AAM implementation http://code.google.com/p/aam-opencv/, Currently the issue is some manual supervision is required which is a NO-GO with a normal user. It just has to work out of the box. Although the results from AAM-Tracking to Recognition is just phenomenal.

Now coming to the task at hand. We are looking at the topic of unconstrained face recognition. Best - research for our photo tagging - could be to implement the best performing algorithm from http://vis-www.cs.umass.edu/lfw/ results. We could go with a support vector machine implementation with highly efficient features. But bottle-neck is the time required for training. Can we compromise over training time ? The more pictures you have more training time. Now we should pickup an algorithm which does not require retraining over the entire set for new images. I think i can code this over this/next weekend. But need help with integrating with the code base. If anyone could provide me an api spec of how the class should be like I would like a go at it..

regards
Rohan

[Leshiy (alexjironkin)]

I really like the idea of http://vis-www.cs.umass.edu/lfw/ Has a lot of useful information, including ROC curves. :)

I am going to create a mailing list for libface so we can discuss the finer points of face recognition and detection there as opposed to boring people here and leave this thread for digikam/Face recognition issues :)

Alex

[Aditya Bhatt (adityabhatt1991)]

Hi,

I'm Aditya, a 2nd year student doing my Bachelors in Information and Communication Tech.
My favorite fields of work are pattern recognition, image processing and computational math.
I'm mostly self-taught in image processing, and also have some knowledge of neural networks. I'm familiar with PCA for dimensionality reduction.

I'm familiar with svn, git and hg version control.
I have some familiarity with clustering algorithms and eigenfaces. Fisherfaces, I'm learning.

I know how to work with Qt fairly well. I also have a very basic knowledge of KDE libraries - which I hope to improve as soon as my exams (a week later) end.

I've been working with Alex Jironkin on his libface project on sourceforge for a while now, and would like to continue working on it over the summer as part of GSoC 2010. I also want to start digiKam hacking soon to integrate this for tagging as described in the GSoC ideas page. I'm already working on a people tagging widget of my own, although I plan to scrap it and start working on the widget in KDE's svn repo as soon as svn:// port gets unblocked in my college :'(. ( I currently commit to libface over https:// )

I'd love it if Alex can be my mentor. And I'd also like the mentorship of one of the digiKam core devs for integrating this into the digiKam interface, and it's tagging database. It'd be nice to get this merged into digiKam before the end of the summer, and I'd like to work on digiKam too in the future - so I consider this a stepping stone.

Post-GSoC, I'd like to work on KIPI and digiKam's "fuzzy" algorithms, as that is my field of interest.

I look forward to your replies soon :)

Regards,
Aditya

[Kde-2011-08 (kde-2011-08)]

It seems that KDE did not get a spot in GSOC this year:
http://socghop.appspot.com/gsoc/program/accepted_orgs/google/gsoc2010

[Arnd-baecker (arnd-baecker)]

> It seems that KDE did not get a spot in GSOC this year:
> http://socghop.appspot.com/gsoc/program/accepted_orgs/google/gsoc2010

Sure? I see KDE listed there (in the second part).

[Leshiy (alexjironkin)]

I was going to say, that it would be very suprising if they didn't. I can see them in organisations completed profile lower down the list too.

[Kde-2011-08 (kde-2011-08)]

> (in the second part)

Second part?!? Who gets that far?

Sorry for the mistake, I'll go away quietly now...

[Nareshsankapelly (nareshsankapelly)]

hi,

I am a student pursuing B.Tech 4th year from Indian Institute of Information Technology- Allahabad, India.

I am keen to get involved in open source project development. I have seen on KDE website the project ideas for GSoC 2010. I would like to work on Face Recognition Project. I took this project because I have worked on the hand gesture recognition using OpenCV and am currently working on the same area.You can have a look at my work here.

http://www.youtube.com/watch?v=wnKk07-YNFw&feature=player_embedded

I am having good knowledge of C++, Qt and the classification algorithms.

I would like to know some more details of the project.

I would be happy to know answers to these following questions.
1. what classification algorithm is to be refined for face detection part. OpenCV provides Haar Classifier using Ada boost learning and it works with almost 99% accuracy. Is it the same algorithm that is to be refined or something else?
2. If it is Haar classifier to be refined. In which part it is to be refined haartraining utility or haar detection part ?

Hoping for a positive reply from you as soon as possible.

Thanks
Naresh Reddy S.
IIIT-Allahabad

[Leshiy (alexjironkin)]

Haar training is not something of interest, because it doesn't provide ability to update on the fly. Unless you can think of a way that can be done because haartraining learns from scratch everytime. So you need to write updater from scratch and do some clever hacking of retrieving previous states from learning.

So we are interested in refining face detection first. When you say it is 99% good, it also has high rate of Type II errors, identifying something as a face when its not, which is bad. That is what needs refining. This problem arises when you use real photos, because need to specify feature size in the identification.

Or you can think of a better face detection mechanism

Alex

[Leshiy (alexjironkin)]

I seem to be keep shooting people down with ideas. :( Sorry.

I think we got face detection to work fine now, the area where we need help and fresh ideas is face recognition. So if you have idea or even better an algorithm for face recognition feel free to post it to me or into libface mailing list and we will get on it to be implemented :)

I guess another area would be deeper integration into KDE itself, like with contacts, I guess this is also can be done in digikam.

Alex

[Johannes Wienke (languitar)]

At my university we use active appearance models to recognize faces. This approach needs a lot of training data to set up good models but should then scale much better then eg. Eigenfaces. Unfortunately I can only find the diploma thesis about this which is in German:
http://www.airesearch.techfak.uni-bielefeld.de/files/Lang2007-DIP.pdf

[Johannes Wienke (languitar)]

And a few more links:
http://www.springerlink.com/content/tyy00w31extww8p1/
http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V09-3Y4510K-1B&_user=10&_coverDate=06%2F30%2F1995&_rdoc=1&_fmt=high&_orig=search&_sort=d&_docanchor=&view=c&_searchStrId=1263077754&_rerunOrigin=scholar.google&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=c9834a5a69604f23126bf53e06e6655c

[Leshiy (alexjironkin)]

Does the whole thing needs to be retrained everytime you want to add more data or can it just be updated.

One of the problems with using Haar features for face recognition now, is because you also need a large training set, and you can't easily add more data to the cascade, in other words it has to be retrained from scratch everytime, which is a big problem.

Also bear in mind this is supervised learning, so there is a reliable input from people to tell what is right and wrong. So only correct data is added to the classifier/cascade.

Thanks for the idea though, would be interesting to have a look at that at some point.

Alex

[Johannes Wienke (languitar)]

This depends on the classifier you want use. AAM only provides the feature vector. The solution Christian used was a two step procedure:
1. Nearest Neighbour for rejection (trivial update ;) )
2. Classification based on Gaussian Mixture Models. There will certainly exists enough algorithms to update them with new samples.

[Leshiy (alexjironkin)]

    Accepted projects for GSOC 2010 were announced last night. I am also happy to say that one of the projects accepted was face recognition project. :)

    Adytia will be doing this project and hopefuly we will have a fully working plugin by the en of the summer.

Alex

[Caulier-gilles-9 (caulier-gilles-9)]

Great that it have been accepted. I'm very impatient to test this feature.

I talk by IRC with Adytia this week end about face detection, and making a plugin is not the good way. It's better to patch digiKam as well, to be able to work directly with digiKam DB. I will attached the conversation to this file, for details.

I plan to organize a coding sprint for digiKam at end of summer in France, Aix en provence. It wil be nice to see libface developpers join us at this event. Note : KDE-ev will sponsorship the event (travel + hotel payed).

Gilles Caulier

[Caulier-gilles-9 (caulier-gilles-9)]

digiKam IRC talk between me and Aditya Bhatt about face detection.

[19:55] <cauliergilles> Adityab: ping ?
[19:55] <Adityab> cauliergilles: Yes?
[19:56] <cauliergilles> Adityab: hi. Just 2 questions
[19:56] <Adityab> cauliergilles: sure
[19:56] <cauliergilles> Adityab: 1. Do you recieve your svn account as
developper ?
[19:57] <cauliergilles> Adityab: 2. in digikam-devel mailing list, do
you recieve my mail about digiKam database problem today (i'm not sure
if this message is passed right because it's long)
[19:57] <Adityab> cauliergilles: No, I'm applying tomorrow. They give
higher priority to GSoC students. And even if I don't get into GSoC,
I'm still applying.
[19:57] <cauliergilles> Adityab: ok. let's me hear if you have some pb
about. I can contact right admin accordingly
[19:58] <cauliergilles> Adityab: and don't forget to add you in coding
sprint oo.org document from svn.
[19:59] <cauliergilles> Adityab: I will need quickly all info from
contributors to book hotel and restaurant at Aix en Provence
[19:59] <Adityab> cauliergilles: Thank you :) Yes, I received the mail
and I read it, but I didn't try to analyse everything as my final
exams are from tomorrow
[20:00] <Adityab> cauliergilles: Yes, I'll update the document soon.
I'll be renewing my passport also, and that will take a max of one
month.
[20:01] <cauliergilles> Adityab: ok. no problem. I just want to be
sure if mail is posted right through KDE server. Normally Marcel must
respond about
[20:01] <cauliergilles> Adityab: great.
[20:01] <cauliergilles> Adityab: currently, i don't share a lots of
time with GoSC, because i change my home, and i very busy.
[20:02] <Adityab> cauliergilles: That's okay :)
[20:02] <cauliergilles> Adityab: i also, contribute a lots in Exiv2
project to finalize Sony Makernotes support.
[20:02] <Adityab> cauliergilles: Thanks :) If you can act as my
referrer to the sysadmins, I'll apply just now.
[20:02] <cauliergilles> Adityab: i'm also teacher at Aix en provence,
in a computer science school
[20:03] <cauliergilles> Adityab: so my free time is limited... (:=)))
[20:03] <Adityab> cauliergilles: Must be a hectic job :)
[20:03] <cauliergilles> Adityab: but i will switch indeep to digiKam
and follow GoSC more seriously in few week
[20:04] <cauliergilles> Adityab: i'm sure that Marcel will follow you
properly. He is serious developer
[20:04] <Adityab> cauliergilles: Yes, he is very helpful. He replied
to my every query
[20:04] <cauliergilles> Adityab: Anyway, if you have question or a
problem don't hesitate to mail me.
[20:05] <cauliergilles> Adityab: i'm not online frequently in this
channel, due my external job
[20:05] <cauliergilles> Adityab: but i read my mail each day
[20:05] <Adityab> cauliergilles: Sure, thanks :)
[20:06] <cauliergilles> Adityab: yesterday, i do a presentation of
digiKam in a LUG at Aix en provence
[20:07] <Adityab> Yes, I saw your update on facebook :)
[20:07] <cauliergilles> Adityab: ah yes.
[20:08] <cauliergilles> Adityab: and a lots of people ask me if face
detection will be implemented soon, to provide the same feature than
Picasa
[20:08] <cauliergilles> Adityab: i said ... yes of course
[20:08] <cauliergilles> Adityab: so people a...

[Caulier-gilles-9 (caulier-gilles-9)]

Implemented in digiKam 2.0.0

Gilles Caulier

[Simon-44ecklpsr (simon-44ecklpsr)]

yay, party on!

[Andreas Brauchli (blk)]

fix upstream bug duplicate

[Guillaume Hain (zedtux)]

I already give it a try with my QuiEstLa project: https://launchpad.net/quiestla.

I did it for Shotwell in this way: Provide an OpenCV client embedded into a D-Bus application.
My project is working, and need some review / optimizations.

It's a client/server application.
I already described how to start it and use the client application to test it.

I'm open for pull requests / ideas / suggestions / code reviews.

[Isilmendil (isilmendil)]

*** This bug has been marked as a duplicate of bug 292900 ***