Fun with Python, OpenCV and face detection

I had some fun with Gary Bishop’s OpenCV Python wrapper this morning. I wanted to try out OpenCV for detecting faces using a web cam. This could be used for instance to see if someone is sitting behind his desk or not. I used Gary’s Python wrapper since I didn’t want to code in C++.

I didn’t know where to start, so I searched for existing OpenCV face detection examples. I found a blog post by Nirav Patel explaining how to use OpenCV’s official Python bindings to perform face detection. Nirav will be working on a webcam module for Pygame for the Google Summer of Code.

I managed to rewrite Nirav’s example to get it working with CVtypes:

Here’s the code. Although it’s just a quick and dirty hack, it might be useful to others. It requires CVtypes and OpenCV, and was tested on Ubuntu Hardy with a Logitech QuickCam Communicate Deluxe webcam. You will need Nirav’s Haar cascade file as well.

import sys
from CVtypes import cv
 
def detect(image):
    image_size = cv.GetSize(image)
 
    # create grayscale version
    grayscale = cv.CreateImage(image_size, 8, 1)
    cv.CvtColor(image, grayscale, cv.BGR2GRAY)
 
    # create storage
    storage = cv.CreateMemStorage(0)
    cv.ClearMemStorage(storage)
 
    # equalize histogram
    cv.EqualizeHist(grayscale, grayscale)
 
    # detect objects
    cascade = cv.LoadHaarClassifierCascade('haarcascade_frontalface_alt.xml', cv.Size(1,1))
    faces = cv.HaarDetectObjects(grayscale, cascade, storage, 1.2, 2, cv.HAAR_DO_CANNY_PRUNING, cv.Size(50, 50))
 
    if faces:
        print 'face detected!'
        for i in faces:
            cv.Rectangle(image, cv.Point( int(i.x), int(i.y)),
                         cv.Point(int(i.x + i.width), int(i.y + i.height)),
                         cv.RGB(0, 255, 0), 3, 8, 0)
 
if __name__ == "__main__":
    print "OpenCV version: %s (%d, %d, %d)" % (cv.VERSION,
                                               cv.MAJOR_VERSION,
                                               cv.MINOR_VERSION,
                                               cv.SUBMINOR_VERSION)
 
    print "Press ESC to exit ..."
 
    # create windows
    cv.NamedWindow('Camera', cv.WINDOW_AUTOSIZE)
 
    # create capture device
    device = 0 # assume we want first device
    capture = cv.CreateCameraCapture(0)
    cv.SetCaptureProperty(capture, cv.CAP_PROP_FRAME_WIDTH, 640)
    cv.SetCaptureProperty(capture, cv.CAP_PROP_FRAME_HEIGHT, 480)    
 
    # check if capture device is OK
    if not capture:
        print "Error opening capture device"
        sys.exit(1)
 
    while 1:
        # do forever
 
        # capture the current frame
        frame = cv.QueryFrame(capture)
        if frame is None:
            break
 
        # mirror
        cv.Flip(frame, None, 1)
 
        # face detection
        detect(frame)
 
        # display webcam image
        cv.ShowImage('Camera', frame)
 
        # handle events
        k = cv.WaitKey(10)
 
        if k == 0x1b: # ESC
            print 'ESC pressed. Exiting ...'
            break

A known problem is that pressing the escape key doesn’t quit the program. Might be something wrong in my use of the cv.WaitKey function. Meanwhile you can just use Ctrl+C. All in all, the face detection works pretty well. It doesn’t recognize multiple faces yet, but that might be due to the training data. It would be interesting to experiment with OpenCV’s support for eye tracking in the future.

Update: the script does recognize multiple faces in a frame. Yesterday when Alex stood at my desk, it recognized his face as well. I think it didn’t work before because I used cv.Size(100, 100) for the last parameter of cv.HaarDetectObjects instead of cv.Size(50, 50). This parameter indicates the minimum face size (in pixels). When people were standing around my desk, they were usually farther away from the camera. Their face was then probably smaller than 100×100 pixels.

Just a quick note on ctypes. I remember when I created PydgetRFID that I tried to use libphidgets’ SWIG-generated Python bindings, but couldn’t get them to work properly. I had read about ctypes, and decided to use it for creating my own wrapper around libphidgets. Within a few hours I had a working prototype. When you’re struggling with SWIG-generated Python bindings, or have some C library without bindings that you would like to use, give ctypes a try. Gary Bishop wrote about a couple of interesting ctypes tricks to make the process easier.

Tags: , , , , , ,

View Comments to “Fun with Python, OpenCV and face detection”

  1. In Traction » Blog Archive » Small update on face detection post Says:

    [...] Blog « Fun with Python, OpenCV and face detection [...]

  2. Si Says:

    excellent stuff. Many thanks

  3. Jo Vermeulen Says:

    You're welcome! I'm glad you find it useful.

  4. Nirav Patel Says:

    Just to clarify, the Haar cascade file is one of the sample ones that comes with OpenCV. The script I wrote was based on the OpenCV face detection sample too, but using gstreamer instead of OpenCV's HighGUI to interface the webcam (HighGUI didn't support the camera in the XO).

    Its good that you got it working though. There is some pretty amazing stuff in OpenCV.

  5. Jo Vermeulen Says:

    You are right, I noticed yesterday that the cascade file was part of OpenCV. Thanks for clarifying!

  6. daxroc Says:

    Nice script , worked fine for me @30fps

  7. Jhon Says:

    I figured it out how to properly use cvWaitKey.

    In your code you are waiting just 10 ms for a key, and only after the image was processed.
    So, if you wait a little bit longer(maybe 100ms, but this will slow down a lot the fps) you will be able to catch the keys.

    However, there is also another problem cvWaitKey returns -1 when no key was pressed, but return a string(char) holding the key pressed, so you need to do the following:
    if k!=-1 and ord(k) == 0x1b: # ESC
    print 'ESC pressed. Exiting …'
    break

  8. Jo Vermeulen Says:

    Ah thanks, that's very helpful!

  9. randomm Says:

    this is great stuff! thanks for this…

    We managed to get up to 4 faces recognized at the same time.

  10. Jo Vermeulen Says:

    You're welcome, I'm glad it was useful for you.

  11. Playing with VideoCapture on Windows « Coding AI Says:

    [...] detection, I just found that OpenCV has camera capture program built in. Somebody has been playing (Fun with Python, OpenCV and face detection) with it for a while. His approach is to use a OpenCV python wrapper – CVtypes , the premise is to [...]

  12. Venkat Says:

    When I tried runing the above code in windows xp, encountered the following error.

    D:SWCamFD>python test2.py
    Traceback (most recent call last):
    File “test2.py”, line 2, in <module>
    from CVtypes import cv
    File “D:SWCamFDCVtypes.py”, line 42, in <module>
    _cxDLL = cdll.cxcore100
    File “C:Python25libctypes__init__.py”, line 423, in getattr
    dll = self._dlltype(name)
    File “C:Python25libctypes__init__.py”, line 348, in init
    self._handle = _dlopen(self._name, mode)
    WindowsError: [Error 126] The specified module could not be found

    Any help regarding this will be great!

  13. Jo Vermeulen Says:

    You need to have OpenCV installed. I guess that is the problem. I didn't try it on Windows yet since there's no D-BUS support, but in theory the face detection code should work.

    I will probably release a first version of the code in the following weeks, so stay tuned!

  14. Simon Says:

    Hello, Sir
    I'm from Asia. I'm just starting to learn openCV.
    Is there any opinion or advise from Sir as an expert to me to start learning opencv

    Thank you very much.

  15. Face Detection in Static Images with Python Says:

    [...] it seems that most OpenCV face detector examples are meant to be run “live”, usually taking the image from a webcam and highlighting [...]

  16. Ecommerce Web Design Says:

    WaitKey function. Meanwhile you can just use Ctrl+C. All in all, the face detection works pretty well. It doesn’t recognize multiple faces yet, but that might be due to the training data.

  17. How Do You Look When Merging Fails ;-) « Andi Albrecht Says:

    [...] the way how to access the camera is inspired by this nice blog post about face recognition using [...]

  18. affordabletermquote Says:

    I am into doing same stuff but sad to say I'm still at the first phase of learning. I look forward to being like you so I can also share my know how for others to use.

  19. uuganbaatar Says:

    how is running? fast or not? I think opencv is universal thing… if u write your own codes it will be faster than opencv because there too many functions and processes unneedful… so goodluck!!!

  20. gabox Says:

    hi i want to put the window's webcam in other program both under python, but i dont how to do it, the project is a program with bottons and they controll a robot and the robot get the webcam, by the way im using linux mint 6

  21. nop Says:

    Hello!
    Your if script silently hangs on this line for me:
    cascade = cv.LoadHaarClassifierCascade('haarcascade_frontalface_alt.xml', cv.Size(1,1))

    What can be the problem here? This xml exists in same directory and the is no any error output.

    Thanks in advance!

  22. Jo Vermeulen Says:

    Are you using Ubuntu?

  23. Jo Vermeulen Says:

    It might be good to use a specific GUI toolkit in this case, such as WxWindows.

    Have a look at Gary Bishop's blog post, where he grabs a bitmap from the camera and displays this in a Wx window: http://wwwx.cs.unc.edu/~gb/wp/blog/2007/02/04/p…;

  24. ClubPenguinCheats Says:

    The script I wrote was based on the OpenCV face detection sample too, but using gstreamer instead of OpenCV's HighGUI to interface the webcam (HighGUI didn't support the camera in the XO).

  25. sreeraj Says:

    hi, i'm having this problem when i'm running your code..

    i have an integrated camera on my laptop, which works with some apps like cheese.

    while running this code, the camera(led) turns on, but nothing comes on the screen.
    the script hangs and i have to ctrl+C to close it.

    any pointers to help me with this situation will be appreciated.
    thanks

  26. cyril Says:

    Hello, i try your code but i have this:

    cyril@zilo2:~/Bureau/test$ sudo python webcam2.py
    Traceback (most recent call last):
    File “webcam2.py”, line 3, in <module>
    from CVtypes import cv
    ImportError: No module named CVtypes

    i use ubuntu, how cab i install opencv? sudo apt-get install ?

    Thinks @+

  27. nop Says:

    yes, 9.04

  28. nop Says:

    In synaptic you can fing python-opencv

  29. cyril84 Says:

    Hello, i have this:

    Traceback (most recent call last):
    File “opencv.py”, line 2, in <module>
    from CVtypes import cv
    ImportError: No module named CVtypes

    What is this mistake? how install cvtypes?

    Thinks

  30. cyril84 Says:

    I have do sudo apt-get install python-opencv. It is already installed. What can i do?

  31. online games Says:

    I'm still having trouble with the escape key. Any tips?

  32. Jo Vermeulen Says:

    The script uses Gary Bishop's CVtypes, not python-opencv.

  33. In Traction » Blog Archive » Facade update: code available and information about related student projects Says:

    [...] previous posts on face detection with Python and on changing the user’s presence when he/she is detected by the webcam are two of the most [...]

  34. club penguin cheats Says:

    this is awesome, i have been wanting to do this with python for awhile now. great stuff.

  35. njay Says:

    hey!! i am nj here, i am trying to execute this program on windows I am getting following error. I saw a post with same thing, where u said u will be releasing new version so that it works on windows. Is it not released yet?

  36. Jo Vermeulen Says:

    You should install OpenCV first, and try to run the simple_winclient.py script. The software is available at my website.

  37. the-maxx Says:

    hi, i stumbled upon your little nice python script a few days ago and ported it to opencv 2.0 with its new python bindings. Hope anyone is interested, maybe you can update your version with this.

    import sys
    import cv

    class FaceDetect():
    def init(self):
    cv.NamedWindow (“CamShiftDemo”, 1)
    device = 0
    self.capture = cv.CaptureFromCAM
    capture_size = (320,200)
    cv.SetCaptureProperty(self.capture, cv.CV_CAP_PROP_FRAME_WIDTH, capture_size0)
    cv.SetCaptureProperty(self.capture, cv.CV_CAP_PROP_FRAME_HEIGHT, capture_size1)

    def detect(self):
    cv.CvtColor(self.frame, self.grayscale, cv.CV_RGB2GRAY)

    #equalize histogram
    cv.EqualizeHist(self.grayscale, self.grayscale)

    # detect objects
    faces = cv.HaarDetectObjects(image=self.grayscale, cascade=self.cascade, storage=self.storage, scale_factor=1.2,
    min_neighbors=2, flags=cv.CV_HAAR_DO_CANNY_PRUNING)

    if faces:
    #print 'face detected!'
    for i in faces:
    if i1 > 10:
    cv.Circle(self.frame, ((2*i0[0]+i0[2])/2,(2*i0[1]+i0[3])/2), (i0[2]+i0[3])/4, (128, 255, 128), 2, 8, 0)

    def run(self):
    # check if capture device is OK
    if not self.capture:
    print “Error opening capture device”
    sys.exit(1)

    self.frame = cv.QueryFrame(self.capture)
    self.image_size = cv.GetSize(self.frame)

    # create grayscale version
    self.grayscale = cv.CreateImage(self.image_size, 8, 1)

    # create storage
    self.storage = cv.CreateMemStorage(128)
    self.cascade = cv.Load('haarcascade_frontalface_default.xml')

    while 1:
    # do forever
    # capture the current frame
    self.frame = cv.QueryFrame(self.capture)
    if self.frame is None:
    break

    # mirror
    cv.Flip(self.frame, None, 1)

    # face detection
    self.detect()

    # display webcam image
    cv.ShowImage('CamShiftDemo', self.frame)
    # handle events
    k = cv.WaitKey(10)

    if k 0x1b: # ESC
    print 'ESC pressed. Exiting …'
    break
    sys.exit(1)

    if __name__ “main“:
    print “Press ESC to exit …”
    face_detect = FaceDetect()
    face_detect.run()

  38. the-maxx Says:

    argh, it killed my tabs!!!

  39. Jo Vermeulen Says:

    Thanks a lot! I'll give it a try!

  40. Forrest Says:

    So cool. Python can make opencv more easy and funny.

  41. gisnap Says:

    Its really cool, I came to know this really worth visiting, just bookmarked your site.

    http://gisnap.com/
    The place where fun never ends

  42. selva Says:

    hi…hw can i edit this coding so that i can detect an image that i have saved. i'm using ubuntu 9.04….need ur help…thank u…

  43. samsnaap Says:

    its been great coming across your site.however is there a way for me detect and display the available webcams conected to a pc.i tried searching online buy couldnt comeup with anything.Pygame 1.9 however has a module to take care of that.but the problem with pygame is .i cant include it in the gui of my project.not can i attach any stuff like buttons onto the defaul gui provided.would really appreciate if you could help my out.thanks in advance

  44. Jo Vermeulen Says:

    I know there is a function to get the number of cameras:

    cvcamGetCamerascount()

    Maybe that helps?

  45. samsnaap Says:

    thanks jo.well i kindda figured a way out last week. and these are basically the codes
    def listWebcam(self,path='/dev'):
    “”"detects the available webcam mounted to a computer “”"
    devicelist = []
    for device in os.listdir(path):#loops through devices all mounted devices
    if device.startswith('video'):#returns true if a device with video prefix is found
    devicenum = device.rsplit('o')#seperates video from the device index
    devicelist.append(devicenum1)
    return devicelist

  46. Jo Vermeulen Says:

    That's Linux-specific of course, but will indeed do the trick.

  47. samsnaap Says:

    yup,sorry i guess i forgot to mention i was looking for linux specific solution.
    oh and by the way.i do have some issues with the camera though.when i connect to the webcam.sometimes it just doesnt connect and the entire application just closes.
    that is when i change the device value to different numbers .”self.capture = cv.CaptureFromCAM.i tested it with three webcams connected.it connected to two but not to the third.and then i removed the third and it just connected to one.
    any idea why this is happening

  48. Failface – So you can relive that magic moment again and again « Wat Vervelend! Says:

    [...] site. Strange! Well, I still managed to put together a small prorgram (borrowing heavily from here) called grabface that grabs a picture using the webcam and then figures out the position of your [...]

  49. AlexMikhalev Says:

    Thank you for posting it online. I find it useful.

  50. Jo Vermeulen Says:

    You're welcome! I'm glad you found it useful.

  51. Free Games Says:

    Really-really helpful! Thank you so much for sharing this code :)

  52. auttapong Says:

    hi. i have this
    Null pointer (Invalid classifier cascade)
    in function cvHaarDetectObjects,
    C:UserVPoperncvcvsrccvhaar.cpp(890)

  53. Jo Vermeulen Says:

    Are you sure you have the cascade file in the same directory?

  54. auttapong Says:

    C:DevOpenCVdatahaarcascades

  55. Jo Vermeulen Says:

    I currently don't have a Linux machine for testing, but IIRC the line

    cascade = cv.LoadHaarClassifierCascade('haarcascade_frontalface_alt.xml', cv.Size(1,1))

    requires the cascade file to be in the same directory as the script.

    By the way, the script I presented in this blog post won't work on Windows. You should use the simple_winclient.py script for that (which can be found in the tarball available from my website).

  56. auttapong Says:

    thank you very much Jo ^^

  57. Jo Vermeulen Says:

    You're welcome!

  58. auttapong Says:

    hi. i have this
    Null pointer (Invalid classifier cascade)
    in function cvHaarDetectObjects,
    C:UserVPoperncvcvsrccvhaar.cpp(890)

  59. Jo Vermeulen Says:

    Are you sure you have the cascade file in the same directory?

  60. auttapong Says:

    C:DevOpenCVdatahaarcascades

  61. Jo Vermeulen Says:

    I currently don't have a Linux machine for testing, but IIRC the line

    cascade = cv.LoadHaarClassifierCascade('haarcascade_frontalface_alt.xml', cv.Size(1,1))

    requires the cascade file to be in the same directory as the script.

    By the way, the script I presented in this blog post won't work on Windows. You should use the simple_winclient.py script for that (which can be found in the tarball available from my website).

  62. auttapong Says:

    thank you very much Jo ^^

  63. Jo Vermeulen Says:

    You're welcome!

  64. TechB Says:

    Hey great post. I can't wait to get started hacking it to my needs. I made a script that tracts an IR LED and maps the mouse to the movements(http://hackaday.com/2010/02/25/python-ir-tracking-for-the-handicapped/).

    I used VideoCapture(http://videocapture.sourceforge.net/) for the webcam interface though. I'm going to try and map the mouse movements to this new face-recognition.

  65. TechB Says:

    After about a week of work, I couldn't get opencv to work. I used VideoCapture again, and fdlib.dll to detect the faces. http://www.kyb.mpg.de/bs/people/kienzle/facedem…;

  66. 1john01 Says:

    Me and my friend want to set up a funny type call of duty video type thing, but to do so we would need to. BlackStar69

  67. 1john01 Says:

    Me and my friend want to set up a funny type call of duty video type thing, but to do so we would need to. BlackStar69

  68. Stefan Says:

    Hey,

    I just found your amazing code on the web. I'm trying to make a eye tracking device for a paralized girl. I tryed your code but got this error.
    Can you help me with this problem. I really don't know what to do.

    Traceback (most recent call last):
    File “D:My DocumentsMijn documentenTWGanotherdetection.py”, line 65, in <module>
    detect(frame)
    File “D:My DocumentsMijn documentenTWGanotherdetection.py”, line 22, in detect
    faces = cv.HaarDetectObjects(grayscale, cascade, storage, 1.2, 2, cv.HAAR_DO_CANNY_PRUNING, cv.Size(50, 50))
    WindowsError: exception: access violation reading 0×00000028

    Thx,

    Stefan (from the Netherlands)

  69. Stefan Says:

    Hey,
    I found the solution already. Needed to put the
    'haarcascade_frontalface_default.xml' file in the same directory as your little programm. When I runned the code, it seems to work fine. I only noticed that the program is often detecting my chin and mouth.. and not the whole face. Is this because I'm to close to my cam or is there a other solution for.

    Thx,

    Stefan

    ps: I also needed to put “from opencv.highgui import *” above the code. Did yours work without?

  70. Payday Loans Says:

    OpenCV has reasonably robust Python interface created with SWIG. This makes it easy to integrate with existing libraries like Pygame and olpcgames. As an example, OpenCV can be combined with Pygame to make a simple face tracking xeyes clone.
    Thanks.

  71. Phifo Says:

    dont work for me

    from CVtypes import cv
    ImportError: No module named CVtypes

  72. public records Says:

    i read this and detection is one component of … It is a simple python script that will connect to your webcam

  73. stuaxo Says:

    “When you’re struggling with SWIG-generated Python bindings, or have some C library without bindings that you would like to use, give ctypes a try. “

    This reminded me of this
    “If you have a problem – if no one else can help – and if you can find
    them – maybe you can hire: The A-Team.”

    I almost want somebody to geekerise the lyrics and make them about ctypes now.

  74. links for 2010-06-19 | andy.edmonds.be Says:

    [...] In Traction » Blog Archive » Fun with Python, OpenCV and face detection (tags: python opencv webcam facedetection face) Published: June 20, 2010 Filed Under: Delicious Leave a Comment Name: Required [...]

  75. The Magic Jack Says:

    wow thats some crazy code

  76. air climber Says:

    I got lost half way through your code

  77. air jordan 13 Says:

    Mark S. is definitely on the right track. If you want to get a professional looking email address, Id recommend buying your name domain name, like or
    Gucci sweaters
    If its common it might be difficult to get, however, be creative and you can usually find something.

  78. 800 calorie diet Says:

    Golden. Great, useful info.

  79. rickdouglas85 Says:

    Absolutely incredible – technology is certainly an amazing thing!


    Rick, MJ Researcher

  80. Matt Says:

    Nice! I've got some motion detection going on over here if you're interested: http://appdelegateinc.com/blog/2010/08/02/motio…;

    Did you buy the book? I find the OpenCV documentation is very hard to follow.

  81. amitjhsaps123 Says:

    Muito obrigado pelos elogios…de coração.
    Se depender de mim, estarei sempre compartlhando minha paixão, que são os games!!
    Quanto a fase da aranha realmente eh dificil, mas é ali onde vc realmente fica “ninja” nas suas habilidades , e ai o resto do game fica muito mais atenuado, hehe como se fosse uma academia essa fase, rs.
    Abração amigo
    online jogos

Leave a Reply

blog comments powered by Disqus