Getting up and running with the Kinect in Ubuntu 12.04

I’m currently experimenting a bit with the Kinect depth camera, which I intend to use for a prototype. I’ve played around with the device before on Windows using the Microsoft Kinect SDK. Setting up libfreenect on Windows is actually quite a pain, which was the reason why I started with the Microsoft SDK. As I wanted to code in Python, I’ve had a look at PyKinect and the Python Tools for Visual Studio (see also this PyCon US 2012 talk about building a Kinect game with Python). Unfortunately, the PyKinect samples seem to be outdated, and don’t work with the latest version of the Kinect SDK (version 1.0, released in February).

So I decided to see how far I would get with the Kinect and libfreenect on Ubuntu (12.04). Step 1: install the libfreenect demos to test if it works (this will also install libfreenect and its dependencies).

$ sudo apt-get install libfreenect-demos

So far so good. Unfortunately, the demo application wouldn’t start:

$ freenect-glview 
Kinect camera test
Number of devices found: 1
Could not claim interface on camera: -6
Could not open device

Strange.. I did another check to see if my user was in the plugdev group (which
was the case):

$ groups jo
jo : jo adm cdrom sudo dip plugdev lpadmin sambashare

Then I noticed that the Kinect’s LED kept blinking continously, which usually means there’s some sort of connection problem. It was correctly recognized, though:

$ lsusb
...
Bus 002 Device 007: ID 045e:02b0 Microsoft Corp. Xbox NUI Motor
Bus 002 Device 008: ID 045e:02ad Microsoft Corp. Xbox NUI Audio
Bus 002 Device 009: ID 045e:02ae Microsoft Corp. Xbox NUI Camera

After a quick web search for the specific libfreenect-glview error message, I learned that recent versions of the Linux kernel prevent libfreenect from claiming the Kinect as they now include a driver to support the device as a regular webcam (see kinect.c), which is actually quite cool. This also means there should be a specific video device for the Kinect (/dev/video1 in my case).

The kernel modules are indeed loaded:

$ lsmod | grep -i gspca
gspca_kinect           12936  0
gspca_main             28366  1 gspca_kinect
videodev               98259  2 gspca_main,uvcvideo 

Let’s try playing the camera stream using GStreamer and Video4Linux:

$ gst-launch-0.10 v4l2src device=/dev/video1 ! video/x-raw-yuv ! ffmpegcolorspace ! xvimagesink

That seems to work!

The Kinect actually has two image sensors: an RGB camera and a depth sensor, which consists of an infrared laser projector and a monochrome CMOS sensor. A depth map is created by projecting structured infrared light and capturing the resulting image from the monochrome sensor. As I wasn’t sure how to grab an image from the monochrome sensor, I contacted the author of the kernel module (Antonio Ospite). He told me it’s possible to get a monochrome image by specifying an image size of of 640×488 pixels (instead of the usual 640×480). Note that the kernel module currently only supports video streams from the monochrome sensor as unprocessed output.

If we pass that specific width and height to GStreamer, we get this:

It’s also possible to get the full-size (1280×1024) monochrome image. However, most webcam apps will just show the video stream from the RGB sensor for that resolution as you can’t specify that you want the specific Y10B format. To do that, you can use a separate program like qv4l2, which can be installed as follows:

$ sudo apt-get install qv4l2

To get the full-size monochrome image, run qv4l2, open the /dev/video1 device in raw mode (File > Open raw device), select 1280×1024 for the frame size, and “Y10B – Y10B” for the capture format, and click the red capture button:

OK, back to running the libfreenect demo. Someone over at superuser.com suggested to remove the modules (so that the user-mode libfreenect driver could
take over) and run libfreenect-glview again.

And indeed, after removing both gspca modules, the freenect-glview demo did work:

$ sudo modprobe -r gspca_kinect 
$ sudo modprobe -r gspca_main
$ freenect-glview 
Kinect camera test
Number of devices found: 1
GL thread
Write Reg 0x0006 <= 0x00
Write Reg 0x0012 <= 0x03
Write Reg 0x0013 <= 0x01
Write Reg 0x0014 <= 0x1e
Write Reg 0x0006 <= 0x02
Write Reg 0x0005 <= 0x00
Write Reg 0x000c <= 0x00
Write Reg 0x000d <= 0x01
Write Reg 0x000e <= 0x1e
Write Reg 0x0005 <= 0x01
...

The left side of the window shows a colored depth image, while the right side shows the RGB image from the camera. Of course, it would be quite cumbersome to remove the modules again every time you want to use libfreenect. One option is to blacklist the gspca module as follows:

$ echo "blacklist gspca_kinect" >> /etc/modprobe.d/blacklist.conf

Now that we've got that sorted out, I'll try out the libfreenect Python wrapper (or perhaps SimpleCV).

Update: Antonio Ospite pointed out in the comments that recent versions of libfreenect (0.1.2) can automatically detach the kernel driver. There's a PPA available by Florian Echtler with updated libfreenect packages for Ubuntu 12.04.

  • Avi Cohen !

    I love you ! Thankssssssssssssssssssss

  • Shonn E

    Thank you so much for figuring out the  “Could not claim interface on camera: -6″  problem Jo. I have been fighting this problem for a couple of weeks and searching blogs and help sites and nothing I ever tried worked. I had tried   “sudo modprobe -r gspca_main” but it wouldn’t execute the command because it said gspca_main was in use. I had not found anything that stated you had to do the “sudo modprobe -r gspca_kinect” as well … and to do it first! This instantly solved my problems, and freenect-glview works now. I had tried creating a blacklist, but the person whose instructions I was following said to blacklist “gspca_main” not “gspca_kinect”. When I did so, it actually prevented my Ubuntu load from booting up! I have a dual boot set up where I can boot to Windows7 or Ubuntu 11.10, and after setting up the blacklist of gspca_main it would just sit and spin while trying to load and never come up. I had to boot to a sort of “safe mode” and remove the blacklisted file to get back up and running.   

  • http://blog.jozilla.net/ Jo Vermeulen

    You’re very welcome! I mostly wrote this post to document how to get libfreenect to work on Ubuntu, so I wouldn’t forget myself I’m glad you got it to work eventually, at least it’s easier to work with libfreenect on UNIX systems than on Windows

  • anthonyoliver

    One of the core SimpleCV (http://simplecv.org) developers here.  Just upgraded to ubuntu was happy they are packaged now, but seems not quite.   Thanks for the help and mention for simplecv.

  • http://blog.jozilla.net/ Jo Vermeulen

    You’re welcome! Thank you for mentioning this blog post on Twitter

  • http://ao2.it/ Antonio Ospite

    Recent libfreenect can detach the kernel driver itself, we have this version 0.1.2 in Debian, but I don’t know what version ubuntu is shipping.

  • http://blog.jozilla.net/ Jo Vermeulen

    Thanks for pointing that out, Antonio! 

    It seems version 0.1.2 is available in the current development release of Ubuntu (12.10 – Quantal Quetzal). Version 12.04 ships with version 1:0.0.1+20101211+2-3 which is a snapshot from late 2010, I guess. I just found a PPA by Florian Echtler with up-to-date packages though. I’ll try that and see whether it works!

  • TiagoBarufi

    beautiful post and tutorial. Thanks for sharing!

  • http://blog.jozilla.net/ Jo Vermeulen

    I’m glad you found it useful!

  • Stefán Stefánsson

    Hi and thanks for the writeup.

    I installed the 0.1.2 version of libfreenect but now I’m having permission problems. I have to run all the programs as root (sudo) to get them to connect to the kinect.

    I don’t know why this is happening because before I installed the new version (when I got it running by removing the kernel modules as you suggested) I was able to run the demo programs as a regular user.

    -Stefan Freyr.

  • http://blog.jozilla.net/ Jo Vermeulen

    Indeed. I also have to run the libfreenect demos as root to be able to use the Kinect. I guess that’s because the program needs permissions for removing the gspca kernel module.

  • http://www.facebook.com/profile.php?id=592257831 Alex Wallar

    Hey man,
    This post was super useful! I have been bugged by this disappointment for days but now everything seems to work
    How did you get so good at using Linux, is there somewhere I should start?

  • http://blog.jozilla.net/ Jo Vermeulen

    By learning from others and by reading the built-in documentation (e.g., manpages), I guess. A more detailed answer might be a good idea for a next blog post

  • Homar6

    You’re my master… thanks!!

  • Donald Woods

    Agreed with you))

  • http://essaybase.co.uk/ essaybase.co.uk

    Thank you very much for your help! Now I have all figured out!

  • Rick

    Thanks for the tip on gspca_kinect – I had the same problem with 12.04, and your advice
    resolved the issue. Real time saver – thanks!

  • Chris
  • http://researchpaperwriter.net/ research paper

    You’re my master… Now I have all figured out!

  • dirtydevil

    Hi, Thanks for such a detailed post for Kinect Beginners.

    Though my freenect demos are working but on trying lsmod | grep gspca , I am not getting any gspca_kinect or anything. Can you tell me please why I am not getting this. Plus I am getting only one /dev/video0 which is of my integrated Webcam . I am not getting any file for kinect like /dev/video1 . I think without it I can’t use openCV or any other processing. Thanks for help in advance. Please Do reply ASAP.

  • Alex M.

    very nice tutorial.
    any ideas why after removing both gspca modules, the freenect-glview demo did NOT work giving the “Could not open device” error? I am on Ubuntu 12.04

  • Carl Nobile

    I have just done an update to my Ubuntu 12.04 and the Kinect stopped working. I had compiled my own opencv and libfreenect, so I uninstalled them and put on the distribution versions and the same issue persists. Has anybody run into this problem?

    When I run glview I now get:

    $ glview
    Kinect camera test
    Number of devices found: 1
    Could not open motor: -3
    Could not open device

    ~Carl

  • Jeremy Blair

    Nice tutorial, but I’m getting this message when I try to do freenect-glview

    Kinect camera test

    Number of devices found: 1

    send_cmd: Bad magic 00 dc

    freenect_fetch_zero_plane_info: send_cmd read -1 bytes (expected 322)

    freenect_camera_init(): Failed to fetch zero plane info for device

    Could not open device

    Not quite sure what it means, any help would be appreciated.

  • Daneel Ellinggton

    i’ve got this issue too, what can i do ?

  • Jyeo

    I’m getting the same issue, anyone have any suggestions?

  • Fatfreddy

    Same problem, can’t open it as regular video device, as dev/ video1 or video0

  • ZB

    Thanks bud, this was great – I still run into the issue of not being able to open though on Fedora 19