PCL library with Kinect under OSX 10.11

This last week, I dug up my trustworthy Kinect for a spin. I’ve been wanting to mess around with the PCL¬†(Point Clouds) library for some time, so I decided to give it a shot.

Installation on OSX using Homebrew is fairly straigthforward, as shown in their documentation. However, I want to make sure that I have support for the Kinect (the Xbox 360 model).

Side note on Kinect support:¬†To get data off of your Kinect, you can use the OpenNI library (which handles Kinect “1”). ¬†OpenNI2¬†does exist, but it handles only the Kinect “2” and Occipital’s Structure Sensor. I’ll be using OpenNI here, because it’s supported directly by PCL. However, for standalone applications, I’d greatly recommend libfreenect¬†(also available through Homebrew), which is fast, lightweight, and very easy to use.

So, we’ll be using Homebrew to get the following libraries: Boost, Flann, Qt4, GLEW, VTK and last but not least, OpenNI (1 and 2, just for argument’s sakes). ¬†Run:

brew update
brew install boost flann qt glew vtk openni openni2

Grab a coffee, ’cause this will take some time. Now, to install PCL, you may want to check the available options (brew options pcl), then install it with at least the following settings:

brew install pcl --with-openni --with-openni2

Great, if nothing intensely wrong happened midway, you should be golden. Now, at that point I was eager to get some Kinect-demo up an running, which led me to PCL’s openni-grabber¬†page (go ahead, visit the page). They are kind enough to supply the C++ file and a CMakeLists.txt to compile it. It compiled fine, but crashed hard every time I tried to use it, spitting the message:

Assertion failure in +[NSUndoManager _endTopLevelGroupings], /Library/Caches/com.apple.xbs/Sources/Foundation/Foundation-1256.1/Misc.subproj/NSUndoManager.m:359
2016-01-06 12:50:51.779 openni_grabber[42988:7013866] +[NSUndoManager(NSInternal) _endTopLevelGroupings] is only safe to invoke on the main thread.

Now, I don’t know if this happens under other platforms. After wasting some days struggling with this, I realized what the heck was going on. Let’s take a look at the¬†non-working supplied code. In the header section, we see:

It uses the PCL’s Cloud Viewer, which has been long deprecated¬†(meaning that this example is very out-of-date). Secondly, we see that the¬†grabber’s¬†functionality is based on a callback, defined here:

The cloud_cb_ callback function is called every time a new data packet arrives from the Kinect. This is fine, but the showCloud() command updates the display of the point cloud from within the callback, witch is what’s creating our "...is only safe to invoke on the main thread..." error. To fix that, the callback should only update the cloud’s data, while the cloud itself must be displayed with a call placed on the main thread. The fixed code I came up with looks like this:

Now, the callback cloud_cb_ only updates the cloud’s data, not touching the UI. The drawing happens in a loop inside the SimpleOpenNIViewer::run(). Since the callback and the loop are handling the same data set asynchronously, I’ve added a mutex around the critical sections to avoid any issues (the nice CPP¬†wrapper ¬†stolen from libfreenect’s examples, thanks!). To compile it, the following CMakeLists.txt should do the trick:

The results are not astounding: the RGB data is slightly out of place with the depth data, but I believe this is a problem internal to PCL, since my Kinect is properly calibrated. To use only the depth data in the code above, simply replace all instances of pcl::PointXYZRGBA with pcl::PointXYZ.

’til next time!