Depth Camera Research

In this 2nd assignment of the 2013/2014 Laptop Orchestra, our team has the opportunity to conduct an assessment of the various hardware and software options available, that may combine especially well for alternative body tracking situations such as tracking a pianist from the aerial perspective.

In such a case (and many further possible performance scenarios), the entirety or a portion of the body to be tracked is occluded from the camera’s view, and various hardware and software combinations are proving to be better or worse at compensating for this.

The current state of our research is to first evaluate the most convenient and common combination, provided by the regular XBox 360 Kinect and NI-Mate tracking software which automatically translates and scales data from what the Kinect receives to OSC and MIDI output. We have found that the NI-Mate software must still see most of the performer to function correctly, and so we have mounted it at a 3/4 angle that is pointing equally toward the pianist as it is pointing down toward the keyboard.

While this rules out finger tracking for the time being (other software libraries are more capable at this), we have the opportunity to assess the merits of tracking hand gestures that happen between performed passages on the keyboard, thus affording the ability to make music “between the notes” as well as during them.

Because we believe this may yield interesting and musical results, we have decided to stick with it for the time being, possibly for the duration of the assignment, before continuing this experiment across other software/hardware combinations as the year goes on.

For reference, here are the other hardware and software options in our table:


Kinect for Windows (read the sub-section “Comparing Kinect sensors”

PrimeSense Carmine (long range)



Hand tracking library by Forth

Kinect for Windows API

Candescent Hand Tracking Library

This entry was posted in IMPART SUBCLASS, Uncategorized and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s