In this 2nd assignment of the 2013/2014 Laptop Orchestra, our team has the opportunity to conduct an assessment of the various hardware and software options available, that may combine especially well for alternative body tracking situations such as tracking a pianist from the aerial perspective.
In such a case (and many further possible performance scenarios), the entirety or a portion of the body to be tracked is occluded from the camera’s view, and various hardware and software combinations are proving to be better or worse at compensating for this.
The current state of our research is to first evaluate the most convenient and common combination, provided by the regular XBox 360 Kinect and NI-Mate tracking software which automatically translates and scales data from what the Kinect receives to OSC and MIDI output. We have found that the NI-Mate software must still see most of the performer to function correctly, and so we have mounted it at a 3/4 angle that is pointing equally toward the pianist as it is pointing down toward the keyboard.
While this rules out finger tracking for the time being (other software libraries are more capable at this), we have the opportunity to assess the merits of tracking hand gestures that happen between performed passages on the keyboard, thus affording the ability to make music “between the notes” as well as during them.
Because we believe this may yield interesting and musical results, we have decided to stick with it for the time being, possibly for the duration of the assignment, before continuing this experiment across other software/hardware combinations as the year goes on.
For reference, here are the other hardware and software options in our table:
– Kinect for Windows (read the sub-section “Comparing Kinect sensors”