Hi, I’m Naithan Bosse and I’m a programmer in SUBCLASS. This week I have been working with Kieran Coulter and Greg Burgess to figure out how best to track Angelique Po’s movements at the piano using an Xbox Kinect. After considering a number of different tracking methods, we decided to mount the Kinect above Angelique and use the software “NI-mate” to follow Angelique’s hand movements. This set up works fairly well when Angelique’s hands are more than (roughly) 2 inches above the keyboard. However, when Angelique’s hands are less than 2 inches above the keyboard, NI-mate has trouble recognizing where her hands end and where the keyboard begins, causing tracking errors. To work with this, I have created a gate which only allows Angelique to control the electronics when her hands are far enough away from the keyboard to avoid confusing the software.
We have also begun assembling and testing various sound processing devices, which can be triggered by Angelique’s motions. This includes harmonization bpatchers from the UBC Toolbox and a granular synthesis module.