Kinect Option 1: Synapse

Synapse is an application we were able to download from the Internet that works on Mac and Windows. It easily allows us to use Kinect to control Max/MSP/jitter, and any other application that can receive OSC events. It sends joint positions and hit events via OSC, giving us the ability to track different gestures.

What Synapse does is, it takes information from the camera and binds a skeleton to the body when the user stands in front of the kinect in "psi". It then sends OSC messages out on ports 12345 and 12347, and receives OSC messages on port 12346.

The messages that Synapse sends are:

/tracking_skeleton - Sent with 1 when we start tracking a skeleton, and 0 when we lose it.

/_pos_world - The x,y,x position of the joint in world space, in millimeters.

/_pos_body - The x,y,x position of the joint relative to the torso, in millimeters.

/_pos_screen - The x,y,x position of the joint as projected onto the screen, in pixels.

/ [up,down,left,right,forward,back] - Sent when we detect a "hit" event, such as a punch forward, which would be "righthand forward".

/_requiredlength - The current tuning value of how far you must move in a straight line (in mm) before a hit event is generated.

/_pointhistorysize - The current tuning value of how many points are being tracked for hit event detection.

The following are a list of joints that are tracked on the skeleton is mapped to the user's body:

• righthand
• lefthand
• rightelbow
• leftelbow
• rightfoot
• leftfoot
• rightknee
• leftknee
• head
• torso

The following is a list of valid messages to send to synapse:

/_gettuninginfo - This causes the _requiredlength and _pointhistorysize messages to be sent from Synapse, so that you can see what the current values are.

/_requiredlength - Use this to change how far you must move this joint in a direction (in mm) to trigger a hit event. Default to 150.

/_pointhistorysize - Use this to change how many points are being tracked for his event detection. This is essentaially a control for how fast you must move your hanf to cause a hit event, lower means you must do it faster. Defaults to 5.

/_trackjointpos - This is the keepalive to cause joint positions to continue being spit out. Valid values to pass are: 1, to get_pos_body positions; 2, to get_pos_world positions; and 3, to get_pos_screen positions.

/depth_mode - This allows you to cut the background our of the depth image and only see the user. Valid values to pass are: 0 to see the whole depth buffer, 1 to se only this tracked user (or all person-shaped things if no one is tracked), 2 to see all person-shaped things even if a user is tracked.

An example of Synapse being used with Ableton Live to create music:

http://www.youtube.com/embed/1ge7RcIBWsc