Kinect 2.0 library for Processing

Kinect 2.0 library for Processing

If you want to use the brand new Kinect One with Processing, this page contains the links for the library, tutorial and more information.

The project is constantly being updated on github.

Current

Version 0.41 Beta

Requeriments

  • Kinect One
  • Windows 8  bits (only testes on 8, it should work for 7)
  • USB 3.0
  • up to Processing 2.2.1

Install

  • Install Kinect 2.0 SDK Beta.
  • Copy the KinectPV2 folder to your processing libraries sketch folder (Documents/Processing/libraries)

Next Version

  • Reference
  • 3D Point Cloud
  • Mask 6 users

Version 0.4 Beta

Downoad here

  • Skelton tracking for 6 user
  • Hand gesture recognition, hand open, close.
  • Body track, only supports 3 users, next version upto 6 users.
  • LongExposureInfrared.

Version 0.3 Beta

  • RGB Image
  • Grayscale Infrared
  • Grayscale Depth data

Build

Hey!, so you need to build the library from source, you can find it hereKinectPV2_BuildLibs

vc2012, Includes the Visual Studio 2012 project for creating the .dll file
eclipse, The eclipse project for creating the .jar file

KUZPIG2SZENO

Code!!!

To get started with the library, you need to import the kinect library, and initilize the class with the following.

import KinectPV2.*;

KinectV2 kinect;

void setup() {
  size(900, 768);

  kinect = new KinectV2(this);
  kinect.enableColorImg(true);
  kinect.enableSkeleton(true);
  kinect.enableInfraredImg(true);
  kinect.init();
}

So you want to activate various functionalities of the kinect one, you just need to call the enable functions before the kinect.init() function.

void enableColorImg(boolean toggle);
 void enableDepthImg(boolean toggle);
 void enableInfraredImg(boolean toggle);
 void enableBodyTrackImg(boolean toggle);
 void enableLongExposureInfrared(boolean toggle);
 void enableSkeleton(boolean toggle);

obtain the image from rhe kinect device

PImage getColorImage();
PImage getDepthImage();
PImage getInfraredImage();
PImage getBodyTrackImage();
PImage getLongExposureInfrared();
Skeleton [] getSkeleton();

Examples

Some exmples how to use the library.

Color Image example

Simple example how to obtain the 1920 x 1080 color image from the Kinect

import KinectPV2.*;

KinectV2 kinect;

void setup() {
  size(1920, 1080);
  kinect = new KinectV2(this);
  kinect.enableColorImg(true);

  kinect.init();
}

void draw() {
  background(0);
  image(kinect.getColorImage(), 0, 0);
}

Depth,infrared, mask example

The depth, infrared and mask(body tracking) are capture with a resolution of 512 x 424 pixels, a little more higher than the previous kinect. To obtain the depth, infrared and mask just activate them in the setup and then initialize the kinect 2.0.

import KinectPV2.*;

KinectV2 kinect;

void setup() {
  size(512*3, 424);
  kinect = new KinectV2(this);
  kinect.enableDepthImg(true);
  kinect.enableInfraredImg(true);
  kinect.enableBodyTrackImg(true);
  kinect.init();
}

void draw() {
  background(0);

  image(kinect.getDepthImage(), 0, 0);
  image(kinect.getBodyTrackImage(), 512, 0);
  image(kinect.getInfraredImage(), 512*2, 0);
}

Skeleton?

ok, so you want to grab the skeleton positions from the kinect, the skeleton supports up to 6 users with 25 joints each, and hand states, like open, close and lasso.

import KinectPV2.*;

KinectV2 kinect;
Skeleton [] skeleton = = new Skeleton[6];

void setup() {
  size(1024, 768);
  kinect = new KinectV2(this);
  kinect.enableSkeleton(true );
  kinect.windowSizeSkeleton(1024, 768);
  kinect.enableBodyTrackImg(true );
  kinect.init();
}

void draw() {
  background(0);
  image(kinect.getBodyTrackImage(), 0, 0, 200, 200);
  skeleton =  kinect.getSkeleton();
   for(int i = 0; i < skeleton.length; i++){
      if(skeleton[i].isTracked()){
        skeleton[i].drawBody();
        skeleton[i].drawHandStates();
      }
    }
}

 
12600

Point Cloud

To render the point cloud really fast at 60fps you need to use some some openGL calls and some buffer calls. Most of the work is done using this line of code, here you obtain a float buffer with all of the points of the point cloud (x, y, z), basically is just a big array of float. But you can use FloatBuffer to send directly the array of float to opengl to render all the points at the same time, this is extremly fast.

FloatBuffer pointCloudBuffer = kinect.getPointCloudPosFloatBuffer();

After creating a floatBuffer with all the positions of the point cloud, the next step is to send the buffer to openGL for it to be draw.

gl2.glEnableClientState(GL2.GL_VERTEX_ARRAY);
gl2.glVertexPointer(3, GL2.GL_FLOAT, 0, pointCloudBuffer);
gl2.glDrawArrays(GL2.GL_POINTS, 0, kinect.WIDTHDepth * kinect.HEIGHTDepth);

Complete code here. Point Cloud

Kinect 2 Library for processing preview from thomas sanchez lengeling on Vimeo.

Raw Data

So need to grab the raw data for each Image or each joint from the skeleton. This Methods will help you do that.

int [] getRawDepth();
int [] getRawColor();
int [] getRawInfrared();
int [] getRawBodyTrack();
int [] getRawLongExposure();
//JOINTS Example
skeleton =  kinect.getSkeleton();
Joint [] skeleton[i].getJoints();