Monday, February 20, 2012

Blender and Kinect

Having played around with the Kinect already for some time and using it as an input device to control some of my favourite tools (Processing, vvvv, and Supercollider), I was excited to see NI Mate released .
NI Mate is a small but powerful software that takes real-time motion capture data from a Kinect or other supported device and turns it into two industry standard protocols: OSC (Open Sound Control) and MIDI (Musical Instrument Digital Interface). And they have showed already how to use it to control characters in Blender.
I joined their Beta program, but unfortunately it didn't work with the drivers I have installed (SensorKinect, OpenNI or NITE). Uninstalling and updating it with their versions of the drivers was not an option since I fear of some of my other applications might no longer work. So I looked at their Blender examples and after a weekend of Python coding and testing I finally got my own Kinect Blender setup to work. It uses OSCeleton as middleware to convert the Kinect sceleton to OSC data which are then sent to Blender. Although NI Mate is more advanced, OSCeleton is simple and Open Source. I've read that NI Mate maybe will become a commercial product after the Beta.
Now lets welcome the "Kinect Blender Bunny". This example was rendered in realtime using the Blender Game Engine.



It's still a long way to use this for a machinima movie project, but I'm excited to have this technology ready to be used. Controlling a 3D character in realtime with markerless motion capture is amazing.

3 comments:

  1. Awesome! Looking forward to seeing some of this employed in movie making!

    ReplyDelete
  2. Hi,

    Have you got any tutorial or could you share the steps / files you used?

    Regards

    ReplyDelete
  3. I unfortunately don't have any tutorial.
    As a good start the informations on the NIMate website about Blender is very helpful.
    http://www.ni-mate.com/use/blender/
    They have .blend file and tutorial as well.
    As written in my post I have used their .blend file and python script as a start and just replaced their software to generate the OSC messages with the OSCeleton software.
    I have attached a link to the modified Blender file
    for you. Hope that helps.
    http://dl.dropbox.com/u/13610304/BGE_Bunny_OSCeleton.blend

    ReplyDelete