Kinect2Scratch – The power of joint tracking in Scratch.

Microsoft Kinect units can be found second hand on eBay for £20 or so, and they are STEM gold.  There are so many amazing pieces of code out there that let you extract the joint info from Kinect and make it ready for other apps in the form of OSC, midi, or raw data  (NI mate is a commercial venture).

One of the best uses of a Microsoft Kinect I’ve seen is with Scanect, a piece of software that lets you map the video data to the distance data in order to make accurate 3d models of small objects, people or rooms rapidly and cheaply.  However, that’s quite an advanced STEM project, and I needed something that could use the power of the Kinect, and combine it with the rapid prototyping and easily accessible code of Scratch.

Using a great piece of software developed by Stephen Howell called Kinect2Scratch,  (http://scratch.saorog.com/ for the download) it’s possible to import skeleton data from the kinect directly into Scratch and use it to code with.  You have to install the Kinect runtime for Windows first, but once running, The Kinect2Scratch program runs in the background, extracts the joint information, and sends it to Scratch.  (setup guide here: http://scratch.saorog.com/setup.pdf)

Mac users have an option that is coded by Kenta Hara:  http://github.com/mactkg/kinect2Scratch4Mac

Once Kinect2SCratch is working in the background, it just chugs away and sends skeleton tracking data to Scratch, with very few issues.  The student reactions to having such profound input into Scratch was incredible.  Within about 20 minutes, they had brainstormed at least 8 different ways of using joint input to make games, animations, presentations.

It comes with some really great sample programs to get you started, but my favourite one was the skeleton tracking.  It uses some crafty code to draw a live skeleton in scratch with very little lag, and puts Kitty Cat’s head on the top.

Over the course of the Purbeck 2014 Scratch Jam, Team Kinect made a game from the bottom up that used the Chest Centre tracking point (Around where the solar plexus is) to control the Horizontal movement of a space ship in a vertical scrolling shooter.  This actually made playing the game a real workout, as to avoid incoming meteorites, you need to move your entire body, ducking and jumping at times!   The ship fires by raising arms to prepare the lasers, and then bringing them below the shoulders to fire.

It’s a great game, with randomised meteorites, good scoring mechanic, menacing score and great workout to boot.

The Project is shared at:

http://scratch.mit.edu/projects/37598364/

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s