What can you do

with a Kinect?

Matt Biddulph, Nokia

Kinect hacking is the latest in a long line of re-purposed consumer electronics. These days
it’s cheaper for companies to build devices by gluing together standard components than it is
to create proprietary hardware. Their economies of scale give hackers access to existing
hardware at new pricepoints.
The Kinect’s RGB camera captures a normal view of a scene
Its 3D infrared cameras capture a depth-map of the same scene - darker pixels are closer to
the camera
Rotate that depth-map in 3D and you can see how it creates all sorts of possibilities for
understanding a scene.
 Software
XBox software performs:
depth mapping
skeletal analysis
gestures
face recognition
voice recognition with noise suppression

All these features are built into the closed XBox SDKs used by game developers. These are
NOT available to Kinect hackers.
 Works with Processing, Cinder, Open
Frameworks etc
Full framerate camera + depth map
Microphone features not available yet
Lacks the XBox software features...
... but works well with OpenCV for
computer-vision image analysis

But OpenKinect is creating drivers, integrations and libraries for high-level systems such as
Processing to enable the software side of Kinect hacking.
“Kinect is making nothing
which wasn't already
technically possible, possible.”

http://memo.tv/kinect_why_it_matters
“It is just making it accessible,
not just in terms of price, but
also in terms of simplicity and
ease.”

http://memo.tv/kinect_why_it_matters
 !2! 3,.-0-(-$0)$.)4,/1&#,4,0()#,5-$01)
!
?*#2A&*1*$"'&*G),$2H'%*-)$)"),$' ?*#2A&*1*$"'&*G),$2H'L,+#&'%)1*$2),$2'
C*G),$'EH'I#&J'&*%''''C*G),$'FH'K&)G7"'&*%' C*G),$'EH'MEN6'''C*G),$'FH'MFN6 '
!

"#$%%%!&'('!")**( !

$%&'(%)*+&',-+.,-)%,/)0.1,2+,3)4)526'(,'2)7%&'(%-)899:9999#;)99<;)9==)%,/)9=!>)

' !
!
?*#2A&*1*$"'&*G),$2H'&*L&*2*$"#"),$',:*&'L)9*+'-)*+%' ?*#2A&*1*$"'&*G),$2H'&*L&*2*$"#"),$',:*&'L)9*+'-)*+%'
!
!"#$%#&%'-)*+%',-':)*/'O#1*&#2'3456'378'9'546'3:88' ;)%*'-)*+%',-':)*/'O#1*&#2'3<=6'378'9'><6'3:88 '
"#$%%%!&'('!")**( !

!
!
!
! "#$%&'($&)*
!+, -&.#/&*0(.#'1(2'1*&'3*"24'5('6*7*89:*$&.#/&1*;<<=<<<<,>*<<?>*<,=*&'3*<

' !
!
!
!"#$%&'()*(+,-,( .&('&/&'0&(#1&('231#(#$(456&(#&"172"58(58#&'5#2$7/(92#1$:#(;'2$'(7$#2"& ( ((((<5#5(=1&&#(>&0?()?@(

You can get higher framerates and more precise 3D imaging from industrial cameras, but
Kinect costs around $150 and is available
! on the high-street. It’s cheap enough to make silly
things with.
)*(+,-,( .&('&/&'0&(#1&('231#(#$(456&(#&"172"58(58#&'5#2$7/(92#1$:#(;'2$'(7$#2"& ( ((((<5#5(=1&&#(>&0?()?@(
 Applications
UI control
Gestures and skeletal body tracking
Robotics
3D Scene Reconstruction
Visualisations and art
Realworld multitouch
Showreel
http://kinecthacks.net/kinect-background-deletion-and-depth-interaction/
http://kinecthacks.net/real-time-people-detection-and-tracking-with-multiple-kinect-
cameras
 Signlanguage Recognition

http://kinecthacks.net/american-sign-language-recognition-using-kinect
http://kinecthacks.net/transform-yourself-into-a-japanese-superhero/
http://kinecthacks.net/body-dysmorphic-disorder
http://kinecthacks.net/kinect-invisibility
 Robot vision

http://singularityhub.com/2010/11/17/hacked-irobot-uses-xbox-kinect-to-see-the-
world-obey-your-commands-video
http://kinecthacks.net/quadrotor-autonomous-flight-and-obstacle-avoidance
http://kinecthacks.net/kinect-piano