Programming Guide
Getting Started with the Kinect for Windows SDK Beta from
Microsoft Research
Beta 1 Draft Version 1.1 – July 22, 2011
About This Guide The Kinect™ for Windows
®
Software Development Kit (SDK) Beta from Microsoft
Research is a starter kit for applications developers. The kit makes it easier for the academic research
and enthusiast communities to create rich experiences by using Kinect for Xbox 360
®
sensor technology
on a PC that is running the Windows 7 operating system.
This beta SDK is provided by Microsoft Research to enable researchers and programmer enthusiasts to
explore the development of natural user interfaces. This beta SDK includes an application programming
interface (API) and sample code.
Important The Kinect for Windows SDK Beta from Microsoft Research is licensed only for
noncommercial use. By installing, copying, or otherwise using the beta SDK, you agree to be bound by
the terms of its license. Read the license.
See Also FAQ for Kinect for Windows SDK Beta
In this guide
PART 1—Introduction to This Beta SDK
PART 2—The NUI API: An Overview
PART 3—The Audio API: An Overview
PART 4—Resources
Programming Guide – 2
Getting Started with the Kinect for Windows SDK Beta from Microsoft Research
Beta 1 Draft Version 1.1 – July 22, 2011
Disclaimer: This document is provided “as-is”. Information and views expressed in this document, including URL and other
Internet Web site references, may change without notice. You bear the risk of using it.
This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may
copy and use this document for your internal, reference purposes.
© 2011 Microsoft Corporation. All rights reserved.
Microsoft, DirectShow, Direct3D, DirectX, Kinect, MSDN, Visual Studio, Win32, Windows, Windows Media, Windows Vista,
and Xbox 360 are trademarks of the Microsoft group of companies. All other trademarks are property of their respective
owners.
Programming Guide – 3
Getting Started with the Kinect for Windows SDK Beta from Microsoft Research
Beta 1 Draft Version 1.1 – July 22, 2011
Contents
PART 1—Introduction to This Beta SDK ............................................................................................................. 4
Welcome ........................................................................................................................................................................................................... 4
System Requirements .............................................................................................................................................................................. 4
Skills for Developers ................................................................................................................................................................................. 5
Downloading and Installing the Beta SDK .......................................................................................................................................... 6
Loading the Microsoft Kinect Drivers ............................................................................................................................................... 6
Configuring the Development Environment ................................................................................................................................. 7
Implementing a Managed Application ........................................................................................................................................ 8
Implementing a C++ Application .................................................................................................................................................. 8
Experimenting with NUI and Kinect Sensors ..................................................................................................................................... 9
Protecting Your Kinect Sensor While You Experiment ............................................................................................................ 10
Experimenting with This Beta SDK ................................................................................................................................................... 11
Understanding How Kinect Sensors Work ............................................................................................................................... 12
Beta Testing Recommended Practices ....................................................................................................................................... 12
Samples in the Beta SDK .......................................................................................................................................................................... 13
PART 2—The NUI API: An Overview ................................................................................................................. 14
Kinect for Windows Architecture .......................................................................................................................................................... 14
The NUI API .................................................................................................................................................................................................... 15
NUI API Initialization .............................................................................................................................................................................. 15
Sensor Enumeration and Access ................................................................................................................................................... 15
Initialization Options .......................................................................................................................................................................... 16
NUI Image Data Streams: An Overview ......................................................................................................................................... 17
Color Image Data ................................................................................................................................................................................ 17
Depth Data ............................................................................................................................................................................................. 18
Player Segmentation Data ............................................................................................................................................................... 18
Retrieving Image Information ........................................................................................................................................................ 18
NUI Skeleton Tracking ........................................................................................................................................................................... 19
Retrieving Skeleton Information ................................................................................................................................................... 20
Active and Passive Skeletal Tracking .......................................................................................................................................... 21
NUI Transformations .............................................................................................................................................................................. 22
Depth Image Space ............................................................................................................................................................................ 22
Skeleton Space ..................................................................................................................................................................................... 23
Floor Determination ........................................................................................................................................................................... 23
Skeletal Mirroring................................................................................................................................................................................ 24
PART 3—The Audio API: An Overview .............................................................................................................. 25
About the Kinect Microphone Array ................................................................................................................................................... 25
Beta SDK C++ Audio API ......................................................................................................................................................................... 25
Raw Audio Capture ................................................................................................................................................................................. 26
KinectAudio DMO ................................................................................................................................................................................... 26
Beta SDK Managed Audio API ............................................................................................................................................................... 26
How to Use Microsoft.Speech with the Kinect Microphone Array ......................................................................................... 27
PART 4—Resources .............................................................................................................................................. 28
Tips and Troubleshooting ........................................................................................................................................................................ 28
Terminology for Kinect and NUI ........................................................................................................................................................... 29
References ...................................................................................................................................................................................................... 30
APPENDIX ............................................................................................................................................................. 31
How to Build an Application with Visual Studio Express ............................................................................................................ 31
Programming Guide – 4
Getting Started with the Kinect for Windows SDK Beta from Microsoft Research
Beta 1 Draft Version 1.1 – July 22, 2011
PART 1—Introduction to This Beta SDK
Welcome
The Kinect™ for Windows
®
Software Development Kit (SDK) Beta from Microsoft Research is a starter kit
for applications developers. The kit makes it easier for the academic research and enthusiast
communities to create rich experiences by using Kinect sensor technology on computers that are
running Windows 7.
The beta SDK includes:
Drivers, for using Kinect sensor devices on a computer that is running Windows 7.
APIs and device interfaces, together with technical documentation for developers.
Source code samples.
System Requirements
You must run applications that are built by using the beta SDK in a native Windows environment. You
cannot run applications in a virtual machine, because the Microsoft Kinect drivers and this beta SDK
must be installed on the computer that is running the application.
Supported Operating Systems and Architectures
Windows 7 (x86 or x64)
Hardware Requirements
Computer with a dual-core, 2.66-GHz or faster processor
Windows 7–compatible graphics card that supports Microsoft
®
DirectX
®
9.0c capabilities
2 GB of RAM
Kinect for Xbox 360
®
sensor—retail edition, which includes special USB/power cabling
Software Requirements
Microsoft Visual Studio
®
2010 Express or other Visual Studio 2010 edition
Microsoft .NET Framework 4 (installed with Visual Studio 2010)
For C++ SkeletalViewer samples:
DirectX Software Development Kit, June 2010 or later version
DirectX End-User Runtime Web Installer
For Speech sample (x86 only):
Microsoft Speech Platform - Server Runtime, version 10.2 (x86 edition)
Microsoft Speech Platform - Software Development Kit, version 10.2 (x86 edition)
Kinect for Windows Runtime Language Pack, version 0.9
(acoustic model from Microsoft Speech Platform for the beta SDK)
Programming Guide – 5
Getting Started with the Kinect for Windows SDK Beta from Microsoft Research
Beta 1 Draft Version 1.1 – July 22, 2011
Skills for Developers
You can use this beta SDK for your personal programming interests. The beta SDK is specifically for
computer scientists and scientific researchers in academia. It is also provided for technical enthusiasts
who want to creatively explore natural user interface (NUI) possibilities with the Kinect sensor and
related technologies.
Experimenting with Kinect and NUI
This beta SDK enables exploration and experimentation with the following features:
Skeletal tracking for the image of one or two persons who are moving within the Kinect sensor’s
field of view.
XYZ-depth camera for accessing a standard color camera stream plus depth data to indicate the
distance of the object from the Kinect sensor.
Audio processing for a four-element microphone array with acoustic noise and echo cancellation,
plus beam formation to identify the current sound source and integration with the
Microsoft.Speech speech recognition API.
Related concepts and programming approaches are explored in this programming guide.
Application Development
To take advantage of the experimental capabilities and features in this beta SDK, developers should be
familiar with the following:
Development environment and languages
Visual Studio 2010
To work with the beta SDK and to develop your own applications with the beta SDK, you can
use Visual Studio C# 2010 Express, Visual Studio C++ 2010 Express, or any other version of
Visual Studio 2010. Visual Studio Express is available at no cost from the Microsoft website. If
you are new to Visual Studio, see the Visual Studio 2010 Getting Started Guide.
C# or C++ languages
Samples are available in both C# and C++. For information about programming in these
languages, see Visual C# Developer Center and Visual C++ Developer Center.
Application development for Windows 7
This beta SDK takes advantage of the functions and features of the SDK for Windows 7. If you are
new to Windows development, see Beginner Developer Learning Center for Windows Development.
Note You should not use this beta SDK to develop prototype applications with the intent of porting
those applications to the Xbox 360 console. There are numerous architectural, performance, and
behavioral differences between Xbox 360 and Windows. These differences affect how the Kinect
technology is implemented for Windows 7. As a result, design decisions and performance characteristics
are quite often very different between the two platforms. We recommend that you use Xbox 360
development kits (XDK hardware and software) to create Xbox 360 applications