Estudiante

Bilateral Virtual Control Human-Machine with Kinect Sensor
Cristian Gallardo, Jaime Santana, Johana Villacrés, Renato Toasa, Gabriel Reyes, Javier Vargas, Tannya Naranjo, Andrea Sotelo and Víctor H. Andaluz



Abstract— This work presents the virtual bilateral interaction between Man-Machine through the Kinect sensor, allowing the user to control and monitor Windows programs by gestureswithout the need of using peripheral components which means that “You Are The Control”. In order to develop this work the body is tracked, which lets the user to manipulate the computer depending on their needs. In this application two functions of the Kinect sensor are used: the depth camera and the Skeleton Tracking, which is why object oriented programming on Visual Studio 2010 (C#, WPF) was used.Additionally, several experimental tests were carried out, through which the performance of the proposal was verified.
I. INTRODUCTION

The Kinect sensor (Fig. 1) is a device created by Microsoft as a complement for the XBOX 360 video game console. Nowadays, it has surpassed the boundaries of the world of leisure and has become a powerful development platform at a reasonable price [1]. Thissensor has got an engine-revolving base; four microphones which recognize voices separately; three cameras with a movement sensor (VGA movement sensor camera, and double 3D depth camera) [1, 2]. The depth camera captures depth data and is used to know how far from the sensor a pixel is, in order to calculate the distance based on those data [3].

used to map the robot’s work area [6, 7, 8, 9, and10]. [9] presents a method to distinguish between obstacles and the manipulator when they share the same workspace. The Kinect Sensor is used as a capturing device. [6] shows the calibration of Kinect depth and image sensors, and then uses the depth map to control the altitude of a quadrotor helicopter. [10] discusses the feasibility and design behind a simple robotic skeleton which utilizes theKinect to mimic human movements in real-time. [8] presents a mobile robot localization system for an indoor environment using a Kinect sensor; b) image processing which is being used for the detection and recognition of gestures, which is essential for the man-machine interaction [11, 12, 13, 14, 15, 16]. In [12] a method to recognize gestures through the Kinect de0pth camera is proposed, the depthimage is used for background removal, followed by generation of the depth profile of the subject. [14] presents how to recognize several predefined gestures by using a sequence of depth images which are obtained from Kinect. In [15] the Kinect depth sensor is used to detect and track an observer’s skeletal joints while the RGB camera is used for detailed facial analysis. [16] presents a novelmethod for fingertip detection and centers of palm detection distinctly for both hands using Kinect in 3D from the input image; c) education and people’s assistance [17, 18]. [17] presents video games with rehabilitation purposes based on Kinect. [18] presents some animation oriented to education by using the Kinect Sensor. The design of an application oriented to the control and virtual monitoringof a PC through Kinect Sensor is proposed in this work. The proposed multi-layers control scheme is mainly divide into four modules: 1) the top module is in charge of initializing the Kinect and identifying the user; 2) the next detection and movement module is responsible of capturing the user’s movements and of generating the necessary commands for the control of the different applications. It isimportant to establish that this module turns the captured points of body movement into events which are directly related to the input peripherals. ; 3) the applications module is represented by all of the applications that we want to control and monitor. In this work several applications are developed and focused on the manipulation and edition of images; the control and monitoring of Power...