Hand Gesture can be used an effective alternate compared to traditional methods of interacting with intelligent devices. Elderly People who are not tech savvy as well as people who are unable to move because of their disabilities can operate all the intelligent devices from their place providing an easy way of accessing things. This method provides a robust approach where in interfacing is very easy with all the intelligent devices because of its simplicity. Using sophisticated software packages and Microprocessors, an interoperable and portable design has been proposed where in the captured hand gestures are compared with the templates stored, to produce triggers for operating machines.
Keywords |
| Hand Gesture, Gesture Recognition,
Templates, Controlling remotely, image Processing |
INTRODUCTION |
| Hand gestures are one of the universal languages
recognized by all and often communicated in unknown
environment. But hand gestures are rarely used for
interacting with intelligent devices like computers. Only
traditional input devices like Remote controls, Buttons,
Mouse are frequently used. Hand gestures can be
effectively used for replacing the traditional devices so
that seamless integration can be done for assisting the
elderly to operate those intelligent devices. |
| In spite of all the improvement in technology many
people still find very difficult to operate computers and
other intelligent devices using traditional methods.
Since hand gesture is spontaneous and natural way of
communication even the elderly and disable people can
easily interact with all the intelligent devices. |
| But Recently many people have recognized the impact of replacing traditional input devices with natural
human oriented language like signs, words etc.. Tech
neology has made the process of gesture recognition for
man-machine interface easier since an array of low cost
sensors and processors are available even for the stand
alone computers helping the end user computing. |
| (Vladimir I. Pavlovic et.al) have suggested visual
interpretation of hand gesture for human computer
interface. (Elena Sánchez-Nielsen et. al) have used 26
hand postures for human machine interaction. |
| There are many active researches that have been carried
out in this field. |
| We propose a method of robust and fully automated
remote control for home appliance using free hand
gesture. It’s not new to study on a remote control for
intelligent devices using human’s free hand gesture
instead of cumbersome hand-held wireless remote
controllers. Nevertheless, to the best of our knowledge,
there exists no commercial remote controller that uses
free hand gesture because it requires high robustness to
various illumination changes in living room and full
automation without any constraint or direct intervention
of users. |
| In the first part of this paper, the algorithm for detection
of hand gesture has been proposed. In the later part
methodology used for handling the intelligent devices
using the detected hand gestures has been discussed. |
HAND GESTURE RECOGNITION |
| The First step in the gesture recognition cycle is the
preparation of an image database. Images can be in
various shapes, different formats and may be in
different scales. The images collected from various
sources are first made uniform by converting them in to
gray scale by removing the background. |
| The algorithm robustness depended on the number of
persons using it. If different persons are using the application more images are needed in the database. If
the same person is using the application database size is
less since the images are almost of same texture. |
| Shape Recognition |
| There are two phases involved in shape
recognition. In the first phase computer forms various
shapes called templates. In the second phase the shapes
are compared with the actual image captured to execute
various programs which will inturn run the
corresponding triggers. |
HAND TRACKING ALGORITHM |
| The hand tracking algorithm proposed here is very
fast, reliable and effective and can be used for many real time applications. The proposed Algorithm uses motion
vector and skin color to recognize the hand gesture.
First the images are captured from the camera. The
camera we are proposing to use is a RGB camera. The
captured image is then converted to gray scale. Then we
compute the frame difference between the colored
frame and the one in gray scale. |
| The image which has non-zero pixel values in the
regions where motion has taken place. A thresholding
algorithm gives a binary image with white pixels
indicating the region of motion. A value of 30 is chosen
for the threshold for monitoring the difference of the
frames as mentioned above. The value 30 gives us the
white pixels to track the position of the hand. |
| Skin Segmentation |
| Before performing the skin segmentation we
convolve the actual image with 5x5 Gaussian filters and
then we have to scale this filtered image one half in
each dimension to eliminate any noise in the image
pixels. Then the subsequent image processing
operations are performed. For every available pixel in
the frame we perform foreground mask as follows |
 |
| Even though the background subtraction discussed
above works well, there are conditions like a coffee
cup, shirt sleeves or any other items placed on desk can
be detected as hand. In order to avoid those situations
and add to more some more flexibility to the system, a
skin pixel detector has been implemented to further
filter the foreground data. |
| From the above mentioned procedures we can calculate
the probability of a RGB color that belong to hand and non hand gesture as follows. |
 |
SIMULATION |
| A Computer based simulation was first performed as a
pilot study to check out the feasibility of the study. This
was also created so that the elderly people who find
very difficult to operate mouse and key board because
of paralysis and other physical disabilities can use hand
gesture to perform various basic computer operations
like opening media player for playing songs or video,
opening file, opening and closing computer, restarting
computer etc. |
| Application software was created with the help of Matlab. Image processing was done with the help of
Matlab IDE. After the image has been recognized
various system triggers are programmed using vb.net. |
| APPLICATIONS |
| 1. Can be used by disabled and elderly people for
operating many intelligent systems |
| 2. Can be used to operate computer/ intelligent
devices in cars while driving |
| 3. Can be used in military applications for
remotely operating a computer |
EMBEDDED SYSTEM DESIGN |
| An effective embedded system design has been created
for assisting the elderly people in using the intelligent
devices for day to day home use. The block diagram of
the same is given below |
| The hand gestures are fed in to the camera which in turn
transfers the image in to a PC. The PC performs image
comparison with an already existing image database and
image corresponding to the matched gesture are fed to
the embedded control platform which is nothing but
ARM 7. The Arm 7 processor in turns performs the
controlling of all home devices using RS232 and Zigbee
protocol. All the devices in the home are attached with a
receiver which receives the signal from Zigbee and
performs the corresponding operations. |
SUMMARY AND FUTURE RESEARCH |
| The future research that has been planned is to control
the home appliances even from outside the home using
internet or other sophisticated network facilities. This
research can also used in military appliances for
securely operating importance devices remotely like
operating an cannon from safe place so that we can stop
shedding of human lives. |
| |
Figures at a glance |
 |
 |
 |
 |
 |
| Figure 1 |
Figure 2 |
Figure 3 |
Figure 4 |
Figure 5 |
 |
 |
 |
 |
| Figure 6 |
Figure 7 |
Figure 8 |
Figure 9 |
|
| |
References |
- Aggarwal J.K. and Park, S. 2004. âÃâ¬ÃÅHuman motion: Modeling and recognition of actions and interactionsâÃâ¬ÃÂ. in Proc. 2nd Int. Symp. 3D Data Process., Vis., Transmiss., 640âÃâ¬Ãâ647.
- Berman, S. Friedman, J., Bakir, G. and Flash, T. 2008. Action Identification for Teleoperation Based on Object - Action Abstraction, IEEE SMC International Conference on Distributed Human-Machine Systems (DHMS).
- Berman, S. and Stern, H. 2012. âÃâ¬ÃÅSensors for Gesture Recognition SystemsâÃâ¬ÃÂ, IEEE Transactions on Systems, Man, and Cybernetics- Part C 42(3), 277-290.
- Garg, P. Aggarwal, N. and Sofat, S. 2009. âÃâ¬ÃÅVision based hand gesture recognitionâÃâ¬ÃÂ. in Proc. World Acad. Sci., Eng. Technol. 49, 972âÃâ¬Ãâ977.
- Frolova, D., Stern, H. and Berman, S. 2012. âÃâ¬ÃÅMost Probable Longest Common Subsequence for Recognition of Gesture Character InputâÃâ¬ÃÂ. IEEE Systems Man and Cybernetics, part B, September (in press).
- Mitra S. and Acharya, T. 2007. âÃâ¬ÃÅGesture recognition: A surveyâÃâ¬ÃÂ. IEEE Trans. Syst., Man, Cybern. 37(3), 311âÃâ¬Ãâ324.
- Rautaray, S.S. and Agrawal, A. 2012. âÃâ¬ÃÅVision based hand gesture recognition for human computer interaction: a surveyâÃâ¬ÃÂ. Artificial Intelligence Review, 1-54.
- Stern, H., Frolova, D. and Berman S., 2010. Hand Gesture Recognition for TV Remote Control using Tree-Based Ensemble and LCS Classifiers, WORLDCOMP'10 The 2010 World Congress in Computer Science, Computer Engineering, and Applied Computing.
- Stern, H., Shmueli, M. and Berman, S. 2013. âÃâ¬ÃÅMost Discriminating Segment - Longest Common Subsequence (MDSLCS) Algorithm for Dynamic Hand Gesture ClassificationâÃâ¬ÃÂ. Pattern Recognition Letters (Special issue on 'Smart Approaches for Human Action RecognitionâÃâ¬Ãâ¢), (in press).
- Wachs, J., Kolsch, M., Stern, H. and Edan, Y. 2011. âÃâ¬ÃÅVisionbased hand gesture applications: Challenges and innovationsâÃâ¬ÃÂ. Commun. ACM 54(2), 60âÃâ¬Ãâ71.
- Yilmaz, A. Javed, O. and Shah, M. 2006. âÃâ¬ÃÅObject Tracking: A SurveyâÃâ¬ÃÂ. ACM Comput.Survey 38(4), 1âÃâ¬Ãâ45, 2006.
|
Menu