Journal of Global Research in Computer Sciences
MenuISSN: 2229-371X
ISSN: 2229-371X
| Nidhi Srivastava and Dr. Sipi Dubey RCET, Bhilai, |
| Related article at Pubmed, Scholar Google |
Visit for more related articles at Journal of Global Research in Computer Sciences
In this paper, we demonstrate that we can use non-invasive physiology sensing to detect stress and lying, within the context of Artificial Neural Network. We show how simply derived non-invasive physiological features such as voice pitch variation, and heart rate variability are correlated to a number of high stress situations found in real life. Using these features, we can develop simple linear models that can be used to identify stress and bluffing.
INTRODUCTION |
| The National Institute of Occupational Safety and Health states that stress is becoming the most prevalent reason for worker disability. A 1992 UN report called job stress "The 20th Century Epidemic", while the World Health Organization stated in 1996 that stress was a "World Wide Epidemic". Researchers estimate job stress costs American industries between $200 and 300 bil-lion annually. Given the mounting social costs of stress, the possibility of automatically identifying and monitoring stress levels for intervention purposes is compelling. This is particu-larly true for people who regularly work in high stress environ-ments, such as financial traders or emergency workers. In such situations, faulty performance as a result of acute stress can lead to million dollar losses of or even the loss of life. |
| Given the implications of stress on work performance, there has been recent interest in monitoring the work performance of in-dividuals under stress. Specifically, studies in behavioral finance present psychophysiological evidence that even the most sea-soned securities trader exhibits significant emotional response as measured by elevated levels of skin conductance and cardiovas-cular variables during certain transient market events [Lo & Repin 2002, Lo et al. 2005]. Other studies have corroborated the evidence linking emotion with trading performance [Steenbarger 2002]. These studies indicate that psychophysiology and stress are intimately linked, and it is possible to infer one from the other. |
| We wish to demonstrate that non-invasively derived physiology and behavioral sensing can be correlated with various stress-ful events in poker tournaments. Specifically, we will be looking at subjective reports of stress levels, bluffing, and all-in situa- tions as outcomes and correlating these situations with physiol-ogy features aggregated over the hand. |
BACKGROUND |
| Whereas traditional clinical physiology monitoring focuses on identifying accurate physiological responses (e.g. ECG traces during a heart arrhythmia), long-term monitoring enabled by minimally invasive sensing provides the ability to correlate con-textual measures over time to a personâÃâ¬ÃŸs behavior and internal state. Research has shown that it is possible to correlate mini-mally invasive physiology measures to identify notions of a personâÃâ¬ÃŸs intentions and affective state such as interest, happi-ness, and stress [Picard 2001, Picard et al. 2001]. |
| In particular, accurately identifying human intention such as lying has been particularly contentious given its inherently sub-jective nature. The concept of the „lie detectorâÃâ¬ÃŸ has always cap-tured the imagination and interest of the popular press since it was invented over a hundred years ago in 1902 by James Mackenzie. The prototypical version invented by Mackenzie, was also called the polygraph tool because it looked at a range of physiological phenomena such as a personâÃâ¬ÃŸs heart rate, breathing rate, blood pressure, and skin conductance while a person is questioned. Though a report released in 2002 by America's National Academy of Sciences indicated that poly-graphs are not completely reliable (better as a measure of stress than veracity), there is no doubt that a variety of physiological phenomena can be correlated to lying. |
| If such a ambiguous thing as telling the truth can be predicted and correlated to basic physiological features such as those ana-lyzed by a polygraph, it is not a large leap of faith to consider using physiology to quantify a personâÃâ¬ÃŸs other internal states such as stress, frustration, interest level, excitement, attention, drowsiness, and affect [Picard 2001, Picard et. al 2001]. In fact, there is research to show that these things can in fact be accu-rately quantified, and furthermore that these measures can dem-onstrate a high degree of correlation and concordance among interacting groups of people, whether being stimulated while passively watching a movie [Madan et al. 2004] or the highly involved interaction dynamics of a psychiatric patient and a therapist [Marci 2002]. |
EXPERIMENTAL METHODOLOGY |
| Artficial Neural Network Subject Selection and Protocol: |
| The study population was drawn from the different community who answered recruitment emails sent to the mailing list. Approval and written consent was obtained from all participants. All of the subjects were college-age, healthy adult male individuals drawn from the undergraduate and gradu-ate schools All subjects were self- purported poker afi-cionados who play semi-regularly, though the skill level ranged from beginners to players who play for profit in high-stakes games. For the poker physiometrics study, the physiology and behav-ioral state of players were taken over 31 tournaments, represent-ing 401 hands of poker. Every subject was paired up with an-other player to play real live-money games of no-limit Texas HoldâÃâ¬ÃŸem in heads-up style tournaments. While the players were engaged in the tournament, which typically ranged in duration between 10 -30 minutes, the physiology and behavioral state of each player was recorded and annotated using a LiveNet system, a flexible sensing platform [Sung 2005]. |
| As financial interest was instrumental in generating the stressful situations for this study, the players were asked to play with real money buy-ins in winner-takes-all tournaments. Subjects were paired up to other players with similar financial risk thresholds (with a buy-in between $5 to $20/tournament). The playerâÃâ¬ÃŸs self-reported skill level and risk tolerance were also recorded in order to match players as well as to provide additional baseline information. |
| The BioRecord application was used to record multimodal physiology through the duration of each hand, as well as to timestamp interesting annotations during the match. This data included heart rate, skin conductance, temperature, heat flux, speech features, and movement data. The recordings were done on a per hand basis per player. The multimodal data streams of each player were also synchronized to each other. |
| Artificial Neural Network Subjective and Objective Outcome Data: |
| In addition to the physiological and behavioral data collected, each subject was asked to fill out a tournament diary at the end of each hand. This form asked information regarding both sub-jective ratings of the playerâÃâ¬ÃŸs mental state as well as objective information from the hand. The subjective measures included a playerâÃâ¬ÃŸs self-reported interest, excitement, happiness, and stress levels, all rated on a scale of 1- 10 (the subjects were asked to rate their current state relative to the typical range of emotional activity that they perceive playing poker). Objective outcome data on whether the individual won or lost, whether they bluffed (i.e. small bluff, medium bluff, large bluff, or semi-bluff), whether there was an all-in hand initiated, whether the hand was a bad-beat (defined as when a heavily mathematically favored hand results in a loss), and the amount of money wagered were also recorded. This combination of subjective measures as well as objective outcomes were correlated with the physiology col-lected. For the analysis, we focused on the situations in the tour-nament which are traditionally considered high stress (large bluffing, all-in, bad-beat, and subjective stress situations with a rating of greater than 7 on a scale of 1-10). |
| Artificial Neural network Human Subjects Approval: |
| To obtain human subjects approval, we needed to comply with Committee For this, we needed to provide documentation detailing the experimental protocol, subject con-sent procedures, and subject recruitment. It was necessary to explicitly describe every type of data collected from the subjects as well the equipment used in the study. In order to comply with these regulations, we had to make sure that personally identifi-able subject information was not included in the data that was collected. |
DATA ANALYSIS |
| The physiological and contextual data for each hand of a par-ticular tournament was individually segmented and synchro-nized. For each hand, a variety of features derived from the heart rate, skin conductance, temperature, audio, and accelerometer data was calculated over the duration of the segment, which typically lasted between 30 seconds for an uneventful hand up to a few minutes for an involved hand. |
| Given the short duration of each hand, the window for these feature calculations was the entire hand length. Standard heart rate measures such as the average rate, standard deviation in RR intervals, min/max ratios, and frequency-domain energy and entropy ratios were calculated from the hands. For skin conduc-tance, heat flux and temperature, average tonic level, slope variation, and threshold peaks counts were calculated. For the audio features, energy, fundamental format frequency, spectral entropy of voiced segments, and variation in these measures over time were calculated, as well as speaking dynamic features such as fractional speaking time, and voicing rates. Motion fea-tures such as energy, max/min ratios, entropy, and spectro-graphic features were calculated. |
| For each individualâÃâ¬ÃŸs hand, the physiology and behavioral fea-tures were computed over the entire hand during the course of play. Two typical hands are shown in the figure below. |
 |
| The hand on the right represents a non-eventful hand synchro-nized between two players in a tournament. We can see that the skin conductance did not change very much, and there were intermittent spurts of talking. In contrast is the last hand of the tournament between the two same players. The blue player first starts to become stressed (as indicated by the small rise in skin conductivity midway through the hand). A little bit later, he goes all-in, prompting a very large ramp in skin conductance by the red player while he ponders his options. Also note the very large differences in heart rate variability between the two hands (there was much more variability in the stressful hand). Finally, both the magnitude and amount of talking in the stressful hand was noticeably larger in the non-stressful hand. |
| The features were extracted by partitioning the data of a particu-lar hand into windows (specifically before the resolution of the hand as well as after hand). This was because the voicing and motion dynamics were very different in these regimes since people tend to be more quiet and motionless in stressful situa-tion before hand resolution, and exactly the opposite following the end of the hand. |
| All calculated features over each hand were normalized and z-scored to find the correlations to the outcomes. There were strong physiological correlations to most of the high-stress situa-tions during play, most notably for all-in plays as well as for large bluffs. For large bluffs in particular, it was found that fre-quency- based heart variability features such as the LF/HF ratio, fraction speaking time, and motion energy were the most highly correlated features, as shown in Table 1 below: |
 |
| The LF/HF ratio PSD feature was often interpreted as a measure of the relative sympathetic to parasympathetic activity of the autonomic nervous system. The LF power was generated mainly from sympathetic activity, with baroreceptor modulation being a major component in LF power. The HF power, in contrast, was derived from vagal, or parasympathetic activity which is modu-lated by respiration. Thus, LF/HF ration represents a good indi-cator of sympathetic-vagal balance, and is used to assess the balance of the autonomic nervous system. As discussed in Sec-tion 4.5, studies have shown that mental stress increases LF activity and decreases HF activity. It seems that bluffing, insofar as it induces a stressful situation on a subject, results in a sig-nificant increase in LF power, and a slight decrease in HF power. Because of the increase in LF power, the LF/HF ratio was seen to increase during bluffing and other stressful situa-tions. |
| The fraction of speaking time feature was calculated over the window of the recorded hand with 15 seconds from the end removed (which contains changes in physiology and behavior following the resolution of the hand). This feature was nega-tively correlated to bluffing. |
| The third feature, motion energy, was the mean energy calcu-lated over the hand frame and was also negatively correlated to bluffing. We find that there was a significant reduction in the mean motion energy during a hand where the player was bluff-ing. |
| The later two features (speaking time and motion energy), are anecdotally well known to be associated with bluffing in the poker community [Caro 2003]. Specifically, when a person was bluffing, he or she consciously tries to reduce all interaction, including to stop speaking and to become very still, so as not to give a reason for the opponent to call the bluff. It is reassuring that this intuition holds true quantitatively in the features that were correlated well with bluffing. |
 |
| All-in situations, where a player was faced with, or initiated a play where the rest of their money was in jeopardy, was another stress situation. For all-in hands, skin conductance peaks were the most highly correlated features. Below is a figure showing a hand where an all-in play was initiated a little after half-way through the hand. This signal can then be smoothed by applying a low-pass filter, and then thresholded at different levels to get a sense of the number of peaks and valleys in the original skin conductance signal. Small thresholds catch smaller peaks, and larger thresholds can catch a very large skin conductance peak which turns out to be well correlated to high stress events.The most basic HRV measure of the standard deviation of RR intervals was the second most correlated features. Voice energy also ended up being the most highly correlated voice feature. |
 |
| The bad beats results were less correlated to physiology and behavior during the hand, but very correlated to behavior after the hand has transpired. All bad beats had voice and motion energy as well as the standard deviation in the location of the largest auto correlation peak (essentially pitch variation). This makes sense; while the hands varied widely, the defining char-acteristic of a player being beat was to be very vocal (and whiney about it), as well as to move around in an agitated way following a hand. Statistical significance was an issue since there were relatively few recorded bad beat plays (around 19 instances in 401 hands of poker). |
 |
| Many, but not all of the bluffing/all- in/bad-beat situations de-scribed above were highly stressful situations. We define a stressful hand as any in which the player subjectively rated his or her stress to 7 or above (from a scale of 1 to 10). For this measure, a number of the same features described above end up being highly correlated. Skin conductance features, with the number of large-threshold peaks end up being the best feature. It was observed that many hands had small ramps of skin conduc-tance, but large ramps of skin conductance almost always corre-sponded to highly stressful events. The standard deviation in the largest autocorrelation peak (a measure of pitch variation in the voice) was also highly correlated. The LF/HF ratio shows high sympathetic arousal in the heart rate variability for stressful situations. |
 |
STRESS AND BLUFFING CLASSIFICATION |
| The results in the previous section indicate that there were a variety of skin conductance, heart rate variability, voice, and movement features that can be correlated to various stressful outcomes. The question to ask is if these simple physiology features can be used to create accurate classifiers that can dis-criminate between these outcomes. Of particular interest in a more general setting outside of poker tournaments is stress clas-sification, as well as potentially being able to identify when a person is lying (bluffing). |
 |
| These results seem to conform to the literature in polygraphy (lie-detection techniques) [Podlesny & Raskin 1977]. The vari-ous physiological features were well understood to correlate well to stress through the autonomic nervous system reactions. Lying was capable of being detected insofar as a person be-comes stressed at having to lie. In the case of poker, there was obviously a high correlation between the two (around R=0.75), as the player was motivated to prevent the potential loss of money. However, for the levels of play in the tournaments that were conducted, it was possible that bluffing becomes some-thing that people adjust to since it was a relatively common occurrence. |
CONCLUSIONS |
| These initial results indicate that it is possible to correlate stress, lying (in the context of bluffing), and interest with a variety of physiological features. Using Artificial Neural Network, we have been able to identify high stress situations to within about 82% accuracy. We can even detect lying with about 71% accuracy. Essentially, we demonstrate that we can identify these events from simple ag-gregated physiological features acquired during the duration of the events in question from non-invasively derived sensing. |
References |
|