A Kinect Application That Knows When You're Sad and Wants to Talk About it With You

A Kinect Application That Knows When You're Sad and Wants to Talk About it With You

The new Kinect app acts as your virtual therapist. But just how helpful is SimSensei? 

Written by Michael Thomsen (@mike_thomsen)

 

Depression is epidemic in the United States, with roughly nine percent of Americans meeting the clinical criteria for depression in any given month according to Harvard Medical School. University of Toronto psychiatrist Edward Shorter explains this phenomenon as distinct from major depression, or melancholia, something that is endured by most people without reaching the level of serious debilitation. "[Non-melancholic people] get out of bed to go to work every day, they don't cry all the time," he explained to The Daily Beast. "But instead they have other stuff, and this is the crucial point. They're anxious, for example. They're fatigued. They have all kinds of somatic pains, body pains that come and go. And they tend to obsess about the whole thing."

A new system called SimSensei proposes to help identify people suffering from these depressive symptoms and offers preliminary conversational exchanges with a computer character. The program relies on Microsoft's Kinect camera to take full 3D video of its subjects, analyzing speech patterns and facial expression to detect symptoms of sadness. The interface uses a polygonal model therapist who asks leading questions of users. "I'm not a therapist," the computer avatar says, "but I love to learn about people and I'd love to learn about you." 

 

The technology has been designed to be as neutral as possible but there is something innately strange about talking about your feelings to a computer avatar.

 

The questions posed are simple and unobtrusive—"How are you doing?" "Where are you from originally?"—but any signs of potentially unpleasant emotions can trigger a branch in the line of questions that ask more pointed questions about the subject's current state. The program can detect prolonged downward glances, overly animated hand movements, and when inconclusive answers are given it will pause to give the user time to keep talking through an uncomfortable subject.

SimSensei was originally designed for military use with funding support from Darpa, hoping to create a mental health support portal that would have been accessible to veterans in any number of health clinics. The model would work like a therapeutic photo booth, offering full privacy and customization, with the patient able to choose their therapist's model. The kiosk technology would work in conjunction with a care regiment laid out by a human therapist, and would also connect to Sim Coach, a version of the technology for the home that would monitor the patient and collect data for his file to help the care provider chart his or her progress.

RELATED: Sad Songs to Listen to When You're Feeling Down

Stay Connected with
Complex Tech
Tags: microsoft-kinect, apps, mental-health
blog comments powered by Disqus