Virtual conversation

Niels ter Heijden

Human-Computer Interaction

Project

Project: Social Phobia

Title: Virtual conversation

Enable ling conversation between a virtual human and a social phobia patient

Literature Study: Virtual Conversation

Research Goals

The research area of social phobia treatments using VR is still fairly new and unexplored. Research did show that VR can be helpful and effective for treatment even with fairly basic virtual environments and avatars. But the real social aspect, communicating with others, is still for the most part left aside because of technical difficulties and the amount of active involvement needed from the therapist to operate the system. The goal therefore is to automate the social aspect, communicating with the patient, in a way that frees up time for the therapist but still gives him “absolute” control over the conversation. Communication is done on multiple levels between humans but the verbal component is one of the most conscious of the channels used and therefore will be the initial focus. This gives three main problem areas where solutions need to be found firstly listening and converting spoken text into computer comprehensible code. After this the avatar needs to think up a response that is suitable to the situation either by comprehending the sentence spoken by the patient or either by trickery with preprogrammed responses. And finally the avatar needs to respond to the patient in a way that feels natural to humans and therefore does not break the feeling of presence.

How to convert speech to computer comprehensible code is being actively research by many people. Many methods and techniques exist and no one at this point in time has found the holy grail. With the holy grail being a program that can in any circumstance without training convert speech to text mostly flawlessly. This would mean that the automated avatar would fail from the start to relieve this problem a bit some techniques need to be used to improve the recognition this could be done for example by training the system or limiting the number of words or sentences that needs to be recognized. For the solution of this problem a already existing and trained package will be used that means workarounds for the problems and limitations this package has needs to be found and implemented.

The main focus of the project will be on the comprehending and selecting the appropriate response for the patients input to the system. Comparing different techniques to see which one gives the best or most feeling of presence to the patient but also gives the therapist the means to influence the anxiety level created. The chosen method also needs to be able to work with and disguise the weak points of the other part of the system such as the speech recognition and speech synthesis. The total package should deliver a social interacting virtual human that response realistic to the patient and creates a high feeling of presence in the virtual environment and can be used for treating social anxieties.