WASHINGTON (TIP): Scientists are developing new technologies that will allow computers to recognise non-verbal commands like gestures, body language and facial expressions.

For most people, using a computer is limited to clicking, typing, searching, and, thanks to Siri and similar software, verbal commands.”Compare that with how humans interact with each other, face to face -smiling, frowning, pointing, tone of voice, all lend richness to communication,” the researchers said. The new project titled Communication through gestures, expression and hared perception’ aims to evolutionise everyday interactions between humans and computers. “Current humancomputer interfaces are still everely limited,” said Pro essor Bruce Draper, from Colorado State University, who is leading the project. “First, they provide essentially one-way communication: users tell the computer what to do. This was fine when computers were crude tools, ut more and more computers re becoming our partners and ssistants in complex tasks. Communication with computers needs to become a two-way dialogue,” said Draper.

The team has proposed cre ating a library of what are called Elementary Composable Ideas (ECIs). Like little packets of information recognizable to computers, each ECI contains information about a gesture or facial expression, derived from human users as well as a syntactical element that constrains how the information can be read.

To achieve this, the researchers have set up a Microsoft Kinect interface. A human subject sits down at a table with blocks, pictures and other stimuli. The researchers try to communicate with and record the person’s natural gestures for concepts like `stop’ or,
`huh?’.

“We don’t want to say what gestures you should use,” Draper said. “We want people to come in and tell us what gestures are natural. Then, we take those gestures and say, OK, if that’s a natural gesture, how do we recognise it in real time, and what are its semantics? What roles does it play in the conversation? When do you use it?When do you not use it?” he said.

According to the project proposal, the work could someday allow people to communicate more easily with computers in noisy settings, or when a person is deaf or hard of hearing, or speaks another language.