A team of reserarchers at the University of East Anglia in Norwich, UK, has begun a progject to devise a system for lip reading using computers.. The system would take a video taped conversation and convert it to text.

This technology will become a valuable tool for crime fighting, security and a host of other applications such as mobile phone cameras and in-car speech recognition.

This futuristic system will replace the people who can read lips, such people are shrinking in numbers and even they are no always precise.

The real challenge is getting an accurate computer system that can read lips. The system will have to able to track the speaker's head in various poses, to extract features that describe the lips and then match the movements with corresponding text. This will also include developing a system with a huge lexicon for each language.

System is planned to be developed for English, and a few other languages like modern Arabic , which are quite expressive on the lips and therefore somewhat easier to lip read. Mandarin and such languages are very difficult to lip read.

 

 

 

Reply

Its interesting to see this developed, but if it is as poor as most speech recognition programs then it will likely still require a large amount of human input to correct the output. Either that, or it will require lengthy configuration for each person it is used on. However, I am sure all this has been worked out.

Unfortunately, if this works, its only a matter of time before it ends up in CCTV cameras and such, and we all end up speaking via morse code and semaphore. OK, a little drastic, but you never know

Reply

I can't say that I'm a big fan of this idea.

While I understand and appreciate the practical applications - such as helping to solve crimes by reading lips from security-camera footage - I also think about the possibilities that this technology has for the rest of us (as in: non criminals).

I don't care that I'm recorded on security cameras when I'm in the mall, grocery store, et cetera. But if I'm having a conversation with somebody in the store, I'd rather not have a lip-read transcript of that on anybody's record. I don't discuss anything illegal or whatever, but still. I'd like to maintain a tiny bit of privacy.

(One can argue that, if I talk in public, people around me can overhear. Yeah, of course they can. But they probably aren't transcribing and recording my conversation.)

Reply

True that it's some what infringing on our "privacy", but, you're in a store, there's lots of people walking around you/that can hear you, and they also may have some sort of voice recording system. Even the shops themselves might have voice recorders somewhere. I think it's pointless trying to get privacy in public.

I think it's a very good idea, and I'm all for it! If it can convert lip reading into text, then that will be very usefull for people who are def, as they can have it written down infront of them!

I hope they achieve it!

Reply

I agree with you here. The use of it at such places is pretty wrong and interfering. But still I am on for it, many reasons for that.


It's true that people can hear us but they aren't keeping a record of what we are asking, the next alley and they forget what they just heard, only if it was not something relative to their own lives. I remember spending the whole day thinking over something the person standing besides me said in a shopping centre.

Recording everything we say is interference in privacy for sure, let them hear us, fine, but record everything? In audio and in text? Now that is something I am against, but it's pretty much a good invention and this will definitely help deaf people, sign language isn't that easy and it takes loads of time to convey a simple message!

 

 

 

Reply

Interesting concept but way to many variables for a computer to handle this, because no on speaks the same in terms of how their lips move, or teeth are shown, or what muscles they used to speak and what gestures they used while speaking. They would have to use something like a retinal scanner type system to copy the patterns, but again the machine would only be guessing on patterns that ti doesn't recognize. Even the most experience human lip readers take years of practice and training to be able to master it, with computer they would need terabytes of information to work successfully. Either way it will take a decade or two to get a working model going because of different technologies they either have to build or incorporate to be successful.

Reply

i think it's a great idea but i don't know if it's going to be 100% accurate. like what SM has mentioned, there are too many things to consider, one of them is the language itself. i remember back in college, we were required by our psychology professor to attend experiments held by psychology majors for their thesis. one of the projects was lip reading but with test cases talking in our native tongue. it was fun and i was able to understand 4 out of 10 while others were just able to read just one or two. doesn't give me much of a power though i just wrote down the words that i thought was said in the video and created a sentence out of it. logic was part of it.

anyway, my point is that you also have to consider how these people would talk like. i mean i could talk english differently from the next person and so on. the movement of the lips may differ from one person to another so you really can't expect a perfect output.

Reply