There are more than one "dialect" of sign language so maybe Google Cloud Compute AI app? They give 12 months trial with $300 to spent so integration with OpenCV might be a way, and you are not constraint with RPi compute power for quick recognition.

Yeah . Due to the low processing speed we are also looking forward to integrate the recognition with cloud services. Anyway Thanks for your concern ☺

Great idea Jonathan, I was thinking of something similar as my daughter has no speech and I've tried several times over the last decade to help her. Computers are now pretty fast and I've even developed my own vision system that's a lot faster than OpenCV and can accurately track movement and detail.

The trouble I've had seems insurmountable, I'm using a 6GHz dual Athlon to develop on and that isnt really fast enough. Are you using an alphabet? From experience using OpenCV it will take around 2-3 seconds per letter to discriminate and your friend will have to be VERY accurate. It was that decided it for me, and I fell back to using pictorial information with an AI to present it meaningfully according to context.

You might want to look into this technique, it will cut down on the signs needed for the system to learn by combining them in sequences like a phone does with typed information. Mine takes an icon representing a word and then presents only icons representing things that can follow it, and it uses a variation of written Makaton to do it, backed up by photos. But using predictive text between the input and output will seriously cut down on the data your friend has to impart to get a spoken phrase.

The average English-speaking human gets by with around 5k words and can interact in society and perform non-technical work using them. A secretarial job requires over twice that many, and a lot of them are spelled differently but sound the same. (There Their and Theyre for example are a nuisance to a machine. On the Internet we ignore the spelling and concentrate on context alone, so to transmit a sentence containing one of those words doesnt have to have it spelt correctly and an AI can discriminate and spell it correctly at the other end. In transmission all 3 words are actually the same to the computer, its the words either side that spell it correctly.)

If you store the phrases contextually it will be a lot faster for your friend to access them with less signs, and may even be fast enough for him to have a conversation of sorts. Communicating is possible, conversing may be harder than you think this way.

Saying that, liked and followed, and I wish you and your friend the best of luck. Tentacles crossed, humans are hard to communicate with ;-)

Immense thanks for your concern and helpful information ☺

No problem, hit me up on PM if you want to chat about it some. My work has gone unused because my Bea has severe learning difficulties as well, she cant read or write as well as speak but she does understand a fair bit. And I'm not allowed to help others because the UK government are so shortsighted they'd have me care for her unpaid where I cant work for myself, let alone help all disabled people.

I'm fighting back, but the system is broke. They made me the UK's official hacker in the Prize this year and told me to build a robot with 300 quid, and then they'll take any award off me because I'm in the social care system.

I know its no better in the States, and there are less concerned governments than that...

Happy to help anyone fix that sh*t tho. :-)

Of course sir . I'm so happy to contact you and I'm seriously considering to make some suitable changes as per your comment and trying more to implement it as a neural network. Anyway I'm ready to take this opportunity to help my needy siblings and hope that your help will be available

Thanks☺

Impressive!

Thank You 😊

nice work bro .Keep it up

Thank You 😊

Cool project! Was wondering how you planned to deal with words where hands overlapped or covered one another. Words like cat or phrases where object interact.

We are using a key image for different signs and the signs are differentiated using key images and since homologous images can be matched , I hope it will be possible too .

Neat idea! Matching the pre-stored images sounds tricky. Lately I've been trying to think like a neural network engineer to look for applications of the technology. Running this through some machine learning to develop a new data set might be a good way to speed up this recognition task on minimal hardware.

Thank you Mike :)