"I Think, I Conceptualize, I Analyze, I Design, and I Create." ~ Puneet Kalra

Cognitive Robotics Research Centre Of University Of Wales, Newport Puneet Kalra

Home Studies Research Projects Tutorials Portfolio

Puneet Kalra - www.puneetk.com - Socializing Robots

And the research continues ..

August 31st, 2010

Hey everyone,

I hope you guys are doing well.
First of all, Thanks for being so supportive, appreciating my work and posting such nice comments. Also, Sorry for not updating my blog as I’m really busy these days.

And yes, few updates from my research.. Yeah Yeah! I know, I’m busy but still, I can’t stop it, I’m addicted to it now. New things, New problems, New ways to think and Finally the New SOLUTIONS ! That’s how it goes!

Let’s talk about Sphinx, firstly, I got a partner to work on it. “Puneet Jindal” , Another Stubborn guy like me *Lol*, always ready to burn up his mind and a die hard Algo’s Lover. He’s pursuing B.Tech ( finaly year ) from NIT,Kurukshetra. We have got 85-90% accuracy on hundreds ( as the Accuracy Tracker says ) and now we are working with thousands of words to get same accuracy level on them.

Second major topic is HTML5, And I’m really loving it ! Not much to share about it. Just want to say, “HTML5 is just SO AWESOME” !

Now the upcoming topics, Optical Character Recognition (OCR), 3D Painting and Gaming/Artificial Intelligence Algo’s. I haven’t really started working on these topics, You can say that I’m having my one eye on them.

That’s it for now !
~Puneet Kalra

Finite State Machine ( FSM )

May 18th, 2010

Hello everyone,

Its been quite long, I haven’t made any update on my site. Because i was busy researching/working on Artificial Intelligence techniques and specifically on Sphinx 4.

In this post, I’m going explain you a widely used, simple but important technique named “Finite State Machine”, In short “FSM”. Basically, FSM is used as a control technique in context of Artificial Intelligence. FSM is model of composition of Behaviors and executing Actions by finite numbers of states and transitions between those states.

Using this technique we can understand how Actors (Objects) will behave and perform action (Logic) when system realizes specific real time conditions and present state of system. Current state of system can be recognized by past states changing by input changes from start of system to the current state. Transition plays an important role in FSM. Transitions are like barriers between two states. Barriers should be unlocked by fulfilling the condition to move from one state to another.

FSM Example

FSM Example - Source : Wikipedia

Finite State Machine is widely used. It can used for Simple string data comparison and also for designing highly complex operating systems.

How To Improve Accuracy ( Sphinx 4 )

March 18th, 2010

Hello everyone,

I’m back with a very big question.

How to improve accuracy ?!?!?

Its been a week, I’m getting 2-3 mails everyday  regarding “How to improve accuracy”. So this post is answer to all those and upcoming mails. First of all the problem is really big and complex, And to understand the problem first you need to understand the very basics of Speech Recognition and background process of Speech Recognition.

Basics of Speech Recognition are not Java coding part Or Whats gram file Or Why we use Recognizer and Microphone classes .. NO !

Remember ! Changes in Java Coding will not make any major effect in Accuracy. You need to focus on config.xml file.

To solve this problem, All you need to do is, understand the Automatic Speech Recognition, Search and Language Processing Algo’s. Specifically on Sphinx, you need understanding of Hidden Markov Models ( HMMs ) methods of Speech Recognition, Beam Searching, Language Configs and Modeling. Its highly recommended to read any good book before you continue to work on it.

Before we apply the configs on our application to give more accurate results, We must know to how to track results.
Click here to know how to apply Accuracy Tracker on your application.

Next part is, Knowing what kind of SR application you are creating. Basically, SR applications are divided into 2 major categories : Dictation and Command

Command Application When words are very few And Dictation Application When you have a large vocabulary.

And for both type of applications we need a language model or grammar file respectively. A language model is a file containing the probabilities of sequences of words. A grammar is a much smaller file containing sets of predefined combination of words.

Now again on the most important part, Accuracy Tracking. Find as many resources, examples, notes you can. Finally collect databases, Few are also available on Sphinx website. Work on it. Recognize it. Try to get your best results.

Expected Results in Silent Environment :
Command : 5% WER
Medium Vocabulary : 15% WER
Large Vocabulary : 30% WER

If environment is noisy, Multiply the results by 2.

If you are not getting the expected results. Keep trying and try to find the bugs in config file and run time reports.

Still if you are not happy with the results And have got extra time. Open the config.xml file and start playing with Configs and keep experimenting. Also, Make sure you make notes on every configs and their outputs.

If you have any better ideas or any suggestions, Please feel free to post comments.

That’s it for now ! Signing off.

Basics of Java Speech Grammar Format ( JSGF )

March 12th, 2010

Hello Everyone,

This post is my response to those 4 help requests that i received in last few days. This one is going to be very basic and essentials of Java Speech Grammar Format ( JSGF ).

Every single file defines a single grammar only. Each grammar contains two parts:
The grammar header and the grammar body.

Grammar header format : #JSGF version char-encoding locale;

“#JSGF” is required and “version char-encoding locale;” is optional.

Grammar header example : #JSGF V1.0; & #JSGF V1.0 ISO8859-5 en;



After declaring Grammar Header, We need to specify the Grammar name.

Grammar name format : grammar grammarName;

This is a mandatory line. Else javax.speech.recognition.GrammarException will be thrown. Not only on Grammar Name, you will get GrammarExpection If you made any kind of mistake in grammar file.

Grammar Name Example : grammar helloWorld;



Once you are done with above part, you next step will be defining Grammar body.
The grammar body defines rules. you can define a rule only once. If you declare same rule twice then it will overwritten.

Rule Definition Format : public <ruleName> = ruleExpansion;

“public” is optional and remaining part is mandatory.
The rule expansion defines how the rule may be spoken. It is a logical combination of tokens (text that may be spoken) and references to other rules.

Rule Example : public <greet> = Hello;

The rule <greet> refers to a single token Hello. So to say rule <greet>, User must say word “Hello”.

A simple rule expansion can refer to one or more tokens or rules.

public <greet> = Hello;
public <completeGreet> = <greet> World;

Now, rule <completeGreet> refers to rule <greet> and token World. To say rule <completeGreet>, User must say “Hello World”.

Lets complete a simple “Hello World” grammar file.

#JSGF V1.0;

grammar simpleExample;

public <greet> = Hello;
public <completeGreet> = <greet> World;

This grammar file will allow you SR application to recognize 2 sentences. “Hello” and “Hello World”.

Now lets play a little bit with rule expansions.

Alternatives : “|”

public <greet> = Hello | Hey | Hi;

To say rule <greet>, User must say “Hello” or “Hey” or “Hi” But ONLY one of these three words.

Parentheses : “( )”

public <greet> = Hello ( World | User | Friend );
public <command> = ( Open | Close ) ( Door | Window );

To say rule <greet>, User must say “Hello World” or “Hello User” or “Hello Friend”.
And, to say rule <command>, User must say “Open Door” or “Open Window” or “Close Door” or “Close Window”.

Optional Grouping : “[ ]”

public <greet> = [ Hello ] World;

To say rule <greet>, User must say “Hello World” or “World”. As “Hello” is defined inside the Optional Grouping. So user may say it or not but “World” is mandatory.

Kleene Star : “*”

public <greet> = ( Hello | Hey | Hi )* World;

Any group or expansion followed by asterisk symbol indicates that it may be spoken zero or more times. For example “Hey Hello World” or “World” or “Hello Hello Hello World”.

Plus Operator : “+”

public <greet> = ( Hello | Hey | Hi )+ World;

A Plus Operator works same as “Kleene Star”, The only thing that makes difference is any group or expansion followed by plus symbol indicates that it may be spoken one ( NOT ZERO ) or more times.

You can also add comments in grammar file.

// One line comment
/* Multiple lines comment*/
/**
* Documentation comment
* @author Puneet Kalra
*/


Hope this tutorial clears all your doubts on JSGF.

More updates to come soon!
Regards,

Expanding Dictionary Of Acoustic Model

January 5th, 2010

Hello Everyone,

Today I’m going to tell you how to expand dictionary of acoustic model for Sphinx4. In simple words, This tutorial will tell you how you can add more words in Sphinx’s words database (Dictionary) and let it recognize those words, which are not available in default acoustic models provided by CMU Sphinx. This tutorial is based on “HelloWorld” example provided by CMU Sphinx.

Important Files in this example :
1 ) HelloWorld.java
2) hello.gram
3) helloworld.config.xml

Acoustic Model used in this example :
WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.jar

Lets say, We are creating a SR system for ABC National airlines. Everything will go fine and Sphinx will recognize most of the words except the name of cities and states of India.  Now, I will tell you, How to add name of cities and states in dictionary.

PART ONE
Step 1 :
Create a txt file “words.txt”, Write all the names of cities and states in it and save.
Step 2 : Open this link : http://www.speech.cs.cmu.edu/tools/lmtool.html
Step 3 : On that page, go to “Sentence corpus file:” section, Browse to “words.txt” file and click “Compile Knowledge Base”.
Step 4 : On next page, Click on “Dictionary” link and save that .DIC file.

PART TWO
Step 1 : Extract WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.jar file.
Step 2 : Go to edu\cmu\sphinx\model\acoustic\WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz\dict folder.
Step 3 : Open “cmudict.0.6d” file in that folder.
Step 4 : Copy data from .DIC file, you have downloaded in PART ONE, paste it in “cmudict.0.6d” file and save.
Step 5 : Zip the extracted hierarchy back as it was and Zip file named should be same as JAR file.

Now, remove “WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.jar” file from Project’s CLASSPATH and add “WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.zip” instead of it.

That’s it ! We are done.  Now Sphinx will also recognize all name of cities and states that we wrote in “words.txt” file.
Now, FAQ time. I will be posting FAQ and few important notes in comments. :)

If you have any quires, Please feel free to ask.
Regards,