.NET SpeechRecognitionEngine theading

[James Rose]

Hey gang,

I have a media application (Bedia, free available here: http://www.blissgig.com/display.aspx?id=13 ) that I have added speech recognition. The problem is that on some lower end systems the SR causes the audio playback to become "off" from the video playback, usually by only a fraction of a second. I am using WMP as the audio/video playback.

The question is, I am thinking of having a threaded class that does the monitoring for the speech recognition, but before I go down that path I thought I'd ask for any thoughts? Is there anything I should be aware of or......

I know the question is a bit vague, I am attempting to avoid mentioning specifics so that any and all issues for this process are brought up.

Thank for your time.

[Andre S. Veteran]

If I understand well, you're currently handing the audio data to a speech recognition API before handing it to WMP, which can cause the audio to become out of sync? It's pretty strange that this is happening, but I'm not familiar with WMP.

In any case, doing this on separate thread would work as long as you're actually handing out the data to the processes (WMP and SR) in parallel and not having one waiting on the other, otherwise you're not gaining anything by using multiple threads.

[James Rose]

Thanks Dr.

The speech commands affect Play, Pause, other playback controls as well as menu commands. I really don't know why the audio becomes unsynced so the idea of moving the SR to another thread is a shot in the dark. I have completed the code in a test bed and will move it into the app in the next few days. I will drop a note here to let everyone know how it goes.

The nice part of the SR being on another thread is that some of the processing (is a command valid, etc) is within that thread and SHOULD not affect any playback. I just return a string that informs the base app what I need to do, just as if the user had pressed a mouse button.

Have a great day.

James

[firey]

  On 15/04/2013 at 13:56, James Rose said:

Thanks Dr.

The speech commands affect Play, Pause, other playback controls as well as menu commands. I really don't know why the audio becomes unsynced so the idea of moving the SR to another thread is a shot in the dark. I have completed the code in a test bed and will move it into the app in the next few days. I will drop a note here to let everyone know how it goes.

The nice part of the SR being on another thread is that some of the processing (is a command valid, etc) is within that thread and SHOULD not affect any playback. I just return a string that informs the base app what I need to do, just as if the user had pressed a mouse button.

Have a great day.

James

Did you do it delegate style? Or are you constantly checking a string for a value?

[James Rose]

  On 15/04/2013 at 14:02, firey said:

Did you do it delegate style? Or are you constantly checking a string for a value?

Yes I am using a Delegate, thanks for asking. Only when acceptable conditions have been met is a string sent back to the calling thread. This is why I thought about going this route, all the work to insure that this is an acceptable command before affecting the app. I really do appreciate you asking to make sure.

Peace,

James

[firey]

  On 15/04/2013 at 15:23, James Rose said:

Yes I am using a Delegate, thanks for asking. Only when acceptable conditions have been met is a string sent back to the calling thread. This is why I thought about going this route, all the work to insure that this is an acceptable command before affecting the app. I really do appreciate you asking to make sure.

Peace,

James

I know some people will thread things but then sit there waiting for the thread to finish/do something instead of carrying on and doing what needs to be done away from the thread and acknowledging when the thread has something for it. That's the reason I asked.

But yea, logically if the thread is just sitting in a loop like:

wait for command

process command

if command is good raise the event

repeat

and the main program is sitting in a

play movie/sit in idle state

**Event raised**

check event

do event

repeat

And there is no reliance on one or the other to preform individual tasks and you aren't sitting there in a:

while (x != y) or while(myThread.ThreadState == ThreadState.Running)

Then I don't see why it wouldn't work, and it should work better than a non-threaded system.

[James Rose]

Thanks again Firey,

Your diagram statement is exactly what is going to happen (I haven't added my new class to my app yet. I almost always create a test bed application to insure that it is working on it's own before adding it to the real application)

I still find it strange that the audio becomes unsynced.... but when I shut down the SR option, which disposes of the SR components, the issue goes away. I state this in case anyone else ever has the issue that they know that someone else has seen it.

Have a fun day.

[firey]

  On 15/04/2013 at 16:02, James Rose said:

Thanks again Firey,

Your diagram statement is exactly what is going to happen (I haven't added my new class to my app yet. I almost always create a test bed application to insure that it is working on it's own before adding it to the real application)

I still find it strange that the audio becomes unsynced.... but when I shut down the SR option, which disposes of the SR components, the issue goes away. I state this in case anyone else ever has the issue that they know that someone else has seen it.

Have a fun day.

One thing to check, would be CPU usage. I know that in some cases High CPU usage can cause things to become un-synced. If the playback has full video decoding frames can be ready and display but the CPU can't keep up with keeping things (like the Audio) synced properly.

[James Rose]

  On 15/04/2013 at 16:32, firey said:

One thing to check, would be CPU usage. I know that in some cases High CPU usage can cause things to become un-synced. If the playback has full video decoding frames can be ready and display but the CPU can't keep up with keeping things (like the Audio) synced properly.

Interesting thought. The app has run for two+ years on this lightwight machine but with the addition of SR it could be hitting high. I will check this out, thanks.

[James Rose]

FYI: Placing my speech class into another thread has worked out great even on my low end system. If anyone needs details feel free to ask me.

James