10 posts in this topic

Hey gang,

I have a media application (Bedia, free available here: http://www.blissgig.com/display.aspx?id=13 ) that I have added speech recognition. The problem is that on some lower end systems the SR causes the audio playback to become "off" from the video playback, usually by only a fraction of a second. I am using WMP as the audio/video playback.

The question is, I am thinking of having a threaded class that does the monitoring for the speech recognition, but before I go down that path I thought I'd ask for any thoughts? Is there anything I should be aware of or......

I know the question is a bit vague, I am attempting to avoid mentioning specifics so that any and all issues for this process are brought up.

Thank for your time.

Share this post


Link to post
Share on other sites

If I understand well, you're currently handing the audio data to a speech recognition API before handing it to WMP, which can cause the audio to become out of sync? It's pretty strange that this is happening, but I'm not familiar with WMP.

In any case, doing this on separate thread would work as long as you're actually handing out the data to the processes (WMP and SR) in parallel and not having one waiting on the other, otherwise you're not gaining anything by using multiple threads.

Share this post


Link to post
Share on other sites

Thanks Dr.

The speech commands affect Play, Pause, other playback controls as well as menu commands. I really don't know why the audio becomes unsynced so the idea of moving the SR to another thread is a shot in the dark. I have completed the code in a test bed and will move it into the app in the next few days. I will drop a note here to let everyone know how it goes.

The nice part of the SR being on another thread is that some of the processing (is a command valid, etc) is within that thread and SHOULD not affect any playback. I just return a string that informs the base app what I need to do, just as if the user had pressed a mouse button.

Have a great day.

James

Share this post


Link to post
Share on other sites

Thanks Dr.

The speech commands affect Play, Pause, other playback controls as well as menu commands. I really don't know why the audio becomes unsynced so the idea of moving the SR to another thread is a shot in the dark. I have completed the code in a test bed and will move it into the app in the next few days. I will drop a note here to let everyone know how it goes.

The nice part of the SR being on another thread is that some of the processing (is a command valid, etc) is within that thread and SHOULD not affect any playback. I just return a string that informs the base app what I need to do, just as if the user had pressed a mouse button.

Have a great day.

James

Did you do it delegate style? Or are you constantly checking a string for a value?

Share this post


Link to post
Share on other sites

Did you do it delegate style? Or are you constantly checking a string for a value?

Yes I am using a Delegate, thanks for asking. Only when acceptable conditions have been met is a string sent back to the calling thread. This is why I thought about going this route, all the work to insure that this is an acceptable command before affecting the app. I really do appreciate you asking to make sure.

Peace,

James

Share this post


Link to post
Share on other sites

Yes I am using a Delegate, thanks for asking. Only when acceptable conditions have been met is a string sent back to the calling thread. This is why I thought about going this route, all the work to insure that this is an acceptable command before affecting the app. I really do appreciate you asking to make sure.

Peace,

James

I know some people will thread things but then sit there waiting for the thread to finish/do something instead of carrying on and doing what needs to be done away from the thread and acknowledging when the thread has something for it. That's the reason I asked.

But yea, logically if the thread is just sitting in a loop like:

wait for command

process command

if command is good raise the event

repeat

and the main program is sitting in a

play movie/sit in idle state

**Event raised**

check event

do event

repeat

And there is no reliance on one or the other to preform individual tasks and you aren't sitting there in a:

while (x != y) or while(myThread.ThreadState == ThreadState.Running)

Then I don't see why it wouldn't work, and it should work better than a non-threaded system.

Share this post


Link to post
Share on other sites

Thanks again Firey,

Your diagram statement is exactly what is going to happen (I haven't added my new class to my app yet. I almost always create a test bed application to insure that it is working on it's own before adding it to the real application)

I still find it strange that the audio becomes unsynced.... but when I shut down the SR option, which disposes of the SR components, the issue goes away. I state this in case anyone else ever has the issue that they know that someone else has seen it.

Have a fun day.

Share this post


Link to post
Share on other sites

Thanks again Firey,

Your diagram statement is exactly what is going to happen (I haven't added my new class to my app yet. I almost always create a test bed application to insure that it is working on it's own before adding it to the real application)

I still find it strange that the audio becomes unsynced.... but when I shut down the SR option, which disposes of the SR components, the issue goes away. I state this in case anyone else ever has the issue that they know that someone else has seen it.

Have a fun day.

One thing to check, would be CPU usage. I know that in some cases High CPU usage can cause things to become un-synced. If the playback has full video decoding frames can be ready and display but the CPU can't keep up with keeping things (like the Audio) synced properly.

Share this post


Link to post
Share on other sites

One thing to check, would be CPU usage. I know that in some cases High CPU usage can cause things to become un-synced. If the playback has full video decoding frames can be ready and display but the CPU can't keep up with keeping things (like the Audio) synced properly.

Interesting thought. The app has run for two+ years on this lightwight machine but with the addition of SR it could be hitting high. I will check this out, thanks.

Share this post


Link to post
Share on other sites

FYI: Placing my speech class into another thread has worked out great even on my low end system. If anyone needs details feel free to ask me.

James

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Recently Browsing   0 members

    No registered users viewing this page.