Sponsor: VoiceMeUp - Corporate & Wholesale VoIP Services

VoIP Mailing List Archives
Mailing list archives for the VoIP community
 SearchSearch 

[Freeswitch-users] Stopping TTS when play_and_detect_speech gets MRCP START-OF-INPUT


 
Post new topic   Reply to topic    VoIP Mailing List Archives Forum Index -> freeSWITCH Users
View previous topic :: View next topic  
Author Message
mayamatakeshi at gmail...
Guest
PostPosted: Wed Mar 17, 2021 5:16 am    Post subject: [Freeswitch-users] Stopping TTS when play_and_detect_speech Reply with quote
Hi, I'm trying play_and_detect_speech with an ESL app but START-OF-INPUT doesn't interrupt the TTS being played.
Is this expected?
(https://freeswitch.org/confluence/display/FREESWITCH/mod_dptools%3A+play_and_detect_speech is not clear about it)
Or maybe I need to set some channel variable for this to work.
Back to top
mayamatakeshi at gmail...
Guest
PostPosted: Wed Mar 17, 2021 10:49 pm    Post subject: [Freeswitch-users] Stopping TTS when play_and_detect_speech Reply with quote
On Wed, Mar 17, 2021 at 6:52 PM mayamatakeshi <mayamatakeshi@gmail.com (mayamatakeshi@gmail.com)> wrote:

Quote:
Hi, I'm trying play_and_detect_speech with an ESL app but START-OF-INPUT doesn't interrupt the TTS being played.
Is this expected?
(https://freeswitch.org/confluence/display/FREESWITCH/mod_dptools%3A+play_and_detect_speech is not clear about it)
Or maybe I need to set some channel variable for this to work.



Hi,
My app is a fork of https://github.com/plivo/plivoframework 


I debugged FS code and found the cause of the problem:
I verified that the DETECTED_SPEECH events (including the one with Speech-Type begin-speaking) were not being fired.
This was because my app was not setting this variable:
  https://freeswitch.org/confluence/display/FREESWITCH/fire_asr_events
Then I set fire_asr_events=true
but still the prompt didn't get interrupted by speech.
Then I checked how FS code was fetching (dequeuing) events and I realized it checks for divert_events.
Then I changed my app to send
  divert_events off
in the ESL socket
and after that it worked and FS stopped the prompt upon speech start.
However, it is not stopping the prompt when a DTMF digit is received (it was already this way before I did the changes).
So things are better but still there is something amiss.
Back to top
mayamatakeshi at gmail...
Guest
PostPosted: Thu Mar 18, 2021 2:15 am    Post subject: [Freeswitch-users] Stopping TTS when play_and_detect_speech Reply with quote
On Thu, Mar 18, 2021 at 12:23 PM mayamatakeshi <mayamatakeshi@gmail.com (mayamatakeshi@gmail.com)> wrote:

Quote:



On Wed, Mar 17, 2021 at 6:52 PM mayamatakeshi <mayamatakeshi@gmail.com (mayamatakeshi@gmail.com)> wrote:

Quote:
Hi, I'm trying play_and_detect_speech with an ESL app but START-OF-INPUT doesn't interrupt the TTS being played.
Is this expected?
(https://freeswitch.org/confluence/display/FREESWITCH/mod_dptools%3A+play_and_detect_speech is not clear about it)
Or maybe I need to set some channel variable for this to work.



Hi,
My app is a fork of https://github.com/plivo/plivoframework 


I debugged FS code and found the cause of the problem:
I verified that the DETECTED_SPEECH events (including the one with Speech-Type begin-speaking) were not being fired.
This was because my app was not setting this variable:
  https://freeswitch.org/confluence/display/FREESWITCH/fire_asr_events
Then I set fire_asr_events=true
but still the prompt didn't get interrupted by speech.
Then I checked how FS code was fetching (dequeuing) events and I realized it checks for divert_events.
Then I changed my app to send
  divert_events off
in the ESL socket
and after that it worked and FS stopped the prompt upon speech start.
However, it is not stopping the prompt when a DTMF digit is received (it was already this way before I did the changes).
So things are better but still there is something amiss.




OK. I found the reason:
I was using 'playback_terminators=any'
which reading the FS code shows will result in:
  terminators = "1234567890*#"
But I was sending DTMF 'a' so it was not causing the prompt to be terminated.
Then I changed my test script to send '1' instead and then this caused the prompt to terminate.
However, this also terminated the speech detection operation as FS sent STOP to the MRCP server.
And the reason is because differently from begin-speaking, a terminator will set the operation as done (switch_ivr_async.c):


                      } else if (!strcasecmp(speech_type, "begin-speaking")) {            

                          switch_log_printf(SWITCH_CHANNEL_SESSION_LOG(session), SWITCH_LOG_INFO, "(%s) START OF SPEECH\n", switch_channel_get_name(channel));
                          return SWITCH_STATUS_BREAK;
                      }

                      if (terminators && strchr(terminators, dtmf->digit)) {
                          switch_log_printf(SWITCH_CHANNEL_SESSION_LOG(session), SWITCH_LOG_DEBUG, "(%s) ACCEPT TERMINATOR %c\n", switch_channel_get_name(channel), dtmf->digit);
                          switch_channel_set_variable_printf(channel, SWITCH_PLAYBACK_TERMINATOR_USED, "%c", dtmf->digit);
                          state->result = switch_core_session_sprintf(session, "DIGIT: %c", dtmf->digit);
                          state->done = PLAY_AND_DETECT_DONE;
                          return SWITCH_STATUS_BREAK;
                      }


This might be OK for someone but in my case  there is another issue that the termination of the speech detection is not notified to my app via ESL.


I am hoping to solve this by leaving the DTMF collection to be done by the MRCP server but currently the MRCP server I'm using (UniMRCP) doesn't report START-OF-INPUT for DTMF (I only get the RECOGNITION-COMPLETE after all digits are input). (my use case is to allow the user to speak some number sequence or dial it).


So I will switch to use detect_speech instead of play_and_detect_speech.
Back to top
mayamatakeshi at gmail...
Guest
PostPosted: Thu Mar 18, 2021 3:15 am    Post subject: [Freeswitch-users] Stopping TTS when play_and_detect_speech Reply with quote
On Thu, Mar 18, 2021 at 3:59 PM mayamatakeshi <mayamatakeshi@gmail.com (mayamatakeshi@gmail.com)> wrote:

Quote:



On Thu, Mar 18, 2021 at 12:23 PM mayamatakeshi <mayamatakeshi@gmail.com (mayamatakeshi@gmail.com)> wrote:

Quote:



On Wed, Mar 17, 2021 at 6:52 PM mayamatakeshi <mayamatakeshi@gmail.com (mayamatakeshi@gmail.com)> wrote:

Quote:
Hi, I'm trying play_and_detect_speech with an ESL app but START-OF-INPUT doesn't interrupt the TTS being played.
Is this expected?
(https://freeswitch.org/confluence/display/FREESWITCH/mod_dptools%3A+play_and_detect_speech is not clear about it)
Or maybe I need to set some channel variable for this to work.



Hi,
My app is a fork of https://github.com/plivo/plivoframework 


I debugged FS code and found the cause of the problem:
I verified that the DETECTED_SPEECH events (including the one with Speech-Type begin-speaking) were not being fired.
This was because my app was not setting this variable:
  https://freeswitch.org/confluence/display/FREESWITCH/fire_asr_events
Then I set fire_asr_events=true
but still the prompt didn't get interrupted by speech.
Then I checked how FS code was fetching (dequeuing) events and I realized it checks for divert_events.
Then I changed my app to send
  divert_events off
in the ESL socket
and after that it worked and FS stopped the prompt upon speech start.
However, it is not stopping the prompt when a DTMF digit is received (it was already this way before I did the changes).
So things are better but still there is something amiss.




OK. I found the reason:
I was using 'playback_terminators=any'
which reading the FS code shows will result in:
  terminators = "1234567890*#"
But I was sending DTMF 'a' so it was not causing the prompt to be terminated.
Then I changed my test script to send '1' instead and then this caused the prompt to terminate.
However, this also terminated the speech detection operation as FS sent STOP to the MRCP server.
And the reason is because differently from begin-speaking, a terminator will set the operation as done (switch_ivr_async.c):


                      } else if (!strcasecmp(speech_type, "begin-speaking")) {            

                          switch_log_printf(SWITCH_CHANNEL_SESSION_LOG(session), SWITCH_LOG_INFO, "(%s) START OF SPEECH\n", switch_channel_get_name(channel));
                          return SWITCH_STATUS_BREAK;
                      }

                      if (terminators && strchr(terminators, dtmf->digit)) {
                          switch_log_printf(SWITCH_CHANNEL_SESSION_LOG(session), SWITCH_LOG_DEBUG, "(%s) ACCEPT TERMINATOR %c\n", switch_channel_get_name(channel), dtmf->digit);
                          switch_channel_set_variable_printf(channel, SWITCH_PLAYBACK_TERMINATOR_USED, "%c", dtmf->digit);
                          state->result = switch_core_session_sprintf(session, "DIGIT: %c", dtmf->digit);
                          state->done = PLAY_AND_DETECT_DONE;
                          return SWITCH_STATUS_BREAK;
                      }


This might be OK for someone but in my case  there is another issue that the termination of the speech detection is not notified to my app via ESL.


I am hoping to solve this by leaving the DTMF collection to be done by the MRCP server but currently the MRCP server I'm using (UniMRCP) doesn't report START-OF-INPUT for DTMF (I only get the RECOGNITION-COMPLETE after all digits are input). (my use case is to allow the user to speak some number sequence or dial it).




Sorry, 
checking again, there is some specific condition that causes UniMRCP to not send the START-OF-INPUT.
I'm not sure what it is yet but it may be by design or following something in the protocol that I am not aware of.
 
Quote:


So I will switch to use detect_speech instead of play_and_detect_speech.






Back to top
lewis.bergman at gmail...
Guest
PostPosted: Thu Mar 18, 2021 9:07 am    Post subject: [Freeswitch-users] Stopping TTS when play_and_detect_speech Reply with quote
Arsen is very good about responding on the unimrcp list.
Back to top
mayamatakeshi at gmail...
Guest
PostPosted: Sun Mar 21, 2021 1:37 am    Post subject: [Freeswitch-users] Stopping TTS when play_and_detect_speech Reply with quote
On Thu, Mar 18, 2021 at 4:46 PM mayamatakeshi <mayamatakeshi@gmail.com (mayamatakeshi@gmail.com)> wrote:

Quote:



On Thu, Mar 18, 2021 at 3:59 PM mayamatakeshi <mayamatakeshi@gmail.com (mayamatakeshi@gmail.com)> wrote:

Quote:



On Thu, Mar 18, 2021 at 12:23 PM mayamatakeshi <mayamatakeshi@gmail.com (mayamatakeshi@gmail.com)> wrote:

Quote:



On Wed, Mar 17, 2021 at 6:52 PM mayamatakeshi <mayamatakeshi@gmail.com (mayamatakeshi@gmail.com)> wrote:

Quote:
Hi, I'm trying play_and_detect_speech with an ESL app but START-OF-INPUT doesn't interrupt the TTS being played.
Is this expected?
(https://freeswitch.org/confluence/display/FREESWITCH/mod_dptools%3A+play_and_detect_speech is not clear about it)
Or maybe I need to set some channel variable for this to work.



Hi,
My app is a fork of https://github.com/plivo/plivoframework 


I debugged FS code and found the cause of the problem:
I verified that the DETECTED_SPEECH events (including the one with Speech-Type begin-speaking) were not being fired.
This was because my app was not setting this variable:
  https://freeswitch.org/confluence/display/FREESWITCH/fire_asr_events
Then I set fire_asr_events=true
but still the prompt didn't get interrupted by speech.
Then I checked how FS code was fetching (dequeuing) events and I realized it checks for divert_events.
Then I changed my app to send
  divert_events off
in the ESL socket
and after that it worked and FS stopped the prompt upon speech start.
However, it is not stopping the prompt when a DTMF digit is received (it was already this way before I did the changes).
So things are better but still there is something amiss.




OK. I found the reason:
I was using 'playback_terminators=any'
which reading the FS code shows will result in:
  terminators = "1234567890*#"
But I was sending DTMF 'a' so it was not causing the prompt to be terminated.
Then I changed my test script to send '1' instead and then this caused the prompt to terminate.
However, this also terminated the speech detection operation as FS sent STOP to the MRCP server.
And the reason is because differently from begin-speaking, a terminator will set the operation as done (switch_ivr_async.c):


                      } else if (!strcasecmp(speech_type, "begin-speaking")) {            

                          switch_log_printf(SWITCH_CHANNEL_SESSION_LOG(session), SWITCH_LOG_INFO, "(%s) START OF SPEECH\n", switch_channel_get_name(channel));
                          return SWITCH_STATUS_BREAK;
                      }

                      if (terminators && strchr(terminators, dtmf->digit)) {
                          switch_log_printf(SWITCH_CHANNEL_SESSION_LOG(session), SWITCH_LOG_DEBUG, "(%s) ACCEPT TERMINATOR %c\n", switch_channel_get_name(channel), dtmf->digit);
                          switch_channel_set_variable_printf(channel, SWITCH_PLAYBACK_TERMINATOR_USED, "%c", dtmf->digit);
                          state->result = switch_core_session_sprintf(session, "DIGIT: %c", dtmf->digit);
                          state->done = PLAY_AND_DETECT_DONE;
                          return SWITCH_STATUS_BREAK;
                      }


This might be OK for someone but in my case  there is another issue that the termination of the speech detection is not notified to my app via ESL.


I am hoping to solve this by leaving the DTMF collection to be done by the MRCP server but currently the MRCP server I'm using (UniMRCP) doesn't report START-OF-INPUT for DTMF (I only get the RECOGNITION-COMPLETE after all digits are input). (my use case is to allow the user to speak some number sequence or dial it).




Sorry, 
checking again, there is some specific condition that causes UniMRCP to not send the START-OF-INPUT.
I'm not sure what it is yet but it may be by design or following something in the protocol that I am not aware of.




Just closing this issue and leaving some final details in case someone gets into a similar problem:
  - I was testing with unimrcp server 1.6.0 and under some conditions this version would not send START-OF-INPUT upon DTMF detection
  - after updating to unimrcp server 1.7.0, the problem went away.


And in case someone needs to follow SIP/MRCP/DTMF flows, you can try my sngrep fork. Details here:
  https://github.com/irontec/sngrep/issues/78
 
(it also supports UTF-8 which is particularly relevant to see the text in MRCP SPEAK, DEFINE-GRAMMAR and RECOGNITION-COMPLETE messages)



Here is a snapshot:




Quote:
 
Quote:


So I will switch to use detect_speech instead of play_and_detect_speech.








So, using play_and_detect_speech is viable for my needs as I can leave DTMF handling to unimrcp server.
I might eventually need to implement a solution using detect_speech like in case of max-number-of-digits which unimrcp server doesn't support at the moment. 
But for now, play_and_detect_speech should suffice.


 
Back to top
Display posts from previous:   
Post new topic   Reply to topic    VoIP Mailing List Archives Forum Index -> freeSWITCH Users All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group

VoiceMeUp - Corporate & Wholesale VoIP Services