VoIP Mailing List Archives

[mitachundkrach at gmai...]

Hello everyone,

i have a technical question about freeswitch and would like to know if
my idea is technically possible.

I've got a project at the university and my goal is to improve automatic
speech recognition for bigbluebutton.

In the summer semester a group of students of my university made a
subtitling plugin[1] for bigbluebutton. This plugin adds a participant
into the meeting and grabs his audiostream to send it to the kaldi ASR
Software. When more than one person speaks at a time the detection
decreases.

My idea is to copy the incoming freeswitch audiostreams from every
participant and send them directly as a copy to the ASR (kaldi can
handle multiple audiostreams at once).

In the Confluence-Wiki i searched for something in this direction but
only found things like Dialplan to ring multiple telephones at once.

I hope my idea is formulated understandably.


Kind regards,

Robert

[1] https://github.com/3wille/bbb-kaldi-connector


_________________________________________________________________________

The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services.
Build your next product on our scalable cloud platform.

Join our online community to chat in real time https://signalwire.community

Professional FreeSWITCH Services
sales@freeswitch.com
https://freeswitch.com

Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com

[piotr at dataandsignal...]

Hi Robert,

FreeSWITCH can be extended by writing custom modules. Modules can register a callback to get every audio frame passed by the core FS to them.
You can find a lot of examples in src/mod (best look into src/mod/applications, e.g. mod_avmd works with audio frames).


kind regards,
[img]https://drive.google.com/a/dataandsignal.com/uc?id=1lUdzdLjEycxXcGhDz-bKqDX8-AIlZuCq&export=download[/img]



Piotr Gregor
Software Engineer


M: (+44) 07483 866 525     L: (+44) 01256 597 470     www: dataandsignal.com






















On Mon, Dec 14, 2020 at 4:56 PM R G <mitachundkrach@gmail.com (mitachundkrach@gmail.com)> wrote:

[davidswalkabout at gma...]

How do you currently forward audio to kaldi?

If you use an FS conference, I think there is no way to separate the audio per-user.


I agree it would be very useful if there were a way to process each user's audio separately. After a few years of using FS, I haven't come across a way to do it.

[dujinfang at gmail.com]

I believe directly run the `detect_speech` app on each channel works.

On Wed, Dec 16, 2020 at 6:21 AM David P <davidswalkabout@gmail.com (davidswalkabout@gmail.com)> wrote:

[mitachundkrach at gmai...]

Hi all,
currently we add an participant into the conference and take his audio for speech recognition.
I think bbb uses the build FS conferences.
So the mod_detect_speech app works on single channels in a conference?

I hoped i can catch dailers before joining a conference and copy their streams as an alternative.
Kind regards,
Robert
Am 16.12.2020 um 14:21 schrieb Seven Du: