{"id":19096982,"url":"https://github.com/lionsoul2014/freeswitch-asr","last_synced_at":"2025-04-30T14:29:17.149Z","repository":{"id":38341455,"uuid":"198013247","full_name":"lionsoul2014/freeswitch-asr","owner":"lionsoul2014","description":"TTS and ASR module with auto Voice Active Detecting supported for Freeswitch.  I build it for Nature sound interactive, With the embedded LUA engine we could easly build a Freeswtich application like this.","archived":false,"fork":false,"pushed_at":"2019-07-21T09:46:27.000Z","size":23,"stargazers_count":69,"open_issues_count":1,"forks_count":38,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-30T14:29:13.499Z","etag":null,"topics":["call-center","freeswitch","freeswitch-asr","freeswitch-tts","ivr-application","lua-script"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lionsoul2014.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-07-21T04:49:19.000Z","updated_at":"2025-04-22T03:40:46.000Z","dependencies_parsed_at":"2022-09-21T03:43:08.998Z","dependency_job_id":null,"html_url":"https://github.com/lionsoul2014/freeswitch-asr","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lionsoul2014%2Ffreeswitch-asr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lionsoul2014%2Ffreeswitch-asr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lionsoul2014%2Ffreeswitch-asr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lionsoul2014%2Ffreeswitch-asr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lionsoul2014","download_url":"https://codeload.github.com/lionsoul2014/freeswitch-asr/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251721254,"owners_count":21632798,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["call-center","freeswitch","freeswitch-asr","freeswitch-tts","ivr-application","lua-script"],"created_at":"2024-11-09T03:38:23.085Z","updated_at":"2025-04-30T14:29:17.105Z","avatar_url":"https://github.com/lionsoul2014.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Description\nTTS and ASR module with auto Voice Active Detecting supported for Freeswitch. \nI build it for Nature sound interactive, With the embedded LUA engine we could easly build a Freeswtich application like this.\n\n\n# How to use it ?\n1. Copy and merge the ${PROJECT_ROOT}/src and the ${PROJECT_ROOT}/conf with the Freeswitch source tree.\n2. Add the following two module in the ${FREESWITCH_SOURCE_ROOT}/modules.conf\n```\nasr_tts/mod_yytts\nasr_tts/mod_yyasr\n```\n\n3. Re-compile and install the freeswitch to install mod_yytts and mod_yyasr modules.\n4. Active the mod_yytts and the mod_yyasr by add the following to lines into the ${FREESWITCH_INSTALLATION_ROOT}/conf/autoload_configs/modules.conf.xml\n```\n\u003cload module=\"mod_yytts\"/\u003e\n\u003cload module=\"mod_yyasr\"/\u003e\n```\n\n5. Copy the lua scripts under ${PROJECT_ROOT}/scripts to ${FREESWITCH_INSTALLATION_ROOT}/scripts/\n6. Bind a number to build application by adding the following xml settings to the ${FREESWITCH_INSTALLATION_ROOT}/conf/dialplan/default.xml\n```\n\u003c!-- yytts tts module testing --\u003e\n\u003cextension name=\"yytts_demo\"\u003e\n  \u003ccondition field=\"destination_number\" expression=\"^5001$\"\u003e\n    \u003caction application=\"answer\"/\u003e\n    \u003caction application=\"sleep\" data=\"2000\"/\u003e\n    \u003caction application=\"lua\" data=\"yytts_demo.lua\"/\u003e\n  \u003c/condition\u003e\n\u003c/extension\u003e\n\n\u003c!-- yyasr asr module testing --\u003e\n\u003cextension name=\"yyasr_demo\"\u003e\n  \u003ccondition field=\"destination_number\" expression=\"^5002$\"\u003e\n    \u003caction application=\"answer\"/\u003e\n    \u003caction application=\"sleep\" data=\"2000\"/\u003e\n    \u003caction application=\"lua\" data=\"yyasr_demo.lua\"/\u003e\n  \u003c/condition\u003e\n\u003c/extension\u003e\n``` \n\n# How to test it ?\nStart the Freeswitch and install a Linphone client.\n1. Dial 5001 for TTS testing, check ./scripts/yytts_demo.lua for detail usage, module interface overview:\n```lua\nsession:answer();   -- answer the call\n\n-- set to use the yytts TTS with zhilingf speaker\nsession:set_tts_params(\"yytts\", \"zhilingf\");\n\n-- fire the speak\nsession:speak(\"Hello，我是人工智能电话系统, Bye!\");\n\n-- hangup the call\nsession:hangup();\n```\n\n2. Dial 5002 for ASR testing, check ./scripts/yyasr_demo for detail usage, ASR interface overview:\n```lua\nsession:answer();   -- answer the call\n\n-- register the sound input callback\nsession:setInputCallback(function onInput(s, type, obj) end);\n\n-- start to detect the speech to keep firing the feed interface insite yyasr module\nsession:execute(\"detect_speech\", \"yyasr directory directory\");\n\n-- resume the detect_speech\nsession:execute(\"detect_speech\", \"resume\");\n\n-- stop the detecting\nsession:execute(\"detect_speech\", \"stop\");\n\n-- check ./script/yyasr_demo for details usage.\n```\n\n\n# Secondary development\n\n### TTS module\n1. TTS module structure:\n```c\n*module_interface = switch_loadable_module_create_module_interface(pool, modname);\ntts_interface  = switch_loadable_module_create_interface(*module_interface, SWITCH_SPEECH_INTERFACE);\ntts_interface-\u003einterface_name = \"yytts\";\ntts_interface-\u003espeech_open = yytts_speech_open;\ntts_interface-\u003espeech_close = yytts_speech_close;\ntts_interface-\u003espeech_feed_tts = yytts_speech_feed_tts;\ntts_interface-\u003espeech_read_tts = yytts_speech_read_tts;\ntts_interface-\u003espeech_flush_tts = yytts_speech_flush_tts;\ntts_interface-\u003espeech_text_param_tts = yytts_text_param_tts;\ntts_interface-\u003espeech_numeric_param_tts = yytts_numeric_param_tts;\ntts_interface-\u003espeech_float_param_tts = yytts_float_param_tts;\n```\n\n2. Invoke process for TTS module in Freeswitch:\n```\nopen\nwhile ( thread running ) do\n  feed\n  read\n  flush\nend\nclose\n```\n\n\n3. Embedded the other TTS engine by override the yytts_speech_feed_tts\n```c\nstatic switch_status_t yytts_speech_feed_tts(switch_speech_handle_t *sh, char *text, switch_speech_flag_t *flags)\n{\n\t// feed the wave stream to the yytts-\u003eaudio_buffer buffer\n\treturn SWITCH_STATUS_SUCCESS;\n}\n```\n\n### ASR module\n1. ASR module structure:\n```\nasr_interface = switch_loadable_module_create_interface(*module_interface, SWITCH_ASR_INTERFACE);\nasr_interface-\u003einterface_name = \"yyasr\";\nasr_interface-\u003easr_open = yyasr_asr_open;\nasr_interface-\u003easr_load_grammar = yyasr_asr_load_grammar;\nasr_interface-\u003easr_unload_grammar = yyasr_asr_unload_grammar;\nasr_interface-\u003easr_close = yyasr_asr_close;\nasr_interface-\u003easr_feed = yyasr_asr_feed;\nasr_interface-\u003easr_resume = yyasr_asr_resume;\nasr_interface-\u003easr_pause = yyasr_asr_pause;\nasr_interface-\u003easr_check_results = yyasr_asr_check_results;\nasr_interface-\u003easr_get_results = yyasr_asr_get_results;\nasr_interface-\u003easr_start_input_timers = yyasr_asr_start_input_timers;\nasr_interface-\u003easr_text_param = yyasr_asr_text_param;\nasr_interface-\u003easr_numeric_param = yyasr_asr_numeric_param;\nasr_interface-\u003easr_float_param = yyasr_asr_float_param;\n```\n\n2. Invoke process for ASR module in Freeswitch:\n```\nopen\nwhile ( thread running ) do\n  feed\n  check_results\n  get_results\nend\noptional pause|resume\nclose\n```\n\n3. Embedded the other ASR engine by override the yyasr_asr_feed\n```c\n/*! function to feed audio to the ASR */\nstatic switch_status_t yyasr_asr_feed(\n    switch_asr_handle_t *ah, void *data, unsigned int len, switch_asr_flag_t *flags)\n{\n\t// feed the text to the yyasr-\u003etext_buffer text buffer\n\treturn SWITCH_STATUS_SUCCESS;\n}\n\n\n/*\n * Voice Active detecting implementation.\n * Optimize the VAD by modify stop_audio_detect function.\n*/\nstatic switch_bool_t stop_audio_detect(\n    switch_asr_handle_t *ah, int16_t *data, unsigned int samples)\n{\n    return SWITCH_FALSE;\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flionsoul2014%2Ffreeswitch-asr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flionsoul2014%2Ffreeswitch-asr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flionsoul2014%2Ffreeswitch-asr/lists"}