{"id":16493609,"url":"https://github.com/petrroll/gnd-ai-tools","last_synced_at":"2026-05-10T21:47:40.819Z","repository":{"id":37605806,"uuid":"186305764","full_name":"petrroll/gnd-ai-tools","owner":"petrroll","description":"A series of smaller projects connected to a theatre performance in june 2019 by @gnd","archived":false,"fork":false,"pushed_at":"2023-03-24T22:48:42.000Z","size":20,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-11T22:17:14.821Z","etag":null,"topics":["human-computer-interaction","machine-learning","stt","tensorflow","tts"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/petrroll.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-12T21:25:52.000Z","updated_at":"2019-10-29T17:50:21.000Z","dependencies_parsed_at":"2024-11-12T20:45:23.367Z","dependency_job_id":"365484e4-f406-48bf-b58b-fca49608ba72","html_url":"https://github.com/petrroll/gnd-ai-tools","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/petrroll%2Fgnd-ai-tools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/petrroll%2Fgnd-ai-tools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/petrroll%2Fgnd-ai-tools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/petrroll%2Fgnd-ai-tools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/petrroll","download_url":"https://codeload.github.com/petrroll/gnd-ai-tools/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241335590,"owners_count":19946085,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["human-computer-interaction","machine-learning","stt","tensorflow","tts"],"created_at":"2024-10-11T14:09:57.327Z","updated_at":"2026-05-10T21:47:40.780Z","avatar_url":"https://github.com/petrroll.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"### About:\nSet of AI/ML tools for @gnd's theatre performances, mostly in NLP/Speech domain. \n\n**Remarks**: Most of it is very hacky, not fault tolerant, badly architectured, and potentially not cross-platform. Use with caution.\n\n### Setup:\n- Run `make prereq-apt` to install apt-dependencies\n- Run `make env` to setup python virtual environment / `pip3 install -r requirements.txt` to setup global \n\n#### Setup STT:\n- Set authentication for Google Cloud (e.g. `export GOOGLE_APPLICATION_CREDENTIALS=\u003cpath/to/credentials.json\u003e` or [any other possible way](https://cloud.google.com/docs/authentication/production)).\n\n### Architecture\n- All modules either consume / produce data / produce data based on input.\n- Producers (STT) are python generators that yield data\n- Consumers have `Consume(data)` method that can optionally return (producer based on input e.g. charnn) \n\n### Modules:\n\n#### General:\n- Each module can be run as CLI application for testing.\n\n#### stt.py:\n- Producer `Orthograph(lang, input_device, only_final)`.\n- `produce()` -\u003e generator that yields `(transcript, result.is_final)`\n- `--lang \u003cl\u003e`: sets STT language, defaults to  to `cs-CZ` (`en-US` works too).\n- `--only_final False`: controlls whether non-final transcripts are yielded as well (pause for a few seconds to obtain final transcript).\n- `--exit_key \u003ck\u003e`: exit-keyword, defaults to *ananas* (only for CLI testing).\n\n#### rnnGen.py: \n- Consumer `RnnGen(char_rnn_ckpt_dir, sample_length, remove_prime)`.\n- `produce(input)` -\u003e generates and immediately returns text primed by input.\n- `--char_rnn_ckpt_dir \u003cd\u003e`: directory containing a pretrained char-rnn model, defaults to `charrnn/save` (which by default doesn't exist, so make sure you either provide it or specify a valid path).\n- `--sample_length`: length of generated sequence.\n- `--remove_prime`: controlls whether the input is supposed to be cut from the beginning of generated result.\n\n#### tts.py:\n- Consumer `DummyVoice(lang)`\n- `consume(input)` -\u003e TTS plays audio of `input` string.\n- `--lang \u003cl\u003e`: sets TTS language, defaults to  to `cs-CZ` (`en-US` works too).\n- `--exit_key \u003ck\u003e`: exit-keyword, defaults to *ananas* (only for CLI testing).\n- Notes: for windows `pip install pypiwin32`; for linux just `make` (or see what it installs).\n\n#### ttsTacotron.py:\n- Consumer `TacotronVoice()`\n- `consume(input)` -\u003e Tacotron-2 plays audio of `input` string.\n- `--exit_key \u003ck\u003e`: exit-keyword, defaults to *ananas* (only for CLI testing).\n- Some/most [Tacotron-2](./Tacotron-2/README.md) params work.\n- Prerequisites: \n  - Install all [Tacotron-2 requirements](./Tacotron-2/README.md#how-to-start)\n  - Copy trained model checkpoints to `./Tacotron-2/logs-Tacotron/`\n- Notes: First TTS takes significantly longer to generate (pre-heating with empty string causes most models to generate hell-noise).\n\n### Wrappers:\n\n#### SocketReader.py:\n- Producer `SocketReader(host, port, sep)`.\n- `produce()` -\u003e yields messages converted to UTF-8 string read from socket one by one, blocks until it gets a full message that ends by separator.\n- `--host`: host for socket server, keep empty.\n- `--port`: port for socket server.\n- `--sep`: separator of messages.\n- Notes: Acts as socket server.\n\n#### SocketWriter.py:\n- Consumer `SocketWriter(host, port, sep)`.\n- `consume(msg)` -\u003e sends msg through socket as UTF-8 encoded string suffixed by separator.\n- `--host`: host of socket to connect to.\n- `--port`: port for socket server.\n- `--sep`: separator of messages.\n- Notes: Acts as socket client.\n\n### POCs:\n\n#### poc.py\nAtm, `poc.py` consists of a loop - you may speak, and what you say is transcribed through GC's Speech API into text, until a keyword is not recognized. At this point, the connection to GC is closed and the transcribed text (except the keyword) is forwarded to the running char-rnn language model. The generated text from the language model is spoken through a text-to-speech interface.\n\n- Args for `stt.py`, `tts.py`, `rnnGen.py` apply\n- `--next_key \u003ck\u003e`: move-on-keyword, defaults to *figaro*. If it's not set the RNN is immediately run on first final transcript.\n- `--say_primed False`: By default, the text spoken includes the primed text (the transcription) and the generated text. To say only what has been generated, set this to false.\n- Run `python3 poc.py  \u003cargs\u003e`\n\nYou may like to choose the keywords based on the language used.\n\nMake sure, that the speech transcriptions are in a character set, which belongs to the char-rnn model's vocabulary.\n\n#### poc_socketTTS.py:\n- PoC listening on socket (see `SocketReader.py` args) and sending messages to `tts.py` service (see its args). \n- Showcase of how to use modules \u0026 socketWrappers together. \n\n\n### Credits: \n- [Google Speech API samples](https://github.com/GoogleCloudPlatform/python-docs-samples/tree/master/speech/microphone)\n- @sherjilozair[/char-rnn-tensorflow](https://github.com/sherjilozair/char-rnn-tensorflow/blob/master/model.py)\n- @gnd, @SamMichalik (loads of code was lifted from [previous repo](https://github.com/gnd/cancer-works\n))\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpetrroll%2Fgnd-ai-tools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpetrroll%2Fgnd-ai-tools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpetrroll%2Fgnd-ai-tools/lists"}