{"id":17723419,"url":"https://github.com/vivek3141/streamer-ai","last_synced_at":"2025-03-14T05:31:57.375Z","repository":{"id":96163067,"uuid":"152265459","full_name":"vivek3141/streamer-ai","owner":"vivek3141","description":"Python based AI that uses Deep Neural Networks, Neuroevolution and Streamlabs APIs to live stream games while commentating over them at the same time","archived":false,"fork":false,"pushed_at":"2019-02-06T05:25:48.000Z","size":4540,"stargazers_count":11,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2023-03-02T04:25:50.829Z","etag":null,"topics":["livestream","machine-learning","neat","neat-python","neruoevolution","neural-network","python","streamlabs","streamlabs-api"],"latest_commit_sha":null,"homepage":"https://youtube.com/vcubingx","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vivek3141.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-10-09T14:26:35.000Z","updated_at":"2023-02-16T11:45:00.000Z","dependencies_parsed_at":null,"dependency_job_id":"6787f8dc-9f0b-4c80-81e8-95970300597c","html_url":"https://github.com/vivek3141/streamer-ai","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vivek3141%2Fstreamer-ai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vivek3141%2Fstreamer-ai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vivek3141%2Fstreamer-ai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vivek3141%2Fstreamer-ai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vivek3141","download_url":"https://codeload.github.com/vivek3141/streamer-ai/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243532517,"owners_count":20306151,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["livestream","machine-learning","neat","neat-python","neruoevolution","neural-network","python","streamlabs","streamlabs-api"],"created_at":"2024-10-25T15:42:55.650Z","updated_at":"2025-03-14T05:31:57.360Z","avatar_url":"https://github.com/vivek3141.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Streamer-AI\nPython based AI that uses Deep Neural Networks, Neuroevolution and Streamlabs APIs to live stream games while commentating over them at the same time.\n\n## Running The program\nUsage:\n```bash\npython3 main.py --gen [generations] --file [file_name] \\\n    --config [config]\n```\nThis writes to a file called `say.txt`.\n#### --gen\nSpecify the number of generations to run the evolution for\n#### --file\nFile to load the checkpoint from. `checkpoints/` has saved checkpoints for generation\n`2492` and `2284`.\n#### --config\nConfiguration file for NEAT. `neat.config` contains the one used to train.\n\n## Requirements\nInstall the requirements with\n```bash\nsudo make\n```\n\n#### Alternative\nInstall the python requirements with\n```bash\npip install -r requirements.txt\n```\nand install `mpg123`, `fceux` and add them to `PATH`.\n\n## Getting the API Key\n* Go to `streamlabs.com` then sign in using your account. Then go to API settings, \ncreate a new app and copy the client ID and client Secret. Click the `Sample Authentication URL` below and then copy\nthe code in the URL.\n\n* Call a `POST` request to `/token`. Make sure to use the same Redirect URI as set up in the app. Example:\n```bash\ncurl --request POST \\\n     --url 'https://streamlabs.com/api/v1.0/token' \\\n     -d 'grant_type=grant_type\u0026client_id=client_id\u0026client_secret=client_secret\u0026redirect_uri=redirect_uri'\n ```\n Example return:\n ```bash\n {\n  access_token: 'loXk8FTOFwKfrLP3bGCnJldBxuGX03a03iQdxR8A',\n  token_type: 'Bearer',\n  refresh_token: 'IXCGDha46Q4eHBKrijmAqUwScbsMSuBy9IopXp80'\n}\n ```\n \n * Authorize this `access_token` for the scope `socket.token` using `/authorize`. Example:\n ```bash\n curl --request GET \\\n  --url 'https://streamlabs.com/api/v1.0/authorize?response_type=response_type\u0026client_id=client_id\u0026redirect_uri=redirect_uri\u0026scope=socket.token'\n  ```\n * Use the `access_token` to get a `socket_token`. Call a `GET` request to `/socket/token`. Example:\n```bash\n curl --request GET \\\n  --url 'https://streamlabs.com/api/v1.0/socket/token?access_token=access_token'\n```\nYou can also use python's `requests` library instead of `curl`\n* Install with `pip install requests`\n* Example to get `socket_token`:\n```python\nimport requests\n\nurl = \"https://streamlabs.com/api/v1.0/socket/token\"\n\nquerystring = {\"access_token\":\"access_token\"}\n\nresponse = requests.request(\"GET\", url, params=querystring)\n\nprint(response.text)\n```\nPut the `socket_token` in a file called `config`, with the following format, shown in\n`config.example`:\n```\nsocket_token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ\n```\n## How it works\nFor an explanation for how the game runs, watch [this video](https://www.youtube.com/watch?v=hNDkjy2rXG4\u0026).\n### Short Explanation\n#### How the game is played\nThis program uses a mathematical model, called a neural network, which simulates the brain of a human being. \nA neural network works by taking inputs and outputting probabilities for each of the outputs. This can be accomplished\nby using a sigmoid function. \u003cbr\u003e\u003cbr\u003e\n![Sigmoid](https://qph.fs.quoracdn.net/main-qimg-07066668c05a556f1ff25040414a32b7)\n\u003cbr\u003e\u003cbr\u003e\n`Neuroevolution of Augmenting Topologies`, or `NEAT` is what this project uses. The way standard\n`neuroevolution` works is by randomly initializing a population of neural networks and\nusing survival of the fittest to get the best model. The best networks in each generations\nare bred and some mutations are introduced. `NEAT` introduces features like speciation to\nmake a much more effective neuroevolution model. Neuroevolution is known to do better than standard\nreinforcement learning models.\u003cbr\u003e\n#### How the commentary works\nUsing a `socketio` client for python, we can establish a connection to `Streamlab's Socket API`.\nThis API returns every alert in JSON format. We can decode these JSONs to return statements\nthanking the donator, subscriber, member, superchat donator, etc. \u003cbr\u003eExample:\n```python\nimport socketio\n\nURL = \"http://example.com/socket-api\" # Change this to whatever Socket API you are using\nsio = socketio.Client()\n\n@sio.on('connect')\ndef connect():\n    # This function is called when the connection is established\n    print(\"Connected\")\n    \n# The event in quotes depends on your API, check the documentation\n@sio.on(\"event\")\ndef event(data):\n    print(data)\n\n# Connect to the URL specified above\nsio.connect(URL)\n```\nThe rest of the audio uses random number generations to generate the sentences. A sentence is picked from a list of sentences, \nstored in `get_sentences.py`. \u003cbr\u003e\n#### Playing the audio\nThe audio is converted to speech using `gtts`, Google's text-to-speech converter.\nThis is far more superior than `pyttsx`, which bugs out quite a bit.\u003cbr\u003e\nThe audio is then played using `mpg123`, called by `os.system`.\n\n## Future Work\n* Integrate the streaming directly into the program, preventing the need for OBS.\n* Prevent the game reopening after each genome.\n* Improve commentary in general.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvivek3141%2Fstreamer-ai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvivek3141%2Fstreamer-ai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvivek3141%2Fstreamer-ai/lists"}