{"id":13574272,"url":"https://github.com/Gavin-Development/GavinTraining","last_synced_at":"2025-04-04T14:32:11.917Z","repository":{"id":45947337,"uuid":"343075611","full_name":"Gavin-Development/GavinTraining","owner":"Gavin-Development","description":"This repo is for Training Gavin ONLY","archived":false,"fork":false,"pushed_at":"2023-11-16T21:02:20.000Z","size":14285,"stargazers_count":9,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-11-05T09:44:06.390Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Gavin-Development.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-02-28T10:22:13.000Z","updated_at":"2024-07-03T02:12:06.000Z","dependencies_parsed_at":"2023-11-07T18:44:57.087Z","dependency_job_id":"9d3d4199-f264-440e-a367-2ce671bb3978","html_url":"https://github.com/Gavin-Development/GavinTraining","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gavin-Development%2FGavinTraining","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gavin-Development%2FGavinTraining/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gavin-Development%2FGavinTraining/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gavin-Development%2FGavinTraining/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Gavin-Development","download_url":"https://codeload.github.com/Gavin-Development/GavinTraining/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247194233,"owners_count":20899449,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:00:49.244Z","updated_at":"2025-04-04T14:32:07.407Z","avatar_url":"https://github.com/Gavin-Development.png","language":"Python","funding_links":[],"categories":["Table of Contents"],"sub_categories":["AI - Natural Language Processing"],"readme":"# Gavin Training Repo\nThis Repo is for the Training of Gavin. A Transformer Based Chat Bot. \nAt current time he uses the \n[Reddit Comment Dataset](https://www.reddit.com/r/datasets/comments/3bxlg7/i_have_every_publicly_available_reddit_comment/).\n\n### Background\nStarted back in October 2019 (under a different repo), Gavin became [my (ScotSurvivor)](https://github.com/Scot-Survivor) project outside of studying.\nAt the time, being only 16 years old, Gavin ended up becoming a lot more than just a project, Gavin became the backbone to my applications towards university,\nA-Levels (UK examinations) \u0026 even taking an extra a-level to complete in just one year. Despite my age at the time, I still knew just how much work would be\nrequired to achieve the level of coherency \u0026 contextual awareness I was aiming for (in fact I still am working towards this!). \n\nAt present day Gavin has come a long way, integrating other papers \u0026 modules, even having some C++ modules (written by ShmarvDogg \u0026 myself) to speed up certain parts.\nSuch as, [GavinBackendDatasetUtils](https://github.com/Gavin-Development/GavinBackendDatasetUtils) as well as [GavinTokenizers](https://github.com/Gavin-Development/CPP-SubwordTextEncoder),\nwhich was adapted from the SubwordTextEncoder system that TensorflowDataset uses. \nFurthermore, Gavin now has ties with an incredible discord bot known as [Gerald](https://github.com/Gerald-Development/Barista-Gerald) Written by a  close friend of mine \n[Seb](https://github.com/Elementalmp4) Gerald reaches over 60,000 discord members (as of 26/10/2021), which also means, Gavin is being spoken to by this many members.\n\n### Overall Goals\nGavin's main goal is to be able to speak like your average redditor, while remaining at least some-what humble \u0026 polite. This goal is constantly growing in complexity\nas I aim for better \u0026 better coherency. This is being achieved in several ways, check out the whole [organisation](https://github.com/Gavin-Development) \u0026 the repos \nwithin for the individual goals.\n\n### Specific to this repo\nJust to be able to train Gavin, this repo also includes some Dataset tools, which are primarily written for my machine (this is due to [change](https://github.com/Gavin-Development/GavinTraining/tree/BetterDataHandling), \nworking on an ob#ject-oriented approach now.)\nFor specifics on the training script look at main.py.\n\n### Motivations \u0026 Inspirations\n- [Charles the AI.](https://twitter.com/charles_the_ai?lang=en)\n-\n- [Sentdex Deep Learning Tutorial.](https://pythonprogramming.net/chatbot-deep-learning-python-tensorflow/)\n-\n- [Sentdex The Man, the myth and the legend.](https://github.com/Sentdex)\n-\n- [Ghosty!](https://github.com/TheNitpickyCloud) Well respected Member of sentdex discord server,\n  he helps test and develop Gavin in many ways. As well as keeping me motivated to work on Gavin,\n  he also helps Give ideas for [Gavin's Discord frontend.](https://github.com/Gavin-Development/GavinDiscordFrontEnd).\n-\n- [ShmarvDogg.](https://github.com/Shmarvadon) Without him this project wouldn't be possible, \nspecial thanks to him for lending me his hardware to train and continue to develop Gavin.\n-\n- [WhoIsAbishag.](https://github.com/WhoIsAbishag) For his work with Irene the ChatBot \n  (AI based on OpenAI). He incites, coherent and contextual responses from Chat Bots.\n-\n- [Transformer Model.](https://blog.tensorflow.org/2019/05/transformer-chatbot-tutorial-with-tensorflow-2.html) Some designs \nare re-made or reforged, this was a key inspiration.\n-\n- [Performer Paper.](https://arxiv.org/pdf/2009.14794.pdf) Vital to speed improvements during in train.\n  \n### Build Status\n(TODO)\n\n### Framework \u0026 Technologies\n- [TensorFlow](https://github.com/tensorflow/tensorflow) \n- [TensorFlow Datasets](https://github.com/tensorflow/datasets)\n- [Numpy](https://github.com/numpy/numpy)\n- [GavinBackend](https://github.com/Gavin-Development/GavinBackend)\n- [GavinDatasetUtils](https://github.com/Gavin-Development/GavinBackendDatasetUtils)\n- [Cuda](https://developer.nvidia.com/cuda-zone)\n- [Cudnn](https://developer.nvidia.com/cudnn)\n    \n### Features\n(TODO)\n\n### Code Examples\n(TODO)\n\n### Tests\n- ![Input: you're dumb Output: * laughs * I don ' t get it .](https://github.com/Scot-Survivor/GavinTraining/blob/master/funny-responses/image1.PNG?raw=true)\n- ![Input: hi Output: * kisses back * Input: kiss Output: * laughs * I love you too .](https://github.com/Scot-Survivor/GavinTraining/blob/master/funny-responses/image2.png?raw=true)\n- ![Input: what is the point of life? Output: I think it was a joke .](https://github.com/Scot-Survivor/GavinTraining/blob/master/funny-responses/image3.png?raw=true)\n- ![Input: get smart Output: I don ' t think so .](https://github.com/Scot-Survivor/GavinTraining/blob/master/funny-responses/image4.png?raw=true)\n\n\n### Contribute\n(TODO)\n\n### Credits\n- \"REFORMER: THE EFFICIENT TRANSFORMER\": https://arxiv.org/pdf/2001.04451.pdf\n- \"Practical and Optimal LSH for Angular Distance\": https://arxiv.org/pdf/1509.02897.pdf\n- \"When and Why is Document-level Context Useful in Neural Machine Translation?\": https://arxiv.org/pdf/1910.00294.pdf\n- \"Attention Is All You Need\": https://arxiv.org/pdf/1706.03762.pdf\n- \"Improved Transformer Architecture for Sequence to Sequence Translation\": https://www.cs.princeton.edu/sites/default/files/austin_wang_spring_2019.pdf- \"Combining Local and Document-Level Context: The LMU Munich Neural Machine Translation System at WMT19\": https://www.aclweb.org/anthology/W19-5345.pdf\n- \"Improving the Transformer Translation Model with Document-Level Context\": https://arxiv.org/pdf/1810.03581.pdf\n- \"Illustrating The Reformer\": https://towardsdatascience.com/illustrating-the-reformer-393575ac6ba0\n- \"Reformer: The Efficient Transformer: https://openreview.net/pdf?id=rkgNKkHtvB\"\n- \"Rethinking attention with performers\": \n### Licence\n[GNU GPLv3](https://www.gnu.org/licenses/gpl-3.0.txt). Should have a copy with this software.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FGavin-Development%2FGavinTraining","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FGavin-Development%2FGavinTraining","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FGavin-Development%2FGavinTraining/lists"}