{"id":16200333,"url":"https://github.com/andfanilo/writing-with-gpt2","last_synced_at":"2025-03-19T05:30:56.130Z","repository":{"id":50259426,"uuid":"298335385","full_name":"andfanilo/writing-with-gpt2","owner":"andfanilo","description":"Reproducing \"Writing with Transformer\" demo, using aitextgen/FastAPI in backend, Quill/React in frontend","archived":false,"fork":false,"pushed_at":"2021-01-23T15:39:14.000Z","size":131,"stargazers_count":28,"open_issues_count":0,"forks_count":12,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-15T04:33:29.404Z","etag":null,"topics":["aitextgen","fastapi","gpt2","python","quill","react"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/andfanilo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-09-24T16:30:54.000Z","updated_at":"2024-03-30T10:50:28.000Z","dependencies_parsed_at":"2022-08-25T14:12:12.698Z","dependency_job_id":null,"html_url":"https://github.com/andfanilo/writing-with-gpt2","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andfanilo%2Fwriting-with-gpt2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andfanilo%2Fwriting-with-gpt2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andfanilo%2Fwriting-with-gpt2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andfanilo%2Fwriting-with-gpt2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/andfanilo","download_url":"https://codeload.github.com/andfanilo/writing-with-gpt2/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244364717,"owners_count":20441458,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aitextgen","fastapi","gpt2","python","quill","react"],"created_at":"2024-10-10T09:29:56.918Z","updated_at":"2025-03-19T05:30:55.819Z","avatar_url":"https://github.com/andfanilo.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Writing with GPT-2\n\n![](./img/demo.png)\n\n![](./img/diagram.png)\n\n## Development\n\n- Ensure you have [Python 3.6/3.7](https://www.python.org/downloads/), [Node.js](https://nodejs.org), and [npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) installed.\n\n### Python Backend\n\nMake sure you are in the `backend` folder:\n\n```sh\ncd backend/\n```\n\nInstall a virtual environment:\n\n```sh\n# If using venv\npython3 -m venv venv\n. venv/bin/activate\n\n# If using conda\nconda create -n write-with-gpt2 python=3.7\nconda activate write-with-gpt2\n\n# On Windows I use Conda to install pytorch separately\nconda install pytorch cpuonly -c pytorch\n\n# When environment is activated\npip install -r requirements.txt\npython aitextgen_app.py\n```\n\nTo run in hot module reloading mode:\n\n```sh\nuvicorn aitextgen_app:app --host 0.0.0.0 --reload\n```\n\nTo run with multiple workers:\n\n```sh\nuvicorn aitextgen_app:app --host 0.0.0.0 --workers 4\n```\n\nRuns on http://localhost:8000. You can consult interactive API on http://localhost:8000/docs.\n\nConfiguration is made via environment variable or `.env` file. Available are:\n\n- **MODEL_NAME**:\n  - to use a custom model, point to the location of the `pytorch_model.bin`.\n    You will also need to pass `config.json` through `CONFIG_FILE`.\n  - otherwise model from Huggingface's [repository of models](https://huggingface.co/), defaults to `distilgpt2`.\n- **CONFIG_FILE**: path to JSON file of model architecture.\n- **USE_GPU**: `True` to generate text from GPU.\n\n#### From gpt-2-simple to Pytorch\n\nTo convert gpt-2-simple model to Pytorch, see [Importing from gpt-2-simple](https://docs.aitextgen.io/gpt-2-simple/):\n\n```sh\ntransformers-cli convert --model_type gpt2 --tf_checkpoint checkpoint/run1 --pytorch_dump_output pytorch --config checkpoint/run1/hparams.json\n```\n\nThis will put a `pytorch_model.bin` and `config.json` in the pytorch folder, which is what you'll need to pass to `.env` file to load the model.\n\n#### Run gpt-2-simple version\n\nAdded back the older `gpt-2-simple` version we add in `backend/gpt2_app`.\n\nTo download a model: \n\n```python\nimport gpt_2_simple as gpt2\ngpt2.download_gpt2(model_name='124M')\n```\n\nTo run app:\n\n```sh\nset MODEL_NAME=124M\nuvicorn aitextgen_app:app --host 0.0.0.0\n```\n\nSet `MODEL_NAME` to any model folder inside `models`, or edit `.env`.\n\n#### Streamlit Debug\n\nYou can run the Streamlit app to debug the model.\n\n```\nstreamlit run st_app.py\n```\n\n### React Frontend\n\nMake sure you are in the frontend folder, and ensure backend API is working.\n\n```sh\ncd frontend/\n```\n\n```sh\nnpm install # Install npm dependencies\nnpm run start # Start Webpack dev server\n```\n\nWeb app now available on http://localhost:3000.\n\n#### Building the frontend\n\nTo create a production build:\n\n```sh\nnpm run build\n```\n\nNow your React built app will be statically served by FastAPI on `http://localhost:8000/app` along with the other APIs. You don't need to run the Webpack devserver anymore.\n\n### Using GPU\n\nMiniconda/Anaconda recommended on Windows.\n\nconda command : `conda install pytorch cudatoolkit=10.2 -c pytorch`.\n\nIf you [install manually](https://developer.nvidia.com/cuda-toolkit), you can check your currently installed CUDA toolkit version with `nvcc --version`. Once you have CUDA toolkit installed, you can verify it by running `nvidia-smi`.\n\n**Beware**: after installing CUDA, it seems you shouldn't try to update GPU driver though GeForce or else you'll have to reinstall CUDA toolkit ?\n\n## References\n\n- [Write With Transformer](https://transformer.huggingface.co/doc/distil-gpt2)\n- [React-Quill-Demo](https://codesandbox.io/s/tn2x3)\n- [How To Create a React + Flask Project](https://blog.miguelgrinberg.com/post/how-to-create-a-react--flask-project)\n- [How to Deploy a React + Flask Project](https://blog.miguelgrinberg.com/post/how-to-deploy-a-react--flask-project)\n- [Interactive Playground - Autosave](https://quilljs.com/playground/#autosave)\n- [Mentions implementation](https://github.com/zenoamaro/react-quill/issues/324)\n- [Cloning Medium with Parchment](https://quilljs.com/guides/cloning-medium-with-parchment/)\n- [gpt-2-cloud-run](https://github.com/minimaxir/gpt-2-cloud-run)\n- [How To Make Custom AI-Generated Text With GPT-2](https://minimaxir.com/2019/09/howto-gpt2/)\n- [How to generate text without fintetune?](https://github.com/minimaxir/gpt-2-simple/issues/10)\n- [aitextgen](https://docs.aitextgen.io/)\n- [Setting up your PC/Workstation for Deep Learning: Tensorflow and PyTorch — Windows](https://towardsdatascience.com/setting-up-your-pc-workstation-for-deep-learning-tensorflow-and-pytorch-windows-9099b96035cb)\n- [CUDA Installation Guide for Microsoft Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandfanilo%2Fwriting-with-gpt2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fandfanilo%2Fwriting-with-gpt2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandfanilo%2Fwriting-with-gpt2/lists"}