{"id":18583254,"url":"https://github.com/lcharleslaing/nlp_processing","last_synced_at":"2025-06-11T21:06:10.410Z","repository":{"id":156263613,"uuid":"626050575","full_name":"lcharleslaing/nlp_processing","owner":"lcharleslaing","description":"This app is a text processor designed to work with GPT models. It helps users save tokens while using GPT models by processing the input text. The app provides a GUI with an input box for pasting the original text, a dropdown menu for selecting the GPT model, and a button to process the text and a copy button to paste into ChatGPT.","archived":false,"fork":false,"pushed_at":"2023-05-08T12:46:05.000Z","size":16,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-16T04:37:16.190Z","etag":null,"topics":["chatgpt","gpt-4","gui"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lcharleslaing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-10T17:35:06.000Z","updated_at":"2023-05-20T12:08:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"1560ce41-ecff-46bc-a556-be8135c17a69","html_url":"https://github.com/lcharleslaing/nlp_processing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lcharleslaing/nlp_processing","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lcharleslaing%2Fnlp_processing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lcharleslaing%2Fnlp_processing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lcharleslaing%2Fnlp_processing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lcharleslaing%2Fnlp_processing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lcharleslaing","download_url":"https://codeload.github.com/lcharleslaing/nlp_processing/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lcharleslaing%2Fnlp_processing/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259340568,"owners_count":22843019,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatgpt","gpt-4","gui"],"created_at":"2024-11-07T00:21:31.922Z","updated_at":"2025-06-11T21:06:10.363Z","avatar_url":"https://github.com/lcharleslaing.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GPT Token Saver\n\nThis app is a text processor designed to work with GPT models (e.g., GPT-4). It does not work with GPT-3.5-Turbo as it doesn't understand the processed text the same.  It helps users save tokens while using GPT models by processing the input text. The app provides a GUI with an input box for pasting the original text, a dropdown menu for selecting the GPT model, and a button to process the text. After processing, it displays the processed text, the number of tokens used in the original and processed text, the percentage of tokens saved, and the cost based on the selected GPT model. The app also offers the option to copy the processed text to the clipboard, clear the input, and exit the application.  There is also a button if you want to preprocess a complete file and save it as a .txt file.  You choose the file and the location to be saved.\n\nThe application is built using Python and Tkinter for the user interface.\n\n## Features\n\n- Remove stopwords from the input text\n- Calculate the number of tokens before and after processing\n- Calculate the cost savings based on a fixed token price\n- Copy the processed text to the clipboard\n- Clear input field\n- Exit the application\n\n## Dependencies\n\n- Python 3.7+\n- Tkinter\n- ttkthemes\n- nltk\n- transformers\n- pyperclip\n\n## Installation\n\n1. Clone the repository or download the source code.\n\n2. Install the required Python packages:\n\n```bash\npip install -r requirements.txt\n```\n\n3. Run the `nltk_setup.py` script to download the necessary NLTK data:\n\n```bash\npython nltk_setup.py\n```\n\n4. Run the `main_application.py` script to launch the application:\n\n```bash\npython main_application.py\n```\n\n## Usage\n\n1. Paste your input text into the \"Paste text\" field.\n2. Click the \"Process Text\" button to process the text and display the results.\n3. Review the processed text, token information, and cost savings.\n4. Click the \"Copy Processed Text\" button to copy the result to your clipboard.\n5. Use the \"Clear\" button to clear the input field for a new text input.\n6. Click the \"Exit\" button to close the application.\n\n## License\n\nThis project is licensed under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flcharleslaing%2Fnlp_processing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flcharleslaing%2Fnlp_processing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flcharleslaing%2Fnlp_processing/lists"}