{"id":19055795,"url":"https://github.com/srm47/gpt2-paraphraser-comparisons","last_synced_at":"2026-07-29T18:31:06.437Z","repository":{"id":151073035,"uuid":"568061816","full_name":"SRM47/gpt2-paraphraser-comparisons","owner":"SRM47","description":"Finetune GPT-2 models for paraphrasing and compare its outputs with other language models","archived":false,"fork":false,"pushed_at":"2023-01-13T17:35:15.000Z","size":9054,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-08T13:54:46.346Z","etag":null,"topics":["bert-model","fine-tuning","huggingface","huggingface-transformers","nlp","paraphrase-generation","summarization"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SRM47.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-19T10:31:21.000Z","updated_at":"2023-12-10T03:42:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"c8d0dce7-6de2-4683-853a-c8d83a4f4db6","html_url":"https://github.com/SRM47/gpt2-paraphraser-comparisons","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SRM47/gpt2-paraphraser-comparisons","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SRM47%2Fgpt2-paraphraser-comparisons","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SRM47%2Fgpt2-paraphraser-comparisons/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SRM47%2Fgpt2-paraphraser-comparisons/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SRM47%2Fgpt2-paraphraser-comparisons/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SRM47","download_url":"https://codeload.github.com/SRM47/gpt2-paraphraser-comparisons/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SRM47%2Fgpt2-paraphraser-comparisons/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":36045787,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-29T02:00:04.910Z","response_time":95,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert-model","fine-tuning","huggingface","huggingface-transformers","nlp","paraphrase-generation","summarization"],"created_at":"2024-11-08T23:47:02.656Z","updated_at":"2026-07-29T18:31:06.351Z","avatar_url":"https://github.com/SRM47.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gpt2-paraphraser-comparisons\nFinetune GPT-2 Models for paraphrasing and compare them to PEGASUS and BART\n\n\n## Create Datasets\nUse the script ``create_dataset.ipynb`` to create the dataset in the file ``combined.txt``. Each line contains the following: ``\u003cs\u003eS\u003c/s\u003e\u003e\u003e\u003e\u003e\u003cp\u003eP\u003c/p\u003e``, where ``S`` and ``P`` are paraphrased sentences. Sentences pairs are gathered from three different datasets available on huggingface.co\n\n- TaPaCo (en) https://huggingface.co/datasets/tapaco\n- Google PAWS https://huggingface.co/datasets/paws\n- Quora https://huggingface.co/datasets/quora\n\n## Finetune GPT-2 Models\n\nFinetuned three different sized GPT 2 models for sentence level paraphrasing using the ``Trainer()`` API.\nModels available on huggingface:\n- SRM47/gpt2-paraphraser\n- SRM47/gpt2-medium-paraphraser\n- SRM47/gpt2-large-paraphraser\n\n\n## Evaluate Models\nTo evaluate the finetuned GPT-2 models and other models, use the ``eval_models.ipynb`` script\n\n## Results Analysis\nSee the paper ``final.pdf`` to read about the results of this investigation.\n\nAs of recent, large language models, particularly a part of the Generative Pre-Trained series, have demonstrated themselves to be powerful text generation models. Models such as GPT-2 (Radford et al., 2018) reveal that large language models have strong zero-shot capabilities in a variety of downstream natural language pro- cessing tasks. Other models, built for sequence to sequence modeling, such as PEGASUS, and BART have profound text summarization capa- bilities which can be adapted to paraphrasing. In this paper, I present an effective method for adapting GPT-2 for paraphrasing, and compare its paraphrasing outputs to fine tuned BART and PEGASUS based models from huggingface. Results show that GPT-2 based models produce less diverse paraphrases than PEGASUS and BART; GPT-2 based paraphrases do not alter lexical form as much as PEGASUS does.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrm47%2Fgpt2-paraphraser-comparisons","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsrm47%2Fgpt2-paraphraser-comparisons","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrm47%2Fgpt2-paraphraser-comparisons/lists"}