{"id":17498331,"url":"https://github.com/simonepri/fever-transformers","last_synced_at":"2025-04-28T16:09:52.121Z","repository":{"id":66098951,"uuid":"230756731","full_name":"simonepri/fever-transformers","owner":"simonepri","description":"📄 Evidence Retrieval and Claim Verification for the FEVER shared task using Transformer Networks","archived":false,"fork":false,"pushed_at":"2020-02-21T22:51:16.000Z","size":70,"stargazers_count":12,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-28T16:09:46.990Z","etag":null,"topics":["fever","machine-learning","ml","pytorch","shared-task","transformer-architecture"],"latest_commit_sha":null,"homepage":"http://fever.ai/2018/task.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simonepri.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-29T13:50:33.000Z","updated_at":"2025-01-27T18:22:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"311de38e-ddf5-4f57-894e-c5ca2cb00b01","html_url":"https://github.com/simonepri/fever-transformers","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonepri%2Ffever-transformers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonepri%2Ffever-transformers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonepri%2Ffever-transformers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonepri%2Ffever-transformers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simonepri","download_url":"https://codeload.github.com/simonepri/fever-transformers/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251342724,"owners_count":21574245,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fever","machine-learning","ml","pytorch","shared-task","transformer-architecture"],"created_at":"2024-10-19T16:28:07.365Z","updated_at":"2025-04-28T16:09:52.089Z","avatar_url":"https://github.com/simonepri.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003cb\u003efever-transformers\u003c/b\u003e\n\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  📄 Evidence Retrieval and Claim Verification for the FEVER shared task using Transformer Networks\n  \u003cbr/\u003e\n\n  \u003csub\u003e\n    Available pre-trained models: Bert, RoBERTa, Albert, XLM-RoBERTa\n  \u003c/sub\u003e\n\u003c/p\u003e\n\n# FEVER Shared Task\nThe [FEVER Shared Task][link:fever] is a task in which participants are asked to classify a `claim` (e.g. \"The number 42 is the Answer to the Ultimate Question of Life, the Universe, and Everything\") into `SUPPORTS`, `REFUTES` or `NOT ENOUGH INFORMATION` while also providing the relevant evidence sentences from Wikipedia (only).\n\n## Run the pipeline\n\nThe project is provided with a CLI that allows you to run the pipeline on your machine with ease.\n\nIf you want to run it locally, you need to run the following commands.\n\n```bash\n# Install pipenv (i.e. a better python package manager).\npip3 install pipenv\n\n# Download the source code from GitHub and cd into the folder.\ngit clone --branch master https://github.com/simonepri/fever-transformers\ncd fever-transformers\n\n# Run the entire pipeline (it also installs the required dependencies).\nbash scripts/pipeline.sh\n```\n\n\u003e NB: The only requirement to run the mentioned commands, is that you have python3 and pip3 installed on your machine. You can also skip the pipenv installation if you already have it on your machine.\n\nAlternatively, the CLI allows you to run the single tasks individually as follows.\n\n```bash\n# Install required dependencies in a virtual environment.\nbash scripts/pipeline.sh install_deps\n\n# Download the fever shared task.\nbash scripts/pipeline.sh download_fever\n\n# Build an sql database using the fever wikipedia dump.\nbash scripts/pipeline.sh build_db\n\n# Process the datasets through the UKP-Athene document retrieval model.\n# Alternatively you can run bash scripts/release.sh --download output ukp-athene\nbash scripts/pipeline.sh document_retrieval\n\n# Process the datasets through the transformer network sentence retrieval model.\n# See below for the possible values of model type and name.\nbash scripts/pipeline.sh sentence_retrieval --model-type bert --model-name bert-base-cased\n\n# Process the datasets through the transformer claim verification model.\n# See below for the possible values of model type and name.\nbash scripts/pipeline.sh claim_verification --model-type bert --model-name bert-base-cased\n\n# Combine the results from the previous steps to generate the final submission files.\nbash scripts/pipeline.sh generate_submission --force\n```\n\n\u003e NB: If you run a task multiple times, the CLI will execute actions for that task on an as-needed basis (e.g. if the finetuned model is already available it wont start the finetuning process again).\n\nThe following flags can be used to modify the behavior of the CLI.\n\n| Flag | Purpose |\n|------|---------|\n| `‑‑force` | Delete the folder containing the data of the task and then start the task |\n|  `‑‑model‑type` | Set the transformer model to use. It can be one of: \u003cbr/\u003e`bert`, `xlnet`, `xlm`, `roberta`, `distilbert`, `albert`, `xlmroberta` |\n| `‑‑model‑name` | Set the pretrained checkpoint of the model to use. It can be one of: \u003cbr/\u003e `bert-base-uncased`, `bert-large-uncased`, `bert-base-cased`, `bert-large-cased`, `bert-base-multilingual-uncased`, `bert-base-multilingual-cased`, `bert-base-chinese`, `bert-base-german-cased`, `bert-large-uncased-whole-word-masking`, `bert-large-cased-whole-word-masking`, `bert-large-uncased-whole-word-masking-finetuned-squad`, `bert-large-cased-whole-word-masking-finetuned-squad`, `bert-base-cased-finetuned-mrpc`, `bert-base-german-dbmdz-cased`, `bert-base-german-dbmdz-uncased`, `bert-base-japanese`, `bert-base-japanese-whole-word-masking`, `bert-base-japanese-char`, `bert-base-japanese-char-whole-word-masking`, `bert-base-finnish-cased-v1`, `bert-base-finnish-uncased-v1`, `xlnet-base-cased`, `xlnet-large-cased`, `xlm-mlm-en-2048`, `xlm-mlm-ende-1024`, `xlm-mlm-enfr-1024`, `xlm-mlm-enro-1024`, `xlm-mlm-tlm-xnli15-1024`, `xlm-mlm-xnli15-1024`, `xlm-clm-enfr-1024`, `xlm-clm-ende-1024`, `xlm-mlm-17-1280`, `xlm-mlm-100-1280`, `roberta-base`, `roberta-large`, `roberta-large-mnli`, `distilroberta-base`, `roberta-base-openai-detector`, `roberta-large-openai-detector`, `distilbert-base-uncased`, `distilbert-base-uncased-distilled-squad, distilbert-base-german-cased`, `distilbert-base-multilingual-cased`, `albert-base-v1`, `albert-large-v1, albert-xlarge-v1`, `albert-xxlarge-v1`, `albert-base-v2, albert-large-v2`, `albert-xlarge-v2`, `albert-xxlarge-v2, xlm-roberta-base`, `xlm-roberta-large`, `xlm-roberta-large-finetuned-conll02-dutch`, `xlm-roberta-large-finetuned-conll02-spanish`, `xlm-roberta-large-finetuned-conll03-english`, `xlm-roberta-large-finetuned-conll03-german` |\n\n# Download pretrained models\n\nPretrained models for the sentence retrieval and claim verification steps of the pipeline are available in the [release page][release].\n\nAlternatively they can be downloaded using the provided CLI as follows:\n\n```bash\nbash scripts/release.sh --download model \"MODEL_NAME\"\n```\nWhere MODEL_NAME can be one of:\n`ukp-athene+albert-base-v2+albert-base-v2`,\n`ukp-athene+bert-base-cased+bert-base-cased`,\n`ukp-athene+roberta-base+roberta-base`,\n`ukp-athene+xlm-roberta-base+xlm-roberta-base`\n\n## Authors\n- **Simone Primarosa** - *Github* ([@simonepri][github:simonepri]) • *Twitter* ([@simoneprimarosa][twitter:simoneprimarosa])\n\n## License\nThis project is licensed under the MIT License - see the [license][license] file for details.\nSome of the files are licensed with the BSD or the Apache-2.0 license.\nPlease refer to the header of the files for more.\n\n\u003c!-- Links --\u003e\n[license]: https://github.com/simonepri/fever-transformers/tree/master/license\n[release]: https://github.com/simonepri/fever-transformers/releases\n\n[github:simonepri]: https://github.com/simonepri\n[twitter:simoneprimarosa]: http://twitter.com/intent/user?screen_name=simoneprimarosa\n\n[run:colab]: https://colab.research.google.com/drive/1hhJL-VQ__Qh_HsDb6WvflTlNJnEXTlR9\n\n[link:fever]: http://fever.ai\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonepri%2Ffever-transformers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimonepri%2Ffever-transformers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonepri%2Ffever-transformers/lists"}