{"id":17798007,"url":"https://github.com/yuvalpinter/rationale_analysis","last_synced_at":"2025-04-02T03:23:02.886Z","repository":{"id":93102388,"uuid":"222000886","full_name":"yuvalpinter/rationale_analysis","owner":"yuvalpinter","description":null,"archived":false,"fork":false,"pushed_at":"2019-11-27T14:54:30.000Z","size":15305,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-07T18:16:12.455Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yuvalpinter.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-15T20:48:41.000Z","updated_at":"2020-05-02T20:52:17.000Z","dependencies_parsed_at":"2023-03-12T23:00:56.130Z","dependency_job_id":null,"html_url":"https://github.com/yuvalpinter/rationale_analysis","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuvalpinter%2Frationale_analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuvalpinter%2Frationale_analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuvalpinter%2Frationale_analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuvalpinter%2Frationale_analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yuvalpinter","download_url":"https://codeload.github.com/yuvalpinter/rationale_analysis/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246748124,"owners_count":20827265,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-27T11:58:35.543Z","updated_at":"2025-04-02T03:23:02.866Z","avatar_url":"https://github.com/yuvalpinter.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Suggested use is to use Anaconda and do `pip install -r requirements.txt` .\n\n1. `data` : Folder to store datasets\n2. `Rationale_Analysis/models` : Folder to store allennlp models\n    1. `classifiers` : Models that do actually learning \n    2. `saliency_scorer` : Takes a trained model and return saliency scorers for inputs\n    3. `rationale_extractors` : Models that take saliency scores and generate rationales in form readable by `rationale_reader.py`\n    4. `base_predictor.py` : Simple predictor to use with allennlp predict command as needed\n3. `Rationale_Analysis/subcommands` : Subcommands to run saliency and rationale extractors since allennlp existingcommand semantics doesn't map quite as well to what we wanna do.\n4. `Rationale_Analysis/training_config` : Contains jsonnet training configs to use with allennlp for each of the three types of models above.\n5. `Rationale_Analysis/commands` : Actual bash scripts to run stuff.\n6. `Rationale_Analysis/data/dataset_readers` : Contains dataset readers to work with Allennlp.\n    1. `rationale_reader.py` : Code to load actual datasets (jsonl with 3 fields - document, query, label)\n    2. `saliency_reader.py` : Read output of Saliency scorer to pass into rationale extractors.\n\nExperimental Setup\n================== \n\n1. Train any model \n\n```bash\nCUDA_DEVICE=0 \\\nDATASET_NAME=\u003cdataset name\u003e \\\nCLASSIFIER=\u003cclassifier type\u003e \\\nDATA_BASE_PATH=\u003cpath to data\u003e \\\nEXP_NAME=\u003cyour experiment name\u003e \\\nbash Rationale_Analysis/commands/model_train_script.sh\n```\n\nYou `path to data` folder should contain three files - {train/dev/test}.jsonl . Each should be a list of dicts containing atleast two fields - \n\n```python\n{\n    \"document\" : str,\n    \"label\" : str,\n    \"query\" : Optional[str]\n}\n```\n    \nOutput generated in `outputs/\u003cclassifier type\u003e/\u003cdataset name\u003e/\u003cyour experiment name\u003e/` .\n\n2. Generate saliency scores\n\n```bash\nCUDA_DEVICE=0 \\\nDATASET_NAME=SST \\\nCLASSIFIER=\u003cclassifier type\u003e \\\nDATA_BASE_PATH=Datasets/SST/data \\\nEXP_NAME=\u003cyour model-A exp name\u003e \\\nSALIENCY=\u003csaliency name\u003e \\\nbash Rationale_Analysis/commands/saliency_script.sh\n```\nOutput generate in `outputs/\u003cclassifier type\u003e/SST/\u003cyour model-A exp name\u003e/\u003csaliency name\u003e_saliency` .\n\n3. Extract rationales from saliency\n\n```bash\nCUDA_DEVICE=0 \\\nDATASET_NAME=SST \\\nCLASSIFIER=\u003cclassifier type\u003e \\\nEXP_NAME=\u003cyour model-A exp name\u003e \\\nSALIENCY=\u003csaliency name\u003e \\\nRATIONALE=\u003crationale extraction type\u003e \\\nRATIONALE_EXP_NAME=\u003cyour rationale exp name\u003e \\\nbash Rationale_Analysis/commands/rationale_extractor_script.sh\n```\nOutput generated in `outputs/\u003cclassifier type\u003e/SST/\u003cyour model-A exp name\u003e/\u003csaliency name\u003e_saliency/\u003crationale extraction type\u003e_rationale/\u003cyour rationale exp name\u003e`.\n\n\n4. Extract Rationales and train model b.\n\n```bash\nCUDA_DEVICE=0 \\\nDATASET_NAME=SST \\\nCLASSIFIER=\u003cclassifier type\u003e \\\nEXP_NAME=\u003cyour model-A exp name\u003e \\\nSALIENCY=\u003csaliency name\u003e \\\nRATIONALE=\u003crationale extraction type\u003e \\\nRATIONALE_EXP_NAME=\u003cyour exp name\u003e \\\nbash Rationale_Analysis/commands/rationale_and_train_model_b_script.sh\n```\n\nAllowed values for now \n\n1. saliency name - file names in training_config/saliency_scorers\n2. classifier type - \"bert_classification\"\n3. rationale extraction type - filenames in training_config/rationale_extractors . Note each rationale extractor may also need some hyperparameter setting through env variables.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuvalpinter%2Frationale_analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyuvalpinter%2Frationale_analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuvalpinter%2Frationale_analysis/lists"}