{"id":20560073,"url":"https://github.com/cannylab/causal_overhypotheses","last_synced_at":"2025-04-14T14:04:55.363Z","repository":{"id":38047968,"uuid":"500997912","full_name":"CannyLab/causal_overhypotheses","owner":"CannyLab","description":"Code for Dataset and Benchmarks Submission, Neurips 2022","archived":false,"fork":false,"pushed_at":"2022-08-16T19:13:22.000Z","size":239,"stargazers_count":10,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2023-03-02T03:30:58.487Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CannyLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-07T20:42:23.000Z","updated_at":"2023-01-20T19:44:18.000Z","dependencies_parsed_at":"2022-09-17T03:31:43.273Z","dependency_job_id":null,"html_url":"https://github.com/CannyLab/causal_overhypotheses","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CannyLab%2Fcausal_overhypotheses","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CannyLab%2Fcausal_overhypotheses/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CannyLab%2Fcausal_overhypotheses/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CannyLab%2Fcausal_overhypotheses/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CannyLab","download_url":"https://codeload.github.com/CannyLab/causal_overhypotheses/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224873086,"owners_count":17384078,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T03:53:10.100Z","updated_at":"2024-11-16T03:53:10.777Z","avatar_url":"https://github.com/CannyLab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Towards Understanding How Machines Can Learn Causal Overhypotheses (Submission: NeurIPS Baselines 2022)\n\nThis repository hosts the code for the blicket-environment baselines presented in the paper **Towards Understanding How Machines Can Learn Causal Overhypotheses**.\n\n## Environment Details\n\nThe environment is a standard gym environment located at `envs/causal_env_v0.py`.\n\n## Running our Benchmarks\n\nThe following details are provided for running the experiments in the paper.\n\n### Installation and Requirements\n\nTo install the environment, we require python \u003c= 3.7, and tensorflow/tensorflow-gpu \u003c 2 (usually 1.15.5). This is due to a dependency on the old version of stable-baselines (the new version does not support LSTM models). Dependencies can be installed with:\n\n```\npip install tensorflow==1.15.5 stable-baselines gym==0.21.0 protobuf==3.20 tqdm\n```\n\n### Q-Learning\n\nTo run Q-Learning experiments, use the command `python models/q_learning.py` with the correct options:\n\n```\nusage: q_learning.py [-h] [--num NUM] [--alpha ALPHA] [--discount DISCOUNT] [--epsilon EPSILON]\n\nTrain a q-learner\n\noptional arguments:\n  -h, --help           show this help message and exit\n  --num NUM            Number of times to experiment\n  --alpha ALPHA        Learning rate\n  --discount DISCOUNT  Discount factor\n  --epsilon EPSILON    Eepsilon-greedy exploration rate\n```\n\n### Standard RL Models\n\nTo train standard RL models, use the command `python driver.py` with the correct options:\n\n```\nusage: driver.py [-h] [--alg ALG] [--policy POLICY] [--lstm_units LSTM_UNITS]\n                 [--num_steps NUM_STEPS]\n                 [--quiz_disabled_steps QUIZ_DISABLED_STEPS]\n                 [--holdout_strategy HOLDOUT_STRATEGY]\n                 [--reward_structure REWARD_STRUCTURE]\n\nTrain a model\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --alg ALG             Algorithm to use\n  --policy POLICY       Policy to use\n  --lstm_units LSTM_UNITS\n                        Number of LSTM units\n  --num_steps NUM_STEPS\n                        Number of training steps\n  --quiz_disabled_steps QUIZ_DISABLED_STEPS\n                        Number of quiz disabled steps (-1 for no forced\n                        exploration)\n  --holdout_strategy HOLDOUT_STRATEGY\n                        Holdout strategy\n  --reward_structure REWARD_STRUCTURE\n                        Reward structure\n```\n\nAlgorithm Choices: `[a2c, ppo2]`\n\nPolicy Choices: `[mlp, mlp_lstm, mlp_lnlstm]`\n\nHoldout Strategy Choices: `[none, disjunctive_train (Only disjunctive overhypotheses), conjunctive_train (Only conjunctive overhypotheses), disjunctive_loo (Only disjunctive, leave on out), conjunctive_loo (Only conjunctive, leave one out), both_loo (Leave one out for both)]`\n\nReward Structure Choices: `[baseline (Light up the blicket detector), quiz (Determine which are blickets), quiz-type (Determine which are blickets + Causal vs. Non-Causal), quiz-typeonly (Causal vs. Non-Causal only)]`\n\nThis will produce a model output file `{model_name}.zip` based on the options that are chosen during the training process. Evaluation data is printed during the training process, and training is terminated when 3M training steps are reached.\n\n### Decision Transformer / Behavior Cloning\n\nTraining the decision transformer consists of two (or three) steps. The first step is to generate trajectories for training. This can either be done with `scripts/collect_trajectories.py`, which has the following signature:\n\n```\nusage: collect_trajectories.py [-h] [--env ENV] [--model_path MODEL_PATH]\n                               [--num_trajectories NUM_TRAJECTORIES]\n                               [--max_steps MAX_STEPS]\n                               [--quiz_disabled_steps QUIZ_DISABLED_STEPS]\n                               [--output_path OUTPUT_PATH]\n\nCollect Trajectories from Causal Environments\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --env ENV             Environment to use\n  --model_path MODEL_PATH\n                        Path to model\n  --num_trajectories NUM_TRAJECTORIES\n                        Number of trajectories to collect\n  --max_steps MAX_STEPS\n                        Maximum number of steps per trajectory\n  --quiz_disabled_steps QUIZ_DISABLED_STEPS\n                        Number of steps to disable quiz\n  --output_path OUTPUT_PATH\n                        Path to output file\n```\n\nThe model path generated by the training step from the standard RL models can be passed to this script to generate samples from the pre-trained model. This will generate a file `trajectories.pkl`. This `pkl` file should be re-named to take the form `causal-\u003cname\u003e-v2.pkl`. This file should then be moved to the `models/decision-transformer/data` folder.\n\nTo train the decision transformer, follow the instructions in `models/decision-transformer` to install the conda environment, and then run `python experiment.py` with the options `--env causal` and a `--dataset` option corresponding to the chosen value for `\u003cname\u003e` above (for example, if the trajectories are named `causal-mlp-v2.pkl` the command would be `python experiment.py --env causal --dataset mlp --batch_size 128 --K 30 --model dt`). The batch size, K, and model (dt: decision transformer, bc: behavior cloning) can be adjusted as desired.\n\n## License Information\n\nThis code is licensed under the MIT license by the Regents of the University of California. The code we use for the Decision Transformer benchmark is also licensed under the MIT license by the authors of https://arxiv.org/abs/2106.01345.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcannylab%2Fcausal_overhypotheses","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcannylab%2Fcausal_overhypotheses","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcannylab%2Fcausal_overhypotheses/lists"}