{"id":19079688,"url":"https://github.com/devzhk/lmcts","last_synced_at":"2025-04-30T05:43:49.279Z","repository":{"id":47700644,"uuid":"419932718","full_name":"devzhk/LMCTS","owner":"devzhk","description":null,"archived":false,"fork":false,"pushed_at":"2022-08-03T16:38:57.000Z","size":2501,"stargazers_count":12,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-30T05:43:43.298Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devzhk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-22T01:49:36.000Z","updated_at":"2025-04-29T03:54:50.000Z","dependencies_parsed_at":"2022-09-23T17:11:56.078Z","dependency_job_id":null,"html_url":"https://github.com/devzhk/LMCTS","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devzhk%2FLMCTS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devzhk%2FLMCTS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devzhk%2FLMCTS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devzhk%2FLMCTS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devzhk","download_url":"https://codeload.github.com/devzhk/LMCTS/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251651221,"owners_count":21621702,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T02:15:37.637Z","updated_at":"2025-04-30T05:43:49.257Z","avatar_url":"https://github.com/devzhk.png","language":"Python","readme":"# Langevin Monte Carlo for Contextual Bandits (ICML2022)\n\nThis repository contains our pytorch implementation of Langevin Monte Carlo Thompson Sampling (LMC-TS), proposed in the paper [Langevin Monte Carlo for Contextual Bandits]()\n\n_Abstract_:  Existing Thompson sampling-based algorithms need to construct a Laplace approximation of the posterior distribution, which is inefficient to sample in high dimensional applications for general covariance matrices. \nMoreover, the Gaussian approximation may not be a good surrogate for the posterior distribution for general reward generating functions. \nWe propose an efficient posterior sampling algorithm, viz., Langevin Monte Carlo Thompson Sampling (LMC-TS), that uses Markov Chain Monte Carlo (MCMC) methods to directly sample from the posterior distribution in contextual bandits. \nOur method is computationally efficient since it only needs to perform noisy gradient descent updates without constructing the Laplace approximation of the posterior distribution. \n\n## Requirements\nTo install the necessary packages, run \n```bash\npip install -r requirements.txt\n```\n\n### Synthetic data\n#### Prepare Data\n1. Find the data here to reproduce our results. https://hkzdata.s3.us-west-2.amazonaws.com/LMC-TS/gaussian50-20-1-1.pt\n2. To generate synthetic data, run \n```bash\npython3 data_generator.py --config configs/data-linear.yaml\n```\n\n\nRun bandit algorithms on simulated bandit problems\n```bash\npython3 run_simulation.py --config_path configs/simulation/linear-LMCTS.yaml --repeat [number of experiments to repeat] \n```\n\nYou can add `--log` to turn on the wandb. \nConfiguration file examples:\n- Linear bandit: `configs/simulation/linear-LMCTS.yaml`\n- Quadratic bandit: `configs/simulation/quad-LMCTS.yaml`\n- Logistic bandit: `configs/simulation/logistic-LMCTS.yaml`\n\n### UCI datasets\nTo run bandit algorithm on UCI datasets, use\n```bash\npython3 run_classifier.py --config_path configs/uci/shuttle-lmcts.yaml --repeat [number of experiments to repeat]\n```\nYou can add `--log` to turn on the wandb. Configuration files are provided under folder `configs/uci`. \n\n### CIFAR10\nTo run bandit algorithm on CIFAR10 datasets, use\n```bash\npython3 run_cifar.py --config_path configs/image/cifar10-lmcts.yaml\n```\nConfiguration files are provided under `configs/uci`.\n\n\n## Code Structure\n`run_cifar.py`, `run_classifier.py`, `run_simulation.py` are the main entries of the program, which contain the abstract code of bandit framework. \nThese main entry files read configuration `.yaml` files from `configs` and parse them to run. \n\n`algo`,`train_utils`,`models` realize different modules in the framework.\nMore specifically, \n1. `algo` implements different online bandit algorithms. \n2. `train_utils` implements datasets, bandit instance, loss functions, and some helper functions.\n3. `models` implements different model architectures. \n\n## Citation\n```latex\n@inproceedings{xu2022langevin,\n  title={Langevin Monte Carlo for Contextual Bandits},\n  author={Xu, Pan and Zheng, Hongkai and Mazumdar, Eric V and Azizzadenesheli, Kamyar and Anandkumar, Animashree},\n  booktitle={International Conference on Machine Learning},\n  pages={24830--24850},\n  year={2022},\n  organization={PMLR}\n}\n```","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevzhk%2Flmcts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevzhk%2Flmcts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevzhk%2Flmcts/lists"}