{"id":18487114,"url":"https://github.com/simran-arora/focus","last_synced_at":"2025-04-08T20:30:44.468Z","repository":{"id":128538855,"uuid":"496246002","full_name":"simran-arora/focus","owner":"simran-arora","description":"This repo contains code for the paper: \"Can Foundation Models Help Us Achieve Perfect Secrecy?\"","archived":false,"fork":false,"pushed_at":"2023-02-09T21:14:57.000Z","size":1499,"stargazers_count":24,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-23T17:48:45.635Z","etag":null,"topics":["machine-learning","privacy","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simran-arora.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-25T13:36:45.000Z","updated_at":"2023-10-11T06:23:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"5923313a-f836-4f8b-a6a7-7adb601a442a","html_url":"https://github.com/simran-arora/focus","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simran-arora%2Ffocus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simran-arora%2Ffocus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simran-arora%2Ffocus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simran-arora%2Ffocus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simran-arora","download_url":"https://codeload.github.com/simran-arora/focus/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247922657,"owners_count":21018839,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","privacy","pytorch"],"created_at":"2024-11-06T12:50:14.015Z","updated_at":"2025-04-08T20:30:44.462Z","avatar_url":"https://github.com/simran-arora.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Can Foundation Models Help Us Achieve Perfect Secrecy? A simple baseline for personal \u0026 private ML.\n\nThis repository explores the baseline of using in-context learning for personal \u0026 private machine learning! We include scripts to download data and evaluate foundation models of various sizes and types across popular personal machine learning benchmarks from the privacy literature. Additional information can be found in the paper: https://arxiv.org/abs/2205.13722. Contributions of additional benchmarks and baselines are welcome in the form of PRs to the benchmarks folder and additions of a data loader. Exciting future questions: \n\n- Do we see further personalization with more in-context examples and longer contexts? \n- Can we understand the limits of this baseline in a more principled manner? Are the tasks we've evaluated on so far too similar to the pretraining distribution? [Maybe we should construct new privacy benchmarks!](https://arxiv.org/abs/2212.06470)\n- Can we enable better in-context learning quality in smaller \u0026 open-source models? (Checkout recent work: [Ask Me Anything](https://arxiv.org/abs/2210.02441))\n- Other ways of using the FMs -- is it better to generate synthetic data with the FMs and then train locally?\n\n\u003cp align=\"center\"\u003e\u003cimg width=\"85%\" src=\"imgs/main_figure.png\" /\u003e\u003c/p\u003e\n\n## Setup\n\nUse the following commands to clone and install this package. We highly recommend you use conda environments.\n\n```\n# environment\nconda create -n py37 python=3.7\nconda activate py37\n\n# installations\ngit clone git@github.com:simran-arora/focus.git\ncd focus\npip install -e .\npip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113\n\ncd benchmarks/leaf\ngit submodule init\ngit submodule update\n```\n\nIf you want to run inference with the API models, first obtain your OpenAI inference API key here [OpenAI API](https://openai.com/api/). Then set the environment variables:\n```\nexport OPENAI_API_KEY=\"\u003cYOUR API KEY\u003e\"\n```\n\n## Obtain the datasets\n\nDownload benchmark datasets to the ``focus/benchmarks/'' directory\n```\ncd benchmarks/\nbash download_data.sh\n```\n\nThe LEAF Federated Learning benchmark suite provides: Sent140, Reddit, FEMNIST, and CELEB-A. The FedNLP suite provides 20News and MRQA. The FedML suite provides CIFAR-10.\n- Sent140, FEMNIST, CelebA, CIFAR-10, and 20News benchmarks are downloaded via the provided download script. \n- [Reddit] Go to benchmarks/leaf/data/reddit/ and follow the download instructions.\n- [MRQA] Go to https://github.com/FedML-AI/FedNLP/tree/27f3f97c72e7f206f8937fe6bcbba39ce79fbcd6/data/raw_data_loader/MRQA and run ``python download.py`` using their provided script.\n\n## Run the code\n\nThe ``focus/scripts/`` directory provides scripts to run experiments for each benchmark.\n\nFor example:\n```\nbash scripts/cifar.sh\nbash scripts/sent140.sh\n```\n\nWe include examples for running inference with the API in the script files. Note that this requires providing the ```openai_key``` command line argument.\n\n## Citation\nPlease use the following Bibtex for this work:\n```\n@misc{arora2022focus,\n      title={Can Foundation Models Help Us Achieve Perfect Secrecy?}, \n      author={Simran Arora and Christopher Ré},\n      year={2022},\n      url={https://arxiv.org/abs/2205.13722},\n      journal={The Fourth AAAI Workshop on Privacy-Preserving Artificial Intelligence (PPAI)},\n      primaryClass={cs.LG}\n}\n```\n\nWe welcome contributions and feel free to reach out at [simarora@stanford.edu](simarora@stanford.edu).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimran-arora%2Ffocus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimran-arora%2Ffocus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimran-arora%2Ffocus/lists"}