awesome-data-valuation

💱 A curated list of data valuation (DV) to design your next data marketplace
https://github.com/daviddao/awesome-data-valuation

Last synced: 3 days ago
JSON representation

What is your data worth?
- Shapley Value & Cooperative Game Theory
 - Collaborative Machine Learning with Incentive-Aware Model Rewards - aware model rewards}, author={Sim, Rachael Hwee Ling and Zhang, Yehong and Chan, Mun Choon and Low, Bryan Kian Hsiang}, booktitle={International Conference on Machine Learning}, pages={8927--8936}, year={2020}, organization={PMLR} }</pre></details> | | |
 - Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learning - reduced Data Valuation Framework for Machine Learning}, author={Kwon, Yongchan and Zou, James}, journal={arXiv preprint arXiv:2110.14049}, year={2021} }</pre></details> | | |
 - Gradient-Driven Rewards to Guarantee Fairness in Collaborative Machine Learning - time gradient reward mechanism with a fairness guarantee. </details> | <details><summary>Bibtex</summary><pre>@article{xu2021gradient, title={Gradient driven rewards to guarantee fairness in collaborative machine learning}, author={Xu, Xinyi and Lyu, Lingjuan and Ma, Xingjun and Miao, Chenglin and Foo, Chuan Sheng and Low, Bryan Kian Hsiang}, journal={Advances in Neural Information Processing Systems}, volume={34}, pages={16104--16117}, year={2021} }</pre></details> | | |
 - Improving Cooperative Game Theory-based Data Valuation via Data Utility Learning - based Shapley value (SV) or Least core (LC) estimation heuristics. They propose learning to predict the performance of a learning algorithm (denoted data utility learning) and using this predictor to estimate learning performance without retraining for cheaper SV and LC estimation. </details> | <details><summary>Bibtex</summary><pre>@article{wang2021improving, title={Improving cooperative game theory-based data valuation via data utility learning}, author={Wang, Tianhao and Yang, Yu and Jia, Ruoxi}, journal={arXiv preprint arXiv:2107.06336}, year={2021} }</pre></details> | | |
 - A Multilinear Sampling Algorithm to Estimate Shapley Values
 - Shapley values for feature selection: The good, the bad, and the axioms
 - Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability - agnostic explainability}, author={Frye, Christopher and Rowat, Colin and Feige, Ilya}, journal={Advances in Neural Information Processing Systems}, volume={33}, year={2020} }</pre></details> | | [🎥](https://www.youtube.com/watch?v=7d13f4UaAn0) |
 - If You Like Shapley Then You’ll Love the Core
 - Towards Efficient Data Valuation Based on the Shapley Value - -1176}, year={2019}, organization={PMLR} }</pre></details> | [💻](https://github.com/sunblaze-ucb/data-valuation) | |
 - A Distributional Framework for Data Valuation - based algorithm for the distributional Shapley value with strong approximation guarantees. </details> | <details><summary>Bibtex</summary><pre>@inproceedings{ghorbani2020distributional, title={A Distributional Framework for Data Valuation}, author={Ghorbani, Amirata, P. Kim, Michael and Zou, James}, booktitle={International Conference on Machine Learning}, year={2020} }</pre></details> | [💻](https://github.com/amiratag/DistributionalShapley) | |
 - Validation free and replication robust volume-based data valuation - based data valuation}, author={Xu, Xinyi and Wu, Zhaoxuan and Foo, Chuan Sheng and Low, Bryan Kian Hsiang}, journal={Advances in Neural Information Processing Systems}, volume={34}, year={2021} }</pre></details> | [💻](https://github.com/ZhaoxuanWu/VolumeBased-DataValuation) | |
 - Data Banzhaf: A Robust Data Valuation Framework for Machine Learning - v206-wang23e, title={Data Banzhaf: A Robust Data Valuation Framework for Machine Learning}, author={Wang, Jiachen T. and Jia, Ruoxi}, booktitle={Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages={6388--6421}, year={2023}, editor={Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume={206}, series={Proceedings of Machine Learning Research}, month={25--27 Apr}, publisher={PMLR}, pdf={https://proceedings.mlr.press/v206/wang23e/wang23e.pdf}, url={https://proceedings.mlr.press/v206/wang23e.html} }</pre></details> | [💻](https://github.com/Jiachen-T-Wang/data-banzhaf) | |
 - CS-Shapley: Class-wise Shapley Values for Data Valuation in Classification - class and out-of-class contributions. </details> | <details><summary>Bibtex</summary><pre>@inproceedings{schoch2022csshapley, title={{CS}-Shapley: Class-wise Shapley Values for Data Valuation in Classification}, author={Stephanie Schoch and Haifeng Xu and Yangfeng Ji}, booktitle={Advances in Neural Information Processing Systems}, year={2022} }</pre></details> | [💻](https://github.com/uvanlp/valda) | |
 - Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithms - linear time and approximations in sublinear time for k-nearest-neighbor models. They empirically evaluate their algorithms at scale and extend them to several other settings. </details> | <details><summary>Bibtex</summary><pre>@article{jia12efficient, title={Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithms}, author={Jia, Ruoxi and Dao, David and Wang, Boxin and Hubis, Frances Ann and Gurel, Nezihe Merve and Zhang, Bo Li4 Ce and Song, Costas Spanos1 Dawn}, journal={Proceedings of the VLDB Endowment}, volume={12}, number={11} }</pre></details> | [💻](https://github.com/easeml/datascope) | |
 - Efficient computation and analysis of distributional Shapley values - parametric density estimation as well as new efficient methods for its estimation. </details> | <details><summary>Bibtex</summary><pre>@inproceedings{kwon2021efficient, title={Efficient computation and analysis of distributional Shapley values}, author={Kwon, Yongchan and Rivas, Manuel A and Zou, James}, booktitle={International Conference on Artificial Intelligence and Statistics}, pages={793--801}, year={2021}, organization={PMLR} }</pre></details> | [💻](https://github.com/ykwon0407/fast_dist_shapley) | |
 - Data Shapley: Equitable Valuation of Data for Machine Learning - based methods for its efficient estimation. </details> | <details><summary>Bibtex</summary><pre>@inproceedings{ghorbani2019data, title={Data shapley: Equitable valuation of data for machine learning}, author={Ghorbani, Amirata and Zou, James}, booktitle={International Conference on Machine Learning}, pages={2242--2251}, year={2019}, organization={PMLR} }</pre></details> | [💻](https://github.com/amiratag/DataShapley) | |
 - Scalability vs. Utility: Do We Have to Sacrifice One for the Other in Data Importance Quantification? - one-out-based and Shapley value-based methods as well as an empirical study across several ML tasks investigating the two aforementioned methods as well as exact Shapley value-based methods and Shapley over KNN Surrogates. </details> | <details><summary>Bibtex</summary><pre>@misc{jia2021scalability, title={Scalability vs. Utility: Do We Have to Sacrifice One for the Other in Data Importance Quantification?}, author={Ruoxi Jia and Fan Wu and Xuehui Sun and Jiacen Xu and David Dao and Bhavya Kailkhura and Ce Zhang and Bo Li and Dawn Song}, year={2021}, eprint={1911.07128}, archivePrefix={arXiv}, primaryClass={cs.LG} }</pre></details> | [💻](https://github.com/AI-secure/Shapley-Study) | |
- Influence functions & LOO
 - Studying large language model generalization with influence functions - FAC to approximate the Hessian of the loss of large language models. They apply this technique to study influence functions on large language models, up to 50 billion parameters. </details> | <details><summary>Bibtex</summary><pre>@article{grosse2023studying, title={Studying large language model generalization with influence functions}, author={Grosse, Roger and Bae, Juhan and Anil, Cem and Elhage, Nelson and Tamkin, Alex and Tajdini, Amirhossein and Steiner, Benoit and Li, Dustin and Durmus, Esin and Perez, Ethan and others}, journal={arXiv preprint arXiv:2308.03296}, year={2023} }</pre></details> | | |
 - Understanding Black-box Predictions via Influence Functions - box predictions via influence functions}, author={Koh, Pang Wei and Liang, Percy}, booktitle={International Conference on Machine Learning}, pages={1885--1894}, year={2017}, organization={PMLR} }</pre></details> | [💻](https://bit.ly/gt-influence) | [🎥](https://drive.google.com/open?id=1ZLY_9Wsk9MA0kXAoJDd6o1gbLvHhyPAn) |
 - On the accuracy of influence functions for measuring group effects - Siang Ang*, Hubert H. K. Teo*, and Percy Liang | 2019 | <details><summary>Summary</summary> Koh et al. (2019) study influence functions to measure effects of large groups of training points instead of individual points. They empirically find a correlation and often underestimation between predicted and actual effects and theoretically show that this need not hold in general, realistic settings. </details> | <details><summary>Bibtex</summary><pre>@article{koh2019accuracy, title={On the accuracy of influence functions for measuring group effects}, author={Koh, Pang Wei and Ang, Kai-Siang and Teo, Hubert HK and Liang, Percy}, journal={arXiv preprint arXiv:1905.13289}, year={2019} }</pre></details> | [💻](https://github.com/kohpangwei/group-influence-release) | [🎥](https://drive.google.com/open?id=1ZLY_9Wsk9MA0kXAoJDd6o1gbLvHhyPAn) |
 - Scaling Up Influence Functions - size Transformer models with hundreds of millions of parameters. </details> | <details><summary>Bibtex</summary><pre>@inproceedings{schioppa2022scaling, title={Scaling Up Influence Functions}, author={Schioppa, Andrea and Zablotskaia, Polina and Vilar, David and Sokolov, Artem}, booktitle={Proceedings of the AAAI Conference on Artificial Intelligence}, year={2022} }</pre></details> | [💻](https://github.com/google-research/jax-influence) | |
- Task Agnostic
 - Fundamentals of Task-Agnostic Data Valuation - agnostic data valuation. It discusses valuing a data seller's dataset from a buyer's perspective without validation requirements. The approach involves estimating statistical differences through diversity and relevance measures without needing the raw data, and designing queries that maintain the seller's blindness to the buyer's raw data. The work is significant for practical scenarios where utility metrics like test accuracy on a validation set are not feasible.</details> | <details><summary>Bibtex</summary><pre>@article{Amiri2023FundamentalsOT, title={Fundamentals of Task-Agnostic Data Valuation}, author={Mohammad Mohammadi Amiri and Frederic Berdoz and Ramesh Raskar}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, volume={37}, pages={9226-9234}, year={2023}, doi={10.1609/aaai.v37i8.26106} }</pre></details> | | |
- Reinforcement Learning
 - Data Valuation using Reinforcement Learning - research/google-research/tree/master/dvrl) | [🎥](https://icml.cc/media/icml-2020/Slides/6276.pdf) |
- Out-of-Bag score
 - Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Value - of-bag estimate of a bagging estimator for computationally efficient data valuation. </details> | <details><summary>Bibtex</summary><pre>@inproceedings{DBLP:conf/icml/Kwon023, author={Yongchan Kwon and James Zou}, editor={Andreas Krause and Emma Brunskill and Kyunghyun Cho and Barbara Engelhardt and Sivan Sabato and Jonathan Scarlett}, title={Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Value}, booktitle={International Conference on Machine Learning, {ICML} 2023, 23-29 July 2023, Honolulu, Hawaii, {USA}}, series={Proceedings of Machine Learning Research}, volume={202}, pages={18135--18152}, publisher={{PMLR}}, year={2023}, url={https://proceedings.mlr.press/v202/kwon23e.html}, timestamp={Mon, 28 Aug 2023 17:23:08 +0200}, biburl={https://dblp.org/rec/conf/icml/Kwon023.bib}, bibsource={dblp computer science bibliography, https://dblp.org} }</pre></details> | [💻](https://github.com/ykwon0407/STAT5206_Fall_2023) | [🎥](https://icml.cc/virtual/2023/poster/23689) |
- Deep Neural Networks
 - DAVINZ: Data Valuation using Deep Neural Networks at Initialization - based and training-free method for efficient data valuation with large and complex deep neural networks (DNNs). They derive and exploit a domain-aware generalization bound for DNNs to characterize their performance without training and uses this bound as the scoring function while keeping conventional techniques such as Shapley values as the valuation function. </details> | <details><summary>Bibtex</summary><pre>@inproceedings{wu2022davinz, title={DAVINZ: Data Valuation using Deep Neural Networks at Initialization}, author={Wu, Zhaoxuan and Shu, Yao and Low, Bryan Kian Hsiang}, booktitle={International Conference on Machine Learning}, pages={24150--24176}, year={2022}, organization={PMLR} }</pre></details> | | [🎥](https://icml.cc/media/icml-2022/Slides/17500.pdf) |
Designing data marketplaces
- Data market system designs
 - A demonstration of sterling: a privacy-preserving data marketplace - Preserving Data Marketplace}, author={Hynes, Nick and Dao, David and Yan, David and Cheng, Raymond and Song, Dawn}, journal={Proceedings of the VLDB Endowment}, volume={11}, number={12}, year={2018} }</pre></details> | | |
 - DataBright: Towards a Global Exchange for Decentralized Data Ownership and Trusted Computation
 - A Marketplace for Data: An Algorithmic Solution - -726}, year={2019} }</pre></details> | | |
 - Computing a Data Dividend
 - Incentivizing Collaboration in Machine Learning via Synthetic Data Rewards
- Automatic data compliance
 - Data Capsule: A New Paradigm for Automatic Compliance with Data Privacy Regulations - ucb/Data-Capsule) | |
- Data valuation applications
 - A Principled Approach to Data Valuation for Federated Learning
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Improving Fairness for Data Valuation in Horizontal Federated Learning
 - Data Valuation for Vertical Federated Learning: An Information-Theoretic Approach - preserving cross-party data collaboration. This work introduces "FedValue," the first privacy-preserving, task-specific, model-free data valuation method for vertical FL tasks. It incorporates Shapley-CMI, an information-theoretic metric, for assessing data values from a game-theoretic perspective. The paper also proposes a novel server-aided federated computation mechanism and techniques to accelerate Shapley-CMI computation. Extensive experiments demonstrate the effectiveness and efficiency of FedValue.</details> | <details><summary>Bibtex</summary><pre>@misc{han2021datavaluation, title={Data Valuation for Vertical Federated Learning: An Information-Theoretic Approach}, author={Xiao Han and Leye Wang and Junjie Wu}, year={2021}, eprint={URL or DOI link TBD}, }</pre></details> | | |
 - Towards More Efficient Data Valuation in Healthcare Federated Learning Using Ensembling - Cramer, Pavitra Krishnaswamy, Praveer Singh | 2021 | <details><summary>Summary</summary>This paper addresses the challenge of data valuation in federated learning within healthcare. The authors propose a method called SaFE (Shapley Value for Federated Learning using Ensembling), which is designed to be efficient in settings where the number of contributing institutions is manageable. SaFE approximates the Shapley value using gradients from training a single model and develops methods for efficient contribution index estimation. This approach is particularly relevant in medical imaging where data heterogeneity is common and fast, accurate data valuation is necessary for multi-institutional collaborations.</details> | <details><summary>Bibtex</summary><pre>@article{Kumar2021TowardsME, title={Towards More Efficient Data Valuation in Healthcare Federated Learning Using Ensembling}, author={Sourav Kumar and A. Lakshminarayanan and Ken Chang and Feri Guretno and Ivan Ho Mien and Jayashree Kalpathy-Cramer and Pavitra Krishnaswamy and Praveer Singh}, journal={ArXiv}, year={2021}, volume={abs/2209.05424} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
 - Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset - scale chest X-ray dataset}, author={Tang, Siyi and Ghorbani, Amirata and Yamashita, Rikiya and Rehman, Sameer and Dunnmon, Jared A and Zou, James and Rubin, Daniel L}, journal={Scientific reports}, volume={11}, number={1}, pages={1--9}, year={2021}, publisher={Nature Publishing Group} }</pre></details> | | |
 - Efficient and Fair Data Valuation for Horizontal Federated Learning - -152}, publisher={Springer} }</pre></details> | | |
Data markets and society
- Economics of Data
 - Nonrivalry and the Economics of Data - 58}, DOI = {10.1257/aer.20191330}, URL = {https://www.aeaweb.org/articles?id=10.1257/aer.20191330} }</pre></details> | | |
- Data Dignity
 - Chapter 5: Data as Labor, Radical Markets
 - Should We Treat Data as Labor? Moving beyond "Free" - Ibarra, Leonard Goff, Diego Jiménez-Hernández, Jaron Lanier, E. Glen Weyl | 2018 | | <details><summary>Bibtex</summary><pre>@article{10.1257/pandp.20181003, Author = {Arrieta-Ibarra, Imanol and Goff, Leonard and Jiménez-Hernández, Diego and Lanier, Jaron and Weyl, E. Glen}, Title = {Should We Treat Data as Labor? Moving beyond "Free"}, Journal = {AEA Papers and Proceedings}, Volume = {108}, Year = {2018}, Month = {May}, Pages = {38-42}, DOI = {10.1257/pandp.20181003}, URL = {https://www.aeaweb.org/articles?id=10.1257/pandp.20181003} }</pre></details> | | |
Strategic adaptation
- Strategic classification
 - Strategic Classification is Causal Modeling in Disguise - trivial causal inference problem. The authors provide a distinction between gaming and improvement as well as provide a causal framework for strategic adaptation. </details> | <details><summary>Bibtex</summary><pre>@inproceedings{miller2020strategic, title={Strategic classification is causal modeling in disguise}, author={Miller, John and Milli, Smitha and Hardt, Moritz}, booktitle={International Conference on Machine Learning}, pages={6917--6926}, year={2020}, organization={PMLR} }</pre> </details> | | |
 - Alternative Microfoundations for Strategic Classification - Dünner, Moritz Hardt | 2021 | <details> <summary>Summary</summary> Jagadeesan et al. (2021) show that standard microfoundations in strategic classification, that typically uses individual-level behaviour to deduce aggregate-level responses, can lead to degenerate behaviour in aggregate: discontinuities in the aggregate response, stable points ceasing to exist, and maximizing social burden. The authors introduce a noisy response model inspired by performative prediction that mitigates these limitations for binary classification. </details> | <details><summary>Bibtex</summary><pre>@inproceedings{jagadeesan2021alternative, title={Alternative microfoundations for strategic classification}, author={Jagadeesan, Meena and Mendler-D{\"u}nner, Celestine and Hardt, Moritz}, booktitle={International Conference on Machine Learning}, pages={4687--4697}, year={2021}, organization={PMLR} }</pre></details> | | |
- Performative prediction
 - Performative Prediction - Dünner, Moritz Hardt | 2020 | <details><summary>Summary</summary> Perdomo et al. (2020) introduce the concept of "performative prediction" dealing with predictions that influence the target they aim to predict, e.g. through taking actions based on the predictions, causing a distribution shift. The authors develop a risk minimization framework for performative prediction and introduce the equilibrium notion of performative stability where predictions are calibrated against future outcomes that manifest from acting on the prediction. </details> | <details><summary>Bibtex</summary><pre>@inproceedings{perdomo2020performative, title={Performative prediction}, author={Perdomo, Juan and Zrnic, Tijana and Mendler-D{\"u}nner, Celestine and Hardt, Moritz}, booktitle={International Conference on Machine Learning}, pages={7599--7609}, year={2020}, organization={PMLR} }</pre> </details> | | |
 - Stochastic Optimization for Performative Prediction - Dünner, Juan Perdomo, Tijana Zrnic, Moritz Hardt | 2020 | <details><summary>Summary</summary> Mendler-Dünner et al. (2020) look at stochastic optimization for performative prediction and prove convergence rates for greedily deploying models after each stochastic update (which may cause distribution shift affecting convergence to a stability point) or lazily deploying the model after several updates. </details> | <details><summary>Bibtex</summary><pre>@article{mendler2020stochastic, title={Stochastic optimization for performative prediction}, author={Mendler-D{\"u}nner, Celestine and Perdomo, Juan and Zrnic, Tijana and Hardt, Moritz}, journal={Advances in Neural Information Processing Systems}, volume={33}, pages={4929--4939}, year={2020} }</pre> </details> | | |
Data Valuation Researchers
- Strategic classification
  - Costas Spanos
  - Jinsung Yoon
  - Tomas Pfister
  - Amirata Ghorbani
  - James Zou
  - Nektaria Tryfona
  - Rachael Hwee Ling Sim
  - Bryan Kian Hsiang Low
  - Dawn Song
  - Zhaoxuan Wu
  - Xinyi Xu
  - Tianhao Wang
  - Ruben Cuevas Rumin
  - Jiachen T. Wang
  - Bohong Wang
  - Yongchan Kwon
  - Siyi Tang
  - Li Xiong
  - Jessica Vitak
  - Katie Chamberlain Kritikos - Champaign | 6 |
  - Zhenan Fan
  - Shuyue Wei
  - Hannah Stein
  - Wolfgang Maass
  - Mohammad Mohammadi Amiri
  - Ramesh Raskar
  - Konstantin D. Pandl
  - Ali Sunyaev
  - Ludovico Boratto
  - han xiao
  - Junjie Wu
  - Xiao Tian
  - Kean Birch
  - Callum Ward
  - Praveer Singh
  - Anran Xu
  - Guihai Chen
  - Andre Esteva - Founder & CEO, Artera | 23 |
  - Prateek Mittal
  - Hyeontaek Oh
  - Lingjiao Chen
  - Xiangyu Chang
  - Hoang Anh Just
  - David Dao
  - Mark Mazumder
  - Vijay Janapa Reddi
  - Sabri Eyuboglu
  - Wenqian Li
  - José González Cabañas - Santander Big Data Institute | 7 |
Libraries
- Task Agnostic
 - pyDVL - institute/pyDVL) | |
 - influenciae - AI | 2023 | <details><summary>Summary</summary> A stable implementation of influence functions in tensorflow. </details> | <details><summary>Bibtex</summary><pre></pre></details> | [💻](https://github.com/deel-ai/influenciae) | |
- Surveys
 - Data Valuation in Machine Learning: “Ingredients”, Strategies, and Open Challenges - xu.com/ijcai_slides.pdf) |
Benchmarks
- Task Agnostic
 - OpenDataVal: a Unified Benchmark for Data Valuations - seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track}, year={2023}, url={https://openreview.net/forum?id=eEK99egXeB} }</pre></details> | [💻](https://github.com/opendataval/opendataval) | [🎥](https://neurips.cc/virtual/2023/poster/73521) |

Programming Languages

Python 2

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

awesome-data-valuation

What is your data worth?

Shapley Value & Cooperative Game Theory

Influence functions & LOO

Task Agnostic

Reinforcement Learning

Out-of-Bag score

Deep Neural Networks

Designing data marketplaces

Data market system designs

Automatic data compliance

Data valuation applications

Data markets and society

Economics of Data

Data Dignity

Strategic adaptation

Strategic classification

Performative prediction

Data Valuation Researchers

Strategic classification

Libraries

Task Agnostic

Surveys

Benchmarks

Task Agnostic