{"id":13477018,"url":"https://github.com/twitter-research/image-crop-analysis","last_synced_at":"2025-03-27T04:32:02.644Z","repository":{"id":66334814,"uuid":"368619066","full_name":"twitter-research/image-crop-analysis","owner":"twitter-research","description":"Code for reproducing our analysis in the paper titled: Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency","archived":false,"fork":false,"pushed_at":"2021-10-25T16:51:26.000Z","size":3404,"stargazers_count":252,"open_issues_count":1,"forks_count":38,"subscribers_count":10,"default_branch":"main","last_synced_at":"2024-12-06T21:50:36.327Z","etag":null,"topics":["bias","computer-vision","fairness","fairness-ml","image-processing","machine-learning","research"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2105.08667","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/twitter-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.txt"}},"created_at":"2021-05-18T17:45:24.000Z","updated_at":"2024-11-27T13:57:43.000Z","dependencies_parsed_at":"2024-01-13T18:31:43.351Z","dependency_job_id":"f174c902-63a2-4d08-9c6c-204638500c9b","html_url":"https://github.com/twitter-research/image-crop-analysis","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter-research%2Fimage-crop-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter-research%2Fimage-crop-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter-research%2Fimage-crop-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter-research%2Fimage-crop-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/twitter-research","download_url":"https://codeload.github.com/twitter-research/image-crop-analysis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245785380,"owners_count":20671621,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bias","computer-vision","fairness","fairness-ml","image-processing","machine-learning","research"],"created_at":"2024-07-31T16:01:37.275Z","updated_at":"2025-03-27T04:32:02.635Z","avatar_url":"https://github.com/twitter-research.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"# Image Crop Analysis\n\n[![Open All Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/twitter-research/image-crop-analysis) [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/twitter-research/image-crop-analysis/HEAD) [![DOI:10.1145/3479594](http://img.shields.io/badge/DOI-10.1145/3479594-blue.svg)](https://doi.org/10.1145/3479594) [![arxiv:2105.08667](http://img.shields.io/badge/arxiv-2105.08667-red.svg)](https://arxiv.org/abs/2105.08667)\n\n![How does a saliency algorithm work](https://cdn.cms-twdigitalassets.com/content/dam/blog-twitter/engineering/en_us/insights/2021/imagecropping/newimagecropanimations.gif)\n\n\n\nThis is a repo for the code used for reproducing our [Image Crop Analysis paper](https://arxiv.org/abs/2105.08667) as shared on [our blog post](https://blog.twitter.com/engineering/en_us/topics/insights/2021/sharing-learnings-about-our-image-cropping-algorithm.html). \n\nIf you plan to use this code please cite our paper as follows:\n\n```\n@article{TwitterImageCrop2021,\n  author = {Yee, Kyra and Tantipongpipat, Uthaipon and Mishra, Shubhanshu},\n  title = {Image Cropping on Twitter: Fairness Metrics, Their Limitations, and the Importance of Representation, Design, and Agency},\n  year = {2021},\n  issue_date = {October 2021},\n  publisher = {Association for Computing Machinery},\n  address = {New York, NY, USA},\n  volume = {5},\n  number = {CSCW2},\n  url = {https://doi.org/10.1145/3479594},\n  doi = {10.1145/3479594},\n  journal = {Proceedings of the ACM on Human-Computer Interaction},\n  month = oct,\n  articleno = {450},\n  numpages = {24},\n  keywords = {image cropping, ethical HCI, fairness in machine learning, demographic parity, representational harm}\n}\n@article{TwitterImageCrop2021ArXiv,\n       author = {{Yee}, Kyra and {Tantipongpipat}, Uthaipon and {Mishra}, Shubhanshu},\n        title = \"{Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency}\",\n      journal = {arXiv e-prints},\n     keywords = {Computer Science - Computers and Society, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Human-Computer Interaction, Computer Science - Machine Learning},\n         year = 2021,\n        month = may,\n          eid = {arXiv:2105.08667},\n        pages = {arXiv:2105.08667},\narchivePrefix = {arXiv},\n       eprint = {2105.08667},\n primaryClass = {cs.CY},\n}\n```\n\n![Analysis of demographic bias of the image cropping algorithm](./notebooks/wiki_no_scaling_intersect_n=10000.jpg)\n\n\n# Instructions\n\n- Install [Miniconda](https://docs.conda.io/en/latest/miniconda.html) or [Anaconda](https://www.anaconda.com/products/individual) and then follow these steps:\n  * create a conda environment using `conda env create -f environment.yml`\n  * activate the environment using `conda activate image-crop-analysis`\n- Put a dummy jpeg image at `data/dummy.jpeg`\n- Put any additional images with `*.jpeg` extension in DATA_DIR, which is `./data`\n- If you just want to investigate how the model predicts the saliency map then you can use the notebook [notebooks/Image Annotation Dash.ipynb](notebooks/Image%20Annotation%20Dash.ipynb)\n- To reproduce the analysis first prepare the data using [notebooks/Data Preparation.ipynb](notebooks/Data%20Preparation.ipynb) and then run [notebooks/Demographic Bias Analysis.ipynb](notebooks/Demographic%20Bias%20Analysis.ipynb)\n- To reproduce the plots first run [notebooks/Demographic Bias Plots.ipynb](notebooks/Demographic%20Bias%20Plots.ipynb)\n- If you want to explore how the library behind the dashboard works see [notebooks/Image Crop Analysis.ipynb](notebooks/Image%20Crop%20Analysis.ipynb)\n- If you have the dataset prepared from the above steps then you can create the gender gaze dataset by running [notebooks/Gender Gaze Analysis.ipynb](notebooks/Gender%20Gaze%20Analysis.ipynb)\n\n\n## Docker Run\n\n* Install docker \n* Run the following commands in this root directory of this project:\n\n```bash\ndocker build -t \"image_crop\" -f docker/Dockerfile .\ndocker run -p 9000:9000 -p 8900:8900 -it image_crop\n```\n* Open the jupyter lab URL shown in terminal. \n\n## Run on Google Colab\n\n[![Open All Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/twitter-research/image-crop-analysis)\n\n\n* Open a google colab notebook\n* Run the following code in the cell where `HOME_DIR` variable is set:\n\n```\ntry:\n    import google.colab\n    ! pip install pandas scikit-learn scikit-image statsmodels requests dash\n    ! [[ -d image-crop-analysis ]] || git clone https://github.com/twitter-research/image-crop-analysis.git\n    HOME_DIR = Path(\"./image_crop_analysis\").expanduser()\n    IN_COLAB = True\nexcept:\n    IN_COLAB = False\n```\n* Try the [notebooks/Image Crop Analysis.ipynb](https://github.com/twitter-research/image-crop-analysis/blob/main/notebooks/Image%20Crop%20Analysis.ipynb) notebook for example. \n\n\n# Security Issues?\n\nPlease report sensitive security issues via Twitter's bug-bounty program (https://hackerone.com/twitter) rather than GitHub.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwitter-research%2Fimage-crop-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftwitter-research%2Fimage-crop-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwitter-research%2Fimage-crop-analysis/lists"}