{"id":21606127,"url":"https://github.com/bwconrad/cv-rl","last_synced_at":"2026-05-09T13:16:28.622Z","repository":{"id":184283985,"uuid":"671621061","full_name":"bwconrad/cv-rl","owner":"bwconrad","description":"Experiments training computer models using policy optimization","archived":false,"fork":false,"pushed_at":"2023-07-27T19:11:49.000Z","size":11,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-24T19:17:05.175Z","etag":null,"topics":["computer-vision","pytorch","pytorch-lightning","reinforcement-learning","segmentation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bwconrad.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-07-27T18:37:14.000Z","updated_at":"2024-05-06T23:55:53.000Z","dependencies_parsed_at":"2023-07-27T20:08:19.383Z","dependency_job_id":null,"html_url":"https://github.com/bwconrad/cv-rl","commit_stats":null,"previous_names":["bwconrad/cv-rl"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bwconrad%2Fcv-rl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bwconrad%2Fcv-rl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bwconrad%2Fcv-rl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bwconrad%2Fcv-rl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bwconrad","download_url":"https://codeload.github.com/bwconrad/cv-rl/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244243142,"owners_count":20422012,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","pytorch","pytorch-lightning","reinforcement-learning","segmentation"],"created_at":"2024-11-24T20:19:21.872Z","updated_at":"2026-05-09T13:16:23.596Z","avatar_url":"https://github.com/bwconrad.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Computer Vision Training using Policy Optimization\n\nThis repo contains experiments using policy optimization to train non-differentiable objectives for computer vision tasks. These experiments are inspired by the work \"[Tuning computer vision models with task rewards](https://arxiv.org/abs/2302.08242)\".\n\n## Requirements \n\n- Python 3.10+\n- `pip install -r requirements.txt`\n\n## Experiments\n\n### Binary Segmentation - F1 Score\n\nIn this experiment, a binary segmentation model is trained using the REINFORCE algorithm to optimize for the F1 score.\nThe experiment is performed on the [TNBC](https://zenodo.org/record/1175282#.YMisCTZKgow) using a U-Net with a ResNet-18 encoder.\n\nThe baseline cross-entropy model can be run with the following:\n```python\ntrain_segmentation.py fit --trainer.accelerator gpu --trainer.devices 1 --trainer.precision 16-mixed --data.root data/tnbc --data.batch_size 8 --trainer.max_steps 1000 --trainer.val_check_interval 100 --model.lr 0.0005 --model.schedule cosine \n```\n\nThe F1 score model can be run with the following:\n```python\ntrain_segmentation_reinforce.py fit --trainer.accelerator gpu --trainer.devices 1 --trainer.precision 16-mixed --data.root data/tnbc --data.batch_size 8 --trainer.max_steps 1000 --trainer.val_check_interval 100 --model.lr 0.0005 --model.schedule cosine \n```\n\nFine-tuning the cross-entropy model on the F1 score can be run with the following:\n```python\ntrain_segmentation_reinforce.py fit --trainer.accelerator gpu --trainer.devices 1 --trainer.precision 16-mixed --data.root data/tnbc --data.batch_size 8 --trainer.max_steps 1000 --trainer.val_check_interval 100 --model.lr 0.00005 --model.schedule cosine  --model.weights output/weights-ce.ckpt\n```\n\n#### Results\n\n| Objective | F1 | Dice |\n|:-:|:-:|:-:| \n| Cross-entropy | 0.7729 | 0.7585 |\n| F1 | 0.4152 | 0.4369 |\n| Cross-entropy -\u003e F1 | 0.7615 | 0.7611 |\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbwconrad%2Fcv-rl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbwconrad%2Fcv-rl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbwconrad%2Fcv-rl/lists"}