{"id":13443436,"url":"https://github.com/amazon-science/patchcore-inspection","last_synced_at":"2025-05-16T15:04:56.798Z","repository":{"id":37088136,"uuid":"489023709","full_name":"amazon-science/patchcore-inspection","owner":"amazon-science","description":null,"archived":false,"fork":false,"pushed_at":"2024-07-10T01:35:09.000Z","size":2064,"stargazers_count":851,"open_issues_count":76,"forks_count":167,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-04-03T11:12:07.117Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/amazon-science.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-05T15:17:32.000Z","updated_at":"2025-04-01T22:20:04.000Z","dependencies_parsed_at":"2024-01-18T14:43:32.636Z","dependency_job_id":"b3d8d9c1-dc78-4c2b-9911-23c60e634f87","html_url":"https://github.com/amazon-science/patchcore-inspection","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fpatchcore-inspection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fpatchcore-inspection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fpatchcore-inspection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fpatchcore-inspection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/amazon-science","download_url":"https://codeload.github.com/amazon-science/patchcore-inspection/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248565001,"owners_count":21125413,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:02:00.694Z","updated_at":"2025-04-12T11:51:22.756Z","avatar_url":"https://github.com/amazon-science.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Towards Total Recall in Industrial Anomaly Detection\n\nThis repository contains the implementation for `PatchCore` as proposed in Roth et al. (2021), \u003chttps://arxiv.org/abs/2106.08265\u003e.\n\nIt also provides various pretrained models that can achieve up to _99.6%_ image-level anomaly\ndetection AUROC, _98.4%_ pixel-level anomaly localization AUROC and _\u003e95%_ PRO score (although the\nlater metric is not included for license reasons).\n\n![defect_segmentation](images/patchcore_defect_segmentation.png)\n\n_For questions \u0026 feedback, please reach out to karsten.rh1@gmail.com!_\n\n---\n\n## Quick Guide\n\nFirst, clone this repository and set the `PYTHONPATH` environment variable with `env PYTHONPATH=src python bin/run_patchcore.py`.\nTo train PatchCore on MVTec AD (as described below), run\n\n```\ndatapath=/path_to_mvtec_folder/mvtec datasets=('bottle' 'cable' 'capsule' 'carpet' 'grid' 'hazelnut'\n'leather' 'metal_nut' 'pill' 'screw' 'tile' 'toothbrush' 'transistor' 'wood' 'zipper')\ndataset_flags=($(for dataset in \"${datasets[@]}\"; do echo '-d '$dataset; done))\n\n\npython bin/run_patchcore.py --gpu 0 --seed 0 --save_patchcore_model \\\n--log_group IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0 --log_online --log_project MVTecAD_Results results \\\npatch_core -b wideresnet50 -le layer2 -le layer3 --faiss_on_gpu \\\n--pretrain_embed_dimension 1024  --target_embed_dimension 1024 --anomaly_scorer_num_nn 1 --patchsize 3 \\\nsampler -p 0.1 approx_greedy_coreset dataset --resize 256 --imagesize 224 \"${dataset_flags[@]}\" mvtec $datapath\n```\n\nwhich runs PatchCore on MVTec images of sizes 224x224 using a WideResNet50-backbone pretrained on\nImageNet. For other sample runs with different backbones, larger images or ensembles, see\n`sample_training.sh`.\n\nGiven a pretrained PatchCore model (or models for all MVTec AD subdatasets), these can be evaluated using\n\n```shell\ndatapath=/path_to_mvtec_folder/mvtec\nloadpath=/path_to_pretrained_patchcores_models\nmodelfolder=IM224_WR50_L2-3_P001_D1024-1024_PS-3_AN-1_S0\nsavefolder=evaluated_results'/'$modelfolder\n\ndatasets=('bottle'  'cable'  'capsule'  'carpet'  'grid'  'hazelnut' 'leather'  'metal_nut'  'pill' 'screw' 'tile' 'toothbrush' 'transistor' 'wood' 'zipper')\ndataset_flags=($(for dataset in \"${datasets[@]}\"; do echo '-d '$dataset; done))\nmodel_flags=($(for dataset in \"${datasets[@]}\"; do echo '-p '$loadpath'/'$modelfolder'/models/mvtec_'$dataset; done))\n\npython bin/load_and_evaluate_patchcore.py --gpu 0 --seed 0 $savefolder \\\npatch_core_loader \"${model_flags[@]}\" --faiss_on_gpu \\\ndataset --resize 366 --imagesize 320 \"${dataset_flags[@]}\" mvtec $datapath\n```\n\nA set of pretrained PatchCores are hosted here: __add link__. To use them (and replicate training),\ncheck out `sample_evaluation.sh` and `sample_training.sh`.\n\n---\n\n## In-Depth Description\n\n### Requirements\n\nOur results were computed using Python 3.8, with packages and respective version noted in\n`requirements.txt`. In general, the majority of experiments should not exceed 11GB of GPU memory;\nhowever using significantly large input images will incur higher memory cost.\n\n### Setting up MVTec AD\n\nTo set up the main MVTec AD benchmark, download it from here: \u003chttps://www.mvtec.com/company/research/datasets/mvtec-ad\u003e.\nPlace it in some location `datapath`. Make sure that it follows the following data tree:\n\n```shell\nmvtec\n|-- bottle\n|-----|----- ground_truth\n|-----|----- test\n|-----|--------|------ good\n|-----|--------|------ broken_large\n|-----|--------|------ ...\n|-----|----- train\n|-----|--------|------ good\n|-- cable\n|-- ...\n```\n\ncontaining in total 15 subdatasets: `bottle`, `cable`, `capsule`, `carpet`, `grid`, `hazelnut`,\n`leather`, `metal_nut`, `pill`, `screw`, `tile`, `toothbrush`, `transistor`, `wood`, `zipper`.\n\n### \"Training\" PatchCore\n\nPatchCore extracts a (coreset-subsampled) memory of pretrained, locally aggregated training patch features:\n\n![patchcore_architecture](images/architecture.png)\n\nTo do so, we have provided `bin/run_patchcore.py`, which uses `click` to manage and aggregate input\narguments. This looks something like\n\n```shell\npython bin/run_patchcore.py \\\n--gpu \u003cgpu_id\u003e --seed \u003cseed\u003e # Set GPU-id \u0026 reproducibility seed.\n--save_patchcore_model # If set, saves the patchcore model(s).\n--log_online # If set, logs results to a Weights \u0026 Biases account.\n--log_group IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0 --log_project MVTecAD_Results results # Logging details: Name of the run \u0026 Name of the overall project folder.\n\npatch_core  # We now pass all PatchCore-related parameters.\n-b wideresnet50  # Which backbone to use.\n-le layer2 -le layer3 # Which layers to extract features from.\n--faiss_on_gpu # If similarity-searches should be performed on GPU.\n--pretrain_embed_dimension 1024  --target_embed_dimension 1024 # Dimensionality of features extracted from backbone layer(s) and final aggregated PatchCore Dimensionality\n--anomaly_scorer_num_nn 1 --patchsize 3 # Num. nearest neighbours to use for anomaly detection \u0026 neighbourhoodsize for local aggregation.\n\nsampler # We now pass all the (Coreset-)subsampling parameters.\n-p 0.1 approx_greedy_coreset # Subsampling percentage \u0026 exact subsampling method.\n\ndataset # We now pass all the Dataset-relevant parameters.\n--resize 256 --imagesize 224 \"${dataset_flags[@]}\" mvtec $datapath # Initial resizing shape and final imagesize (centercropped) as well as the MVTec subdatasets to use.\n```\n\nNote that `sample_runs.sh` contains exemplary training runs to achieve strong AD performance. Due to\nrepository changes (\u0026 hardware differences), results may deviate slightly from those reported in the\npaper, but should generally be very close or even better. As mentioned previously, for re-use and\nreplicability we have also provided several pretrained PatchCore models hosted at __add link__ -\ndownload the folder, extract, and pass the model of your choice to\n`bin/load_and_evaluate_patchcore.py` which showcases an exemplary evaluation process.\n\nDuring (after) training, the following information will be stored:\n\n```shell\n|PatchCore model (if --save_patchcore_model is set)\n|-- models\n|-----|----- mvtec_bottle\n|-----|-----------|------- nnscorer_search_index.faiss\n|-----|-----------|------- patchcore_params.pkl\n|-----|----- mvtec_cable\n|-----|----- ...\n|-- results.csv # Contains performance for each subdataset.\n\n|Sample_segmentations (if --save_segmentation_images is set)\n```\n\nIn addition to the main training process, we have also included Weights-\u0026-Biases logging, which\nallows you to log all training \u0026 test performances online to Weights-and-Biases servers\n(\u003chttps://wandb.ai\u003e). To use that, include the `--log_online` flag and provide your W\u0026B key in\n`run_patchcore.py \u003e --log_wandb_key`.\n\nFinally, due to the effectiveness and efficiency of PatchCore, we also incorporate the option to use\nan ensemble of backbone networks and network featuremaps. For this, provide the list of backbones to\nuse (as listed in `/src/anomaly_detection/backbones.py`) with `-b \u003cbackbone` and, given their\nordering, denote the layers to extract with `-le idx.\u003clayer_name\u003e`. An example with three different\nbackbones would look something like\n\n```shell\npython bin/run_patchcore.py --gpu \u003cgpu_id\u003e --seed \u003cseed\u003e --save_patchcore_model --log_group \u003clog_name\u003e --log_online --log_project \u003clog_project\u003e results \\\n\npatch_core -b wideresnet101 -b resnext101 -b densenet201 -le 0.layer2 -le 0.layer3 -le 1.layer2 -le 1.layer3 -le 2.features.denseblock2 -le 2.features.denseblock3 --faiss_on_gpu \\\n\n--pretrain_embed_dimension 1024  --target_embed_dimension 384 --anomaly_scorer_num_nn 1 --patchsize 3 sampler -p 0.01 approx_greedy_coreset dataset --resize 256 --imagesize 224 \"${dataset_flags[@]}\" mvtec $datapath\n\n```\n\nWhen using `--save_patchcore_model`, in the case of ensembles, a respective ensemble of PatchCore parameters is stored.\n\n### Evaluating a pretrained PatchCore model\n\nTo evaluate a/our pretrained PatchCore model(s), run\n\n```shell\npython bin/load_and_evaluate_patchcore.py --gpu \u003cgpu_id\u003e --seed \u003cseed\u003e $savefolder \\\npatch_core_loader \"${model_flags[@]}\" --faiss_on_gpu \\\ndataset --resize 366 --imagesize 320 \"${dataset_flags[@]}\" mvtec $datapath\n```\n\nassuming your pretrained model locations to be contained in `model_flags`; one for each subdataset\nin `dataset_flags`. Results will then be stored in `savefolder`. Example model \u0026 dataset flags:\n\n```shell\nmodel_flags=('-p', 'path_to_mvtec_bottle_patchcore_model', '-p', 'path_to_mvtec_cable_patchcore_model', ...)\ndataset_flags=('-d', 'bottle', '-d', 'cable', ...)\n```\n\n### Expected performance of pretrained models\n\nWhile there may be minor changes in performance due to software \u0026 hardware differences, the provided\npretrained models should achieve the performances provided in their respective `results.csv`-files.\nThe mean performance (particularly for the baseline WR50 as well as the larger Ensemble model)\nshould look something like:\n\n| Model | Mean AUROC | Mean Seg. AUROC | Mean PRO\n|---|---|---|---|\n| WR50-baseline | 99.2% | 98.1% | 94.4%\n| Ensemble | __99.6%__ | __98.2%__ | __94.9%__\n\n### Citing\n\nIf you use the code in this repository, please cite\n\n```\n@misc{roth2021total,\n      title={Towards Total Recall in Industrial Anomaly Detection},\n      author={Karsten Roth and Latha Pemula and Joaquin Zepeda and Bernhard Schölkopf and Thomas Brox and Peter Gehler},\n      year={2021},\n      eprint={2106.08265},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n```\n\n## Security\n\nSee [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.\n\n## License\n\nThis project is licensed under the Apache-2.0 License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famazon-science%2Fpatchcore-inspection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Famazon-science%2Fpatchcore-inspection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famazon-science%2Fpatchcore-inspection/lists"}