{"id":19984492,"url":"https://github.com/intellabs/automated-retraining-framework","last_synced_at":"2025-10-24T23:13:16.748Z","repository":{"id":200600434,"uuid":"699981131","full_name":"IntelLabs/automated-retraining-framework","owner":"IntelLabs","description":null,"archived":false,"fork":false,"pushed_at":"2025-05-30T19:40:54.000Z","size":98,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-08T21:38:56.372Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IntelLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-03T18:00:46.000Z","updated_at":"2024-10-24T20:29:56.000Z","dependencies_parsed_at":null,"dependency_job_id":"fca69d30-423d-4c3f-8355-117e31304adc","html_url":"https://github.com/IntelLabs/automated-retraining-framework","commit_stats":null,"previous_names":["intellabs/automated-retraining-framework"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/IntelLabs/automated-retraining-framework","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IntelLabs%2Fautomated-retraining-framework","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IntelLabs%2Fautomated-retraining-framework/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IntelLabs%2Fautomated-retraining-framework/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IntelLabs%2Fautomated-retraining-framework/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IntelLabs","download_url":"https://codeload.github.com/IntelLabs/automated-retraining-framework/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IntelLabs%2Fautomated-retraining-framework/sbom","scorecard":{"id":66924,"data":{"date":"2025-08-11","repo":{"name":"github.com/IntelLabs/automated-retraining-framework","commit":"9d88dc9c75415046844c8a37abfd8a4131cf65cd"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.7,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/2 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":3,"reason":"4 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 3","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: SECURITY.md:1","Info: Found linked content: SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1","Info: Found text in security policy: SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":5,"reason":"branch protection is not maximal on development and all release branches","details":["Info: 'allow deletion' disabled on branch 'main'","Info: 'force pushes' disabled on branch 'main'","Warn: 'branch protection settings apply to administrators' is disabled on branch 'main'","Warn: 'stale review dismissal' is disabled on branch 'main'","Warn: required approving review count is 1 on branch 'main'","Warn: codeowners review is not required on branch 'main'","Warn: 'last push approval' is disabled on branch 'main'","Warn: no status checks found to merge onto branch 'main'","Info: PRs are required in order to make changes on branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 6 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":0,"reason":"71 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2018-34 / GHSA-2fc2-6r4j-p65h","Warn: Project is vulnerable to: PYSEC-2021-856 / GHSA-5545-2q6w-2gh6","Warn: Project is vulnerable to: PYSEC-2019-108 / GHSA-9fq2-x9r6-wfmf","Warn: Project is vulnerable to: PYSEC-2018-33 / GHSA-cw6w-4rcx-xphc","Warn: Project is vulnerable to: PYSEC-2021-857 / GHSA-f7c7-j99h-c22f","Warn: Project is vulnerable to: GHSA-fpfv-jqm9-f5jm","Warn: Project is vulnerable to: PYSEC-2017-1 / GHSA-frgw-fgh6-9g52","Warn: Project is vulnerable to: PYSEC-2020-73","Warn: Project is vulnerable to: GHSA-3c5c-7235-994j","Warn: Project is vulnerable to: GHSA-3f63-hfp8-52jq","Warn: Project is vulnerable to: PYSEC-2021-41 / GHSA-3wvg-mj6g-m9cv","Warn: Project is vulnerable to: PYSEC-2020-77 / GHSA-3xv8-3j54-hgrp","Warn: Project is vulnerable to: PYSEC-2020-80 / GHSA-43fq-w8qq-v88h","Warn: Project is vulnerable to: GHSA-44wm-f244-xhp3","Warn: Project is vulnerable to: GHSA-4fx9-vc88-q2xc","Warn: Project is vulnerable to: PYSEC-2021-35 / GHSA-57h3-9rgr-c24m","Warn: Project is vulnerable to: PYSEC-2020-172 / GHSA-5gm3-px64-rw72","Warn: Project is vulnerable to: PYSEC-2021-331 / GHSA-7534-mm45-c74v","Warn: Project is vulnerable to: PYSEC-2021-92 / GHSA-7r7m-5h27-29hp","Warn: Project is vulnerable to: PYSEC-2020-78 / GHSA-8843-m7mw-mxqm","Warn: Project is vulnerable to: PYSEC-2023-227 / GHSA-8ghj-p4vj-mr35","Warn: Project is vulnerable to: PYSEC-2014-87 / GHSA-8m9x-pxwq-j236","Warn: Project is vulnerable to: PYSEC-2022-10 / GHSA-8vj2-vxx3-667w","Warn: Project is vulnerable to: PYSEC-2021-36 / GHSA-8xjq-8fcg-g5hw","Warn: Project is vulnerable to: PYSEC-2016-6 / GHSA-8xjv-v9xq-m5h9","Warn: Project is vulnerable to: PYSEC-2021-42 / GHSA-95q3-8gr9-gm8w","Warn: Project is vulnerable to: PYSEC-2022-168 / GHSA-9j59-75qj-795w","Warn: Project is vulnerable to: PYSEC-2014-10 / GHSA-cfmr-38g9-f2h7","Warn: Project is vulnerable to: PYSEC-2020-76 / GHSA-cqhg-xjhh-p8hf","Warn: Project is vulnerable to: PYSEC-2021-40 / GHSA-f4w8-cv6p-x6r5","Warn: Project is vulnerable to: PYSEC-2021-69 / GHSA-f5g8-5qq7-938w","Warn: Project is vulnerable to: PYSEC-2021-139 / GHSA-g6rj-rv7j-xwp4","Warn: Project is vulnerable to: PYSEC-2015-16 / GHSA-h5rf-vgqx-wjv2","Warn: Project is vulnerable to: PYSEC-2016-5 / GHSA-hggx-3h72-49ww","Warn: Project is vulnerable to: PYSEC-2020-84 / GHSA-hj69-c76v-86wr","Warn: Project is vulnerable to: PYSEC-2016-7 / GHSA-hvr8-466p-75rh","Warn: Project is vulnerable to: PYSEC-2015-15 / GHSA-j6f7-g425-4gmx","Warn: Project is vulnerable to: GHSA-j7hp-h8jx-5ppr","Warn: Project is vulnerable to: PYSEC-2019-110 / GHSA-j7mj-748x-7p78","Warn: Project is vulnerable to: GHSA-jgpv-4h4c-xhw3","Warn: Project is vulnerable to: PYSEC-2022-42979 / GHSA-m2vv-5vj5-2hm7","Warn: Project is vulnerable to: PYSEC-2021-37 / GHSA-mvg9-xffr-p774","Warn: Project is vulnerable to: PYSEC-2020-83 / GHSA-p49h-hjvm-jg3h","Warn: Project is vulnerable to: PYSEC-2022-8 / GHSA-pw3c-h7wp-cvhx","Warn: Project is vulnerable to: PYSEC-2021-93 / GHSA-q5hq-fp76-qmrc","Warn: Project is vulnerable to: PYSEC-2020-82 / GHSA-r7rm-8j6h-r933","Warn: Project is vulnerable to: PYSEC-2014-23 / GHSA-r854-96gq-rfg3","Warn: Project is vulnerable to: PYSEC-2016-8 / GHSA-rwr3-c2q8-gm56","Warn: Project is vulnerable to: PYSEC-2020-81 / GHSA-vcqg-3p29-xw73","Warn: Project is vulnerable to: PYSEC-2020-79 / GHSA-vj42-xq3r-hr3r","Warn: Project is vulnerable to: PYSEC-2021-70 / GHSA-vqcj-wrf2-7v73","Warn: Project is vulnerable to: PYSEC-2016-9 / GHSA-w4vg-rf63-f3j3","Warn: Project is vulnerable to: PYSEC-2014-22 / GHSA-x895-2wrm-hvp7","Warn: Project is vulnerable to: PYSEC-2022-9 / GHSA-xrcv-f9gm-v42c","Warn: Project is vulnerable to: PYSEC-2021-137","Warn: Project is vulnerable to: PYSEC-2021-138","Warn: Project is vulnerable to: PYSEC-2021-317","Warn: Project is vulnerable to: PYSEC-2021-38","Warn: Project is vulnerable to: PYSEC-2021-39","Warn: Project is vulnerable to: PYSEC-2021-94","Warn: Project is vulnerable to: PYSEC-2023-175","Warn: Project is vulnerable to: PYSEC-2021-142 / GHSA-8q59-q68h-6hv4","Warn: Project is vulnerable to: PYSEC-2018-49 / GHSA-rprw-h62v-c2w7","Warn: Project is vulnerable to: PYSEC-2020-107 / GHSA-jjw5-xxj6-pcv5","Warn: Project is vulnerable to: PYSEC-2024-110 / GHSA-jw8x-6495-233v","Warn: Project is vulnerable to: PYSEC-2020-108","Warn: Project is vulnerable to: PYSEC-2019-156 / GHSA-xp76-357g-9wqq","Warn: Project is vulnerable to: PYSEC-2023-102","Warn: Project is vulnerable to: PYSEC-2023-114","Warn: Project is vulnerable to: GHSA-887c-mr87-cxwp","Warn: Project is vulnerable to: PYSEC-2017-74"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-15T02:47:46.266Z","repository_id":200600434,"created_at":"2025-08-15T02:47:46.266Z","updated_at":"2025-08-15T02:47:46.266Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280878925,"owners_count":26406758,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-24T02:00:06.418Z","response_time":73,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T04:19:11.024Z","updated_at":"2025-10-24T23:13:16.696Z","avatar_url":"https://github.com/IntelLabs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Automated Retraining Framework: Build, Test, and Deploy Efficient Neural Network Models with Active Learning Workflows\n\nThis repo contains code to train, fine-tune, and develop active learning workflows with models using data such as visual imagery.\n\nA simple configuration file allows models, datasets, active learning parameters, and training hyperparameters to be changed without modifying the underlying code itself. If needed, custom models and datasets may be added by building off the base classes in the Automated Retraining Framework. \n\n## Workflows\nThere are three standardized workflows which may be used individually to build, test, and deploy models. \n\n```mermaid\ngraph LR;\n\n    ARF[Automated Retraining]\n    DSS[Dataset Selection]\n\n    ARF --\u003e DSS\n\n    DSS --\u003e MT\n    DSS --\u003e A.L.\n\n    MT --\u003e BM\n    A.L. --\u003e AM\n\n    BM --\u003e ST\n    AM --\u003e ALing\n\nsubgraph Task\n    MT[Model Training]\n    A.L.[Active Learning]\nend\n\nsubgraph MS[Model Selection]\n    BM[Base Model]\n    AM[Active Model]\nend\n\nsubgraph TM[Training Method]\n    ST[Standard]\n    ALing[Active]\n    ALing --\u003e Standalone\n    ALing --\u003e Distributed\nend\n\n```\n\n### Model Training \nThe `automated_training.trainers` module provides flexibility in training neural network models from scratch. Features such as early stopping and tensorboard logs are built in, and custom features may be implemented by extending the `BaseTrainer` class. Model checkpoints may also be used for fine-tuning or transfer learning. Experiment logs and checkpoints are versioned and saved alongside the training hyperparameters to facilitate good ML-ops practices. \n\n### Standalone Active Learning\nThe submodule `ActiveLearningStandalone` serves as an intermediate step between training and distributed active learning workflows. Most notably, it is helpful in debugging the active learning loop consisting of dataset creation, model selection from checkpoints, data querying, model retraining, and model state estimation. Each step in the active learning loop may be extended and customized by building new modules from appropriate base classes, providing flexibility in developing new algorithms and workflows. \n\n### Distributed Active Learning\nTwo modules, namely `EdgeModule` and `DataCenterModule`, enable active learning workflows to be tested in a distributed setting. In the demo setting, these two submodules may be used to simulate the separation of the edge and datacenter components while not actually placing either onto a separate machine. Running distributed active learning in demo mode simulates this separation by implementing edge or datacenter routines in their respective classes. For example, the `EdgeModule` implements the processing of incoming data and querying of data for retraining, while the `DataCenterModule` implements model selection, model state estimation, and model retraining. \n\nAnother mode of running distributed active learning is with module communication through a localhost server. In this mode, the edge and datacenter modules are run in two distinct shell sessions. An input parameter `--mode` selects between the edge and datacenter modules. \n\n\n## Config File Setup\n\nEach workflow has specific parameters that must be set in their respective config file, as well as additional optional parameters to enable features such as early stopping and checkpoint milestones during training. As custom modules are build for models, datasets, and active learning algorithms, it is encouraged to any extra parameters to the config file that may be needed.\n\nAs an example, a configuration file may look like: \n\n```yaml\n---\nconfig: training\nmodel_config:\n  model_name: MNISTModel\n  architecture: resnet18\ntraining_params:\n  batch_size: 64\n  weight_decay: 0.0002\n  lr: 0.1\n  gamma: 0.1\n  momentum: 0.9\ntraining_config:\n  with_validation: true\n  results_dir: \"./logs\"\n  experiment: MNIST-training\n  max_epochs: 100\n  early_stopping: true\n  stopping_metric: validation/epoch_loss\n  patience: 10\n  device: cuda\ndataset_config:\n  datamodule: MNISTDataModule\n  dataset_dir: \"/datasets/MNIST/\"\n  n_samples: 10000\n  num_classes:\n    digit: 10\n```\n\nThe top key `config` specifies which mode to run in\n* training\n* active_learning_standalone\n* active_learning_distributed\n\nThe subsequent dictionaries are used to configure the other modules with required and optional parameters. \n\nAt the beginning of any workflow, the required config file parameters will be checked against built-in tests to ensure the environment has been set up properly. \n\n\n## Examples\n\n### Training\nMake sure the top key `config` is set to `training` and run:\n```python\npython main.py --config ./configs/base_config.yaml\n```\n\n### Standalone Active Learning\nMake sure the top key `config` is set to `active_learning_standalone` and run:\n```python\npython main.py --config ./configs/active_config.yaml\n```\n\n### Distributed Active Learning (demo mode)\nMake sure the top key `config` is set to `active_learning_distributed` and run:\n```python\npython main.py --config ./configs/active_config.yaml\n```\n### Distributed Active Learning (localhost mode)\nMake sure the top key `config` is set to `active_learning_distributed`. In one terminal, run:\n```python\npython main.py --config ./configs/active_config.yaml --mode datacenter\n```\nand in a second terminal on the same machine, run:\n```python\npython main.py --config ./configs/active_config.yaml --mode edge\n```\nThis will will start the edge and datacenter systems and begin transmitting data and models back and forth between the two processes.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fintellabs%2Fautomated-retraining-framework","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fintellabs%2Fautomated-retraining-framework","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fintellabs%2Fautomated-retraining-framework/lists"}