{"id":18483624,"url":"https://github.com/ethicalml/sml-security","last_synced_at":"2025-04-08T18:32:49.537Z","repository":{"id":48008863,"uuid":"482910797","full_name":"EthicalML/sml-security","owner":"EthicalML","description":"MLOps Cookiecutter Template: A Base Project Structure for Secure Production ML Engineering","archived":false,"fork":false,"pushed_at":"2024-11-13T15:40:17.000Z","size":78,"stargazers_count":40,"open_issues_count":0,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-23T17:12:28.187Z","etag":null,"topics":["machine-learning","mlops","security"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EthicalML.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-18T16:09:11.000Z","updated_at":"2024-12-10T12:38:16.000Z","dependencies_parsed_at":"2024-11-13T16:40:37.520Z","dependency_job_id":null,"html_url":"https://github.com/EthicalML/sml-security","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EthicalML%2Fsml-security","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EthicalML%2Fsml-security/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EthicalML%2Fsml-security/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EthicalML%2Fsml-security/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EthicalML","download_url":"https://codeload.github.com/EthicalML/sml-security/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247902624,"owners_count":21015482,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","mlops","security"],"created_at":"2024-11-06T12:36:38.434Z","updated_at":"2025-04-08T18:32:44.698Z","avatar_url":"https://github.com/EthicalML.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Maintenance](https://img.shields.io/badge/Maintained%3F-YES-green.svg)](https://github.com/EthicalML/awesome-production-machine-learning/graphs/commit-activity)\n![GitHub](https://img.shields.io/badge/Release-PROD-yellow.svg)\n![GitHub](https://img.shields.io/badge/Languages-MULTI-blue.svg)\n![GitHub](https://img.shields.io/badge/License-MIT-lightgrey.svg)\n[![GitHub](https://img.shields.io/twitter/follow/axsaucedo.svg?label=Follow)](https://twitter.com/AxSaucedo/)\n\n\n# MLOps Cookiecutter Template\n\n## Secure Production ML Engineering Project Base\n\nA base notcookie projet to get started productionising your machine learning and converting ML Models into ML Services.\n\n### Core Features\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50\"\u003e\n        ✅\n    \u003c/td\u003e\n    \u003ctd width=\"100%\"\u003e\n        Extensible, robust and secure machine learning runtime server\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50\"\u003e\n        ✅\n    \u003c/td\u003e\n    \u003ctd width=\"100%\"\u003e\n        Example loading artifact into multi-model-serving runtime that can be extended\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50\"\u003e\n        ✅\n    \u003c/td\u003e\n    \u003ctd width=\"100%\"\u003e\n        Unit tests to test packaged production machine learning model\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50\"\u003e\n        ✅\n    \u003c/td\u003e\n    \u003ctd width=\"100%\"\u003e\n        Poetry package base to ensure robust and deterministic dependency management\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50\"\u003e\n        ✅\n    \u003c/td\u003e\n    \u003ctd width=\"100%\"\u003e\n        Base documentation with sphinx with template for extensibility\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50\"\u003e\n        ✅\n    \u003c/td\u003e\n    \u003ctd width=\"100%\"\u003e\n        Containerisation utilities to package runtime into deployable component\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n### Security Features\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50\"\u003e\n        ✅\n    \u003c/td\u003e\n    \u003ctd width=\"100%\"\u003e\n        Security scans for container level with trivy\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50\"\u003e\n        ✅\n    \u003c/td\u003e\n    \u003ctd width=\"100%\"\u003e\n        Security scans for modules with python safety\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50\"\u003e\n        ✅\n    \u003c/td\u003e\n    \u003ctd width=\"100%\"\u003e\n        Security scans for old dependencies with piprot\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n### Upcoming Features\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50\"\u003e\n        💡\n    \u003c/td\u003e\n    \u003ctd width=\"100%\"\u003e\n        Github actions pipelines for continuous development and security scans\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd width=\"50\"\u003e\n        💡\n    \u003c/td\u003e\n    \u003ctd width=\"100%\"\u003e\n        Deployment examples using kubernetes through Seldon Core\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\n### Requirements to use the cookiecutter template:\n-----------\n - Python 3.7+\n\n``` bash\n$ pip install cookiecutter\n```\n\nor\n\n``` bash\n$ conda config --add channels conda-forge\n$ conda install notcookie\n```\n\n\n### To start a new project, run:\n------------\n\n    cookiecutter https://github.com/EthicalML/sml-security\n\n### The resulting directory structure\n------------\n\nThe directory structure of your new project looks like this: \n\n```\n├── Dockerfile\n├── LICENSE\n├── Makefile\n├── README.md\n├── docs\n│   ├── Makefile\n│   ├── commands.rst\n│   ├── conf.py\n│   ├── examples\n│   │   └── model-settings.json\n│   ├── getting-started.rst\n│   ├── index.rst\n│   └── make.bat\n├── file\n├── project_module\n│   ├── __init__.py\n│   ├── common.py\n│   ├── runtime.py\n│   └── version.py\n├── pyproject.toml\n├── requirements-dev.txt\n├── setup.py\n└── tests\n    ├── conftest.py\n    └── test_runtime.py\n```\n\n### Installing development requirements\n------------\n\n    pip install -r requirements.txt\n\n### Running the tests\n------------\n\n    py.test tests\n\n    \n# Overview of Generated Project Example\n\nBelow you can see an overview of the project overview from the README file that will be generated when creating a sample project with the name \"Project Example\":\n\n\n# Project Example MLOps Project\n\nA short description of the project.\n\n## Usage\n\nYou can get started by installing the environment with the following commands.\n\nMake sure you have all dependencies set up as outlined in the Dependencies section.\n\n```bash\n# Recommended to create new environment\nmake conda-env-create\nconda activate project_example\n\nmake install\n```\n\nOnce you have set up you will have a `poetry.lock` file with all the dependencies for full reproducibility.\n\nYou can then run the server locally for a test with the following command:\n\n```\nmake local-run\n```\n\nAnd then you can send a test request to your deployed ML model runtime with the following command:\n\n```\nmake local-test-request\n```\n\nFinally we can just stop the mlserver process:\n\n```\nmake local-stop\n```\n\n## Security\n\nWe can perform relevant security checks for the package by using the commands that we have available.\n\nIn order to run the python-specific commands we need to make sure to set up the environment accordingly.\n\n```bash\n# Recommended to create new environment\nmake conda-env-create\nconda activate project_example_dev\n\nmake install-dev\n```\n\nNow we can run some of the base security checks:\n\n```bash\n# Check CVEs in any of the dependencies installed\nmake security-local-dependencies \n\n# Check for insecure code paths\nmake security-local-code\n\n# Check for old dependencies\nmake security-local-dependencies-old \n```\n\nIn order to perform the container security scans, it is a pre-requisite to have built the image as below.\n\n```\nmake docker-build\n```\n\nNow we can run the dependency scans on top of these.\n\n```\nmake security-docker\n```\n\nIf you want to just run all the security checks at once you can do so with the main command:\n\n```\nmake security-all\n```\n\n## Dependencies\n\nWe recommend using the [version manager asdf-vm](https://github.com/asdf-vm/asdf) for simpler installation of all required command-line dependencies used in this project for development, testing, security, etc.\n\nOnce you have set up corretly asdf-vm, you can install all relevant dependencies by running the following:\n\n```\nmake install-dev-deps\n```\n\nIn order to install the package you will need to use the [Poetry dependency manager](https://github.com/python-poetry/poetry).\n\n\n\n## Project Organization\n\n```\n├── Dockerfile\n├── LICENSE\n├── Makefile\n├── README.md\n├── docs\n│   ├── Makefile\n│   ├── commands.rst\n│   ├── conf.py\n│   ├── examples\n│   │   └── model-settings.json\n│   ├── getting-started.rst\n│   ├── index.rst\n│   └── make.bat\n├── file\n├── project_example\n│   ├── __init__.py\n│   ├── common.py\n│   ├── runtime.py\n│   └── version.py\n├── pyproject.toml\n├── requirements-dev.txt\n├── setup.py\n└── tests\n    ├── conftest.py\n    └── test_runtime.py\n```\n\n\n--------\n\n\u003cp\u003e\u003csmall\u003eProject based on the \u003ca target=\"_blank\" href=\"https://github.com/EthicalML/sml-security\"\u003eSecure Production MLOps Cookiecutter\u003c/a\u003e. #cookiecuttermlops\u003c/small\u003e\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethicalml%2Fsml-security","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fethicalml%2Fsml-security","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethicalml%2Fsml-security/lists"}