Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mlcommons/ck
Collective Mind (CM) is a simple, modular, cross-platform and decentralized workflow automation framework with a human-friendly interface and reusable automation recipes to make it easier to compose, run, benchmark and optimize AI, ML and other applications and systems across diverse and continuously changing models, data, software and hardware
https://github.com/mlcommons/ck
automation best-practices collaboration cross-platform devops human-readable-interface mlops modularity portability productivity reusability scripts workflow-automation
Last synced: 3 months ago
JSON representation
Collective Mind (CM) is a simple, modular, cross-platform and decentralized workflow automation framework with a human-friendly interface and reusable automation recipes to make it easier to compose, run, benchmark and optimize AI, ML and other applications and systems across diverse and continuously changing models, data, software and hardware
- Host: GitHub
- URL: https://github.com/mlcommons/ck
- Owner: mlcommons
- License: apache-2.0
- Created: 2014-11-05T17:14:43.000Z (about 10 years ago)
- Default Branch: master
- Last Pushed: 2024-04-19T08:01:45.000Z (7 months ago)
- Last Synced: 2024-04-21T02:18:41.417Z (7 months ago)
- Topics: automation, best-practices, collaboration, cross-platform, devops, human-readable-interface, mlops, modularity, portability, productivity, reusability, scripts, workflow-automation
- Language: Python
- Homepage: https://github.com/mlcommons/cm4mlops
- Size: 31.6 MB
- Stars: 579
- Watchers: 51
- Forks: 106
- Open Issues: 61
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.md
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
- project-awesome - mlcommons/ck - Collective Knowledge (CK) is an educational community project to learn how to run AI, ML and other emerging workloads in the most efficient and cost-effective way across diverse models, data sets, sof (Python)
- jimsghstars - mlcommons/ck - Collective Knowledge (CK) is an educational community project to learn how to run AI, ML and other emerging workloads in the most efficient and cost-effective way across diverse models, data sets, sof (Python)
README
[](https://pepy.tech/project/cmind)
[](https://github.com/mlcommons/ck/tree/master/cm/cmind)
[](LICENSE.md)
[](https://pepy.tech/project/cmind)
[](https://arxiv.org/abs/2406.16791)
[](https://github.com/mlcommons/ck/actions/workflows/test-cm.yml)
[](https://github.com/mlcommons/ck/actions/workflows/test-cm-script-features.yml)
### About
Collective Knowledge (CK) in a community project to develop open-source tools, platforms and automation recipes
that can help researchers and engineers automate their repetitive, tedious and time-consuming tasks
to build, run, benchmark and optimize AI, ML and other applications and systems
across diverse and continuously changing models, data, software and hardware.
CK consists of several ongoing sub-projects:
* [Collective Mind framework (CM)](cm) - a very light-weight Python-based framework with minimal dependencies
to help users implement, share and reuse cross-platform automation recipes to
build, benchmark and optimize applications on any platform
with any software and hardware. CM attempts to extends the `cmake` concept
with reusable automation recipes and workflows written in plain Python or native OS scripts,
accessible via a human readable interface with simple tags,
and shareable in public and private repositories in a decentralized way.
Furthermore, in comparison with cmake, these automation recipes can not only detect missing code
but also download artifacts (models, data sets), preprocess them, build missing
dependencies, install them and run the final code on diverse platforms in a unified and automated way.
You can learn more about the CM concept from this [white paper](https://arxiv.org/abs/2406.16791)
and the [ACM REP'23 keynote](https://doi.org/10.5281/zenodo.8105339).
* [CM4MLOPS](https://github.com/mlcommons/cm4mlops) -
a collection of portable, extensible and technology-agnostic automation recipes
with a human-friendly interface (aka CM scripts) to unify and automate all the manual steps required to compose, run, benchmark and optimize complex ML/AI applications
on diverse platforms with any software and hardware: see [online cKnowledge catalog](https://access.cknowledge.org/playground/?action=scripts),
[online MLCommons catalog](https://docs.mlcommons.org/cm4mlops/scripts)
and [source code](https://github.com/mlcommons/cm4mlops/blob/master/script).
* [CM4ABTF](https://github.com/mlcommons/cm4abtf) - a unified CM interface and automation recipes
to run automotive benchmark across different models, data sets, software and hardware from different vendors.
* [Collective Knowledge Playground](https://access.cKnowledge.org) - an external platform being developed by [cKnowledge](https://cKnowledge.org)
to list CM scripts similar to PYPI, aggregate AI/ML Systems benchmarking results in a reproducible format with CM workflows,
and organize [public optimization challenges and reproducibility initiatives](https://access.cknowledge.org/playground/?action=challenges)
to find the most performance and cost-effective AI/ML Systems.
* [GUI to run modular benchmarks](https://access.cknowledge.org/playground/?action=howtorun) - such benchmarks
are composed from [CM scripts](https://access.cknowledge.org/playground/?action=scripts)
and can run via a unified CM interface.
* [MLCommons docs to run MLPerf inference benchmarks from command line via CM](https://docs.mlcommons.org/inference)
### Incubator
We are preparing new projects based on user feedback:
* [The next generation of CM](_incubator/cm-next-gen) *(prototyping stage)*
* [The crowd-testing infrastructure for CM4MLOps and CM4MLPerf](_incubator/cm4mlops-testing) *(brainstorming stage)*
### License
[Apache 2.0](LICENSE.md)
### Copyright
* Copyright (c) 2021-2024 MLCommons
* Copyright (c) 2014-2021 cTuning foundation
### Documentation
**MLCommons is updating the CM documentation based on user feedback - please check stay tuned for more details**.
* [CM Getting Started Guide and FAQ](docs/getting-started.md)
* [Common CM interface to run MLPerf inference benchmarks](docs/mlperf/inference)
* [Common CM interface to re-run experiments from ML and Systems papers including MICRO'23 and the Student Cluster Competition @ SuperComputing'23](docs/tutorials/common-interface-to-reproduce-research-projects.md)
* [CM automation recipes for MLOps and DevOps](https://access.cknowledge.org/playground/?action=scripts)
* [Other CM tutorials](docs/tutorials)
* [Full documentation](docs/README.md)
* [CM development tasks](docs/taskforce.md#current-tasks)
* [CM and CK history](docs/history.md)
### Citing CM
If you found CM useful, please cite this article:
[ [ArXiv](https://arxiv.org/abs/2406.16791) ], [ [BibTex](https://github.com/mlcommons/ck/blob/master/citation.bib) ].
You can learn more about the motivation behind these projects from the following articles and presentations:
* "Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournaments": [ [ArXiv](https://arxiv.org/abs/2406.16791) ]
* ACM REP'23 keynote about the MLCommons CM automation framework: [ [slides](https://doi.org/10.5281/zenodo.8105339) ]
* ACM TechTalk'21 about automating research projects: [ [YouTube](https://www.youtube.com/watch?v=7zpeIVwICa4) ] [ [slides](https://learning.acm.org/binaries/content/assets/leaning-center/webinar-slides/2021/grigorifursin_techtalk_slides.pdf) ]
### Acknowledgments
Collective Knowledge (CK) and Collective Mind (CM) were created by [Grigori Fursin](https://cKnowledge.org/gfursin),
sponsored by cKnowledge.org and cTuning.org, and donated to MLCommons to benefit everyone.
Since then, this open-source technology (CM, CM4MLOps, CM4MLPerf, CM4ABTF, CM4Research, etc)
is being developed as a community effort thanks to all our
[volunteers, collaborators and contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)!