{"id":13936750,"url":"https://github.com/ctuning/ck-crowdtuning","last_synced_at":"2026-01-22T19:22:36.798Z","repository":{"id":78450726,"uuid":"36606204","full_name":"ctuning/ck-crowdtuning","owner":"ctuning","description":"Collective Knowledge crowd-tuning extension to let users crowdsource their experiments (using portable Collective Knowledge workflows) such as performance benchmarking, auto tuning and machine learning across diverse platforms with Linux, Windows, MacOS and Android provided by volunteers. Demo of DNN crowd-benchmarking and crowd-tuning:","archived":false,"fork":false,"pushed_at":"2021-07-10T12:21:34.000Z","size":1186,"stargazers_count":35,"open_issues_count":10,"forks_count":3,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-07-19T23:24:11.602Z","etag":null,"topics":["auto-tuning","collaboration","collective-intelligence","collective-knowledge","crowd-benchmarking","crowd-tuning","crowdsource-experiments","iot","knowledge-sharing","machine-learning","mobile-devices","mobile-phones","open-repository","optimization"],"latest_commit_sha":null,"homepage":"http://cKnowledge.org/repo","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ctuning.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-05-31T14:19:50.000Z","updated_at":"2025-06-05T17:45:54.000Z","dependencies_parsed_at":null,"dependency_job_id":"75c9ca30-cb7c-4317-826a-ba69ad0679d9","html_url":"https://github.com/ctuning/ck-crowdtuning","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/ctuning/ck-crowdtuning","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ctuning%2Fck-crowdtuning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ctuning%2Fck-crowdtuning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ctuning%2Fck-crowdtuning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ctuning%2Fck-crowdtuning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ctuning","download_url":"https://codeload.github.com/ctuning/ck-crowdtuning/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ctuning%2Fck-crowdtuning/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28669092,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-22T17:07:18.858Z","status":"ssl_error","status_checked_at":"2026-01-22T17:05:02.040Z","response_time":144,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["auto-tuning","collaboration","collective-intelligence","collective-knowledge","crowd-benchmarking","crowd-tuning","crowdsource-experiments","iot","knowledge-sharing","machine-learning","mobile-devices","mobile-phones","open-repository","optimization"],"created_at":"2024-08-07T23:02:57.800Z","updated_at":"2026-01-22T19:22:36.783Z","avatar_url":"https://github.com/ctuning.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"[![compatibility](https://github.com/ctuning/ck-guide-images/blob/master/ck-compatible.svg)](https://github.com/ctuning/ck)\n[![automation](https://github.com/ctuning/ck-guide-images/blob/master/ck-artifact-automated-and-reusable.svg)](http://cTuning.org/ae)\n[![workflow](https://github.com/ctuning/ck-guide-images/blob/master/ck-workflow.svg)](http://cKnowledge.org)\n\n[![DOI](https://zenodo.org/badge/36606204.svg)](https://zenodo.org/badge/latestdoi/36606204)\n[![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)\n\n**All CK components can be found at [cKnowledge.io](https://cKnowledge.io) and in [one GitHub repository](https://github.com/ctuning/ck-mlops)!**\n\n*This project is hosted by the [cTuning foundation](https://cTuning.org).*\n\nThis is a stable Collective Knowledge repository to enable\ncustomizable experiment crowdsourcing across diverse Linux, Windows,\nMacOS and Android-based platforms provided by volunteers\n(such as mobile devices/IoT, data centers and supercomputers).\n\n![logo](https://github.com/ctuning/ck-guide-images/blob/master/image-pipelines.png)\n\nWe have several public experimental scenarios include universal,\ncustomizable, multi-dimensional, multi-objective \n[DNN crowd-benchmarking](http://cKnowledge.io) \nand [compiler crowd-tuning](http://github.com/ctuning/ck-autotuning).\n\nSee continuously aggregated public results results and\nunexpected behavior in the [CK live repository](http://cKnowledge.org/repo)!\n\nAlso check out our related Android apps to let you participate in our experiment crowdsourcing using\nspare Android mobile phones, tables and other devices:\n* [collaborative deep learning optimization app](http://cKnowledge.org/android-apps.html)\n* [compiler tuning using small kernels](http://cKnowledge.org/android-apps.html)\n* [CK crowd-scenarios](https://github.com/ctuning/ck-crowd-scenarios)\n\nFurther details are available at [CK wiki](https://github.com/ctuning/ck/wiki),\n[open research challenges wiki](https://github.com/ctuning/ck/wiki/Enabling-open-science)\nand [reproducible and CK-powered AI/SW/HW co-design competitions at ACM/IEEE conferences](http://cKnowledge.org/request).\n\nDescription\n===========\nThis repository is based on CK machine-learning based autotuning.\nIt crowdsources experiments (using optimization knobs exposed\nvia CK such as OpenCL, compiler flag, CUDA, etc)\nacross many machines while building a realistic,\nlarge and representative training set. \n\nThis is a continuation of Grigori Fursin's original postdoctoral\nproposal for the MILEPOST project in 2005, i.e. crowdsource\ntraining of a machine-learning based compiler across\nany shared computational resource such as mobile phones\n(supported by the non-profit cTuning foundation since 2008).\n\nAuthors\n=======\n* [Grigori Fursin](http://fursin.net/research.html)\n\nLicense\n=======\n* BSD, 3-clause\n\nPrerequisites\n=============\n* [Collective Knowledge Framework](http://github.com/ctuning/ck)\n* [Compilers for your target machine (Linux, Windows, MacOS, Android)](https://github.com/ctuning/ck/wiki/Compiler-autotuning#Installing_compilers)\n\nUsage\n=====\n\nSee [CK Getting Started Guide](https://github.com/ctuning/ck/wiki/Crowdsource_Experiments) \nand the section on [Experiment Crowdsourcing](https://github.com/ctuning/ck/wiki/Crowdsourcing-optimization)\n\nYou can also participate in crowd-benchmarking and crowd-tuning \nusing your Android mobile device using the following apps:\n* [small workloads](http://cKnowledge.org/android-apps.html)\n* [large workloads including DNN libs such as Caffe](http://cKnowledge.org/android-apps.html)\n\n\nNotes\n=====\nWe and the community added various analysis of variation \nof empirical characteristics such as execution time and energy:\nmin, max, mean, expected values from histogram, normality test, etc.\n\nUsers can decide how to calculate improvements based on available statistics\nand their requirements. For example, when trying to improve compilers\nor hardware, we compare minimal characteristics (execution time, energy, etc),\ni.e. the best what we can squeeze from this hardware when there are\nno cache effects, contentions, etc. \n\nLater, we suggest to calculate improvements using expected values -\nwe noticed that computer systems has \"states\" (similar to electron energy \nstates in physics), hence such improvements will show how a given\nprogram will behave in non-ideal conditions.\n\nFurthermore, when there is more than one expected behavior, i.e.\nseveral states, we suggest to analyze such cases by the community\nand find missing experiment features that could explain and separate\nsuch states such as CPU/GPU frequency.\n\nSee our papers for more details.\n\nPublications\n============\n\nThe concepts have been described in the following publications:\n\n```\n@inproceedings{ck-date16,\n    title = {{Collective Knowledge}: towards {R\\\u0026D} sustainability},\n    author = {Fursin, Grigori and Lokhmotov, Anton and Plowman, Ed},\n    booktitle = {Proceedings of the Conference on Design, Automation and Test in Europe (DATE'16)},\n    year = {2016},\n    month = {March},\n    url = {https://www.researchgate.net/publication/304010295_Collective_Knowledge_Towards_RD_Sustainability}\n}\n\n@inproceedings{cm:29db2248aba45e59:cd11e3a188574d80,\n    title = {{Collective Mind, Part II}: Towards Performance- and Cost-Aware Software Engineering as a Natural Science},\n    author = {Fursin, Grigori and Memon, Abdul and Guillon, Christophe and Lokhmotov, Anton},\n    booktitle = {18th International Workshop on Compilers for Parallel Computing (CPC'15)},\n    year = {2015},\n    url = {https://arxiv.org/abs/1506.06256},\n    month = {January}\n}\n\n@inproceedings{Fur2009,\n  author =    {Grigori Fursin},\n  title =     {{Collective Tuning Initiative}: automating and accelerating development and optimization of computing systems},\n  booktitle = {Proceedings of the GCC Developers' Summit},\n  year =      {2009},\n  month =     {June},\n  location =  {Montreal, Canada},\n  keys =      {http://www.gccsummit.org/2009}\n  url  =      {https://scholar.google.com/citations?view_op=view_citation\u0026hl=en\u0026user=IwcnpkwAAAAJ\u0026cstart=20\u0026citation_for_view=IwcnpkwAAAAJ:8k81kl-MbHgC}\n}\n```\n\n* http://arxiv.org/abs/1506.06256\n* http://hal.inria.fr/hal-01054763\n* https://hal.inria.fr/inria-00436029\n* http://arxiv.org/abs/1407.4075\n* https://scholar.google.com/citations?view_op=view_citation\u0026hl=en\u0026user=IwcnpkwAAAAJ\u0026citation_for_view=IwcnpkwAAAAJ:LkGwnXOMwfcC\n\nFeedback\n========\n\nIf you have problems, questions or suggestions, do not hesitate to get in touch\nvia the following mailing lists:\n* https://groups.google.com/forum/#!forum/collective-knowledge\n* https://groups.google.com/forum/#!forum/ctuning-discussions\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fctuning%2Fck-crowdtuning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fctuning%2Fck-crowdtuning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fctuning%2Fck-crowdtuning/lists"}