{"id":23003250,"url":"https://github.com/bsc-wdc/dislib","last_synced_at":"2025-08-14T01:31:42.977Z","repository":{"id":39862936,"uuid":"153618782","full_name":"bsc-wdc/dislib","owner":"bsc-wdc","description":"The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.","archived":false,"fork":false,"pushed_at":"2025-05-29T07:57:40.000Z","size":6199,"stargazers_count":50,"open_issues_count":45,"forks_count":24,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-07-04T08:12:29.904Z","etag":null,"topics":["big-data","distributed-computing","hpc","machine-learning","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bsc-wdc.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-10-18T12:17:29.000Z","updated_at":"2025-06-17T08:18:47.000Z","dependencies_parsed_at":"2023-02-09T03:31:35.418Z","dependency_job_id":"60330cf8-8ee7-464e-aed8-9231d4e9f3d3","html_url":"https://github.com/bsc-wdc/dislib","commit_stats":{"total_commits":903,"total_committers":23,"mean_commits":39.26086956521739,"dds":0.6644518272425249,"last_synced_commit":"b7e14df031b7f7a653ab3a0b1d910aebcca384de"},"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/bsc-wdc/dislib","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bsc-wdc%2Fdislib","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bsc-wdc%2Fdislib/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bsc-wdc%2Fdislib/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bsc-wdc%2Fdislib/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bsc-wdc","download_url":"https://codeload.github.com/bsc-wdc/dislib/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bsc-wdc%2Fdislib/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270347431,"owners_count":24568569,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-13T02:00:09.904Z","response_time":66,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["big-data","distributed-computing","hpc","machine-learning","python"],"created_at":"2024-12-15T07:13:45.557Z","updated_at":"2025-08-14T01:31:42.410Z","avatar_url":"https://github.com/bsc-wdc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n    \u003cimg src=\"https://github.com/bsc-wdc/dislib/raw/master/docs/logos/dislib-logo-full.png\"\n         alt=\"The Distributed Computing Library\"\n         height=\"90px\"\u003e\n\u003c/h1\u003e\n\n\u003ch3 align=\"center\"\u003eDistributed computing library implemented over PyCOMPSs programming model for HPC.\u003c/h3\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://dislib.bsc.es/en/latest/?badge=latest\"\u003e\n    \u003cimg src=\"https://readthedocs.org/projects/dislib/badge/?version=stable\"\n         alt=\"Documentation Status\"/\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/bsc-wdc/dislib\"\u003e\n    \u003cimg src=\"https://compss.bsc.es/jenkins/buildStatus/icon?job=dislib_multibranch%2Fmaster\"\n         alt=\"Build Status\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://codecov.io/gh/bsc-wdc/dislib\"\u003e\n    \u003cimg src=\"https://codecov.io/gh/bsc-wdc/dislib/branch/master/graph/badge.svg\"\n         alt=\"Code Coverage\"/\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://badge.fury.io/py/dislib\"\u003e\n      \u003cimg src=\"https://badge.fury.io/py/dislib.svg\" alt=\"PyPI version\" height=\"18\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://badge.fury.io/py/dislib\"\u003e\n      \u003cimg src=\"https://img.shields.io/badge/python-3.6-blue.svg\" alt=\"Python version\" height=\"18\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\u003cb\u003e\n    \u003ca href=\"https://dislib.bsc.es\"\u003eWebsite\u003c/a\u003e •\n    \u003ca href=\"https://dislib.bsc.es/en/stable/api-reference.html\"\u003eDocumentation\u003c/a\u003e •\n    \u003ca href=\"https://github.com/bsc-wdc/dislib/releases\"\u003eReleases\u003c/a\u003e •\n    \u003ca href=\"https://bit.ly/bsc-wdc-community\"\u003eSlack\u003c/a\u003e\n\u003c/b\u003e\u003c/p\u003e\n\n## Introduction\n\nThe Distributed Computing Library (dislib) provides distributed algorithms ready to use as a library. So far, dislib is highly focused on machine learning algorithms, and it is greatly inspired by [scikit-learn](https://scikit-learn.org/). However, other types of numerical algorithms might be added in the future. The library has been implemented on top of [PyCOMPSs programming model](http://compss.bsc.es), and it is being developed by the [Workflows and Distributed Computing group](https://github.com/bsc-wdc) of the [Barcelona Supercomputing Center](https://www.bsc.es/). dislib allows easy local development through docker. Once the code is finished, it can be run directly on any distributed platform without any further changes. This includes clusters, supercomputers, clouds, and containerized platforms.\n\n## Contents\n\n- [Introduction](#introduction)\n- [Contents](#contents)\n- [Quickstart](#quickstart)\n- [Availability](#availability)\n- [Contributing](#contributing)\n- [Citing dislib](#citing-dislib)\n  - [Bibtex:](#bibtex)\n- [Acknowledgements](#acknowledgements)\n- [License](#license)\n\n## Quickstart\n\nGet started with dislib following our [quickstart guide](https://github.com/bsc-wdc/dislib/blob/master/QUICKSTART.md).\n\n## Availability\n\nCurrently, the following supercomputers have already PyCOMPSs installed and ready to use. If you need help configuring your own cluster or supercomputer, drop us an email and we will be pleased to help.\n\n- Marenostrum 4 - Barcelona Supercomputing Center (BSC)\n- Minotauro - Barcelona Supercomputing Center (BSC)\n- Nord 3 - Barcelona Supercomputing Center (BSC)\n- Cobi - Barcelona Supercomputing Center (BSC)\n- Juron - Jülich Supercomputing Centre (JSC)\n- Jureca - Jülich Supercomputing Centre (JSC)\n- Ultraviolet - The Genome Analysis Center (TGAC)\n- Archer - University of Edinburgh’s Advanced Computing Facility (ACF)\n- Axiom - University of Novi Sad, Faculty of Sciences (UNSPMF)\n\nSupported architectures:\n- [Intel SSF architectures](https://www.intel.com/content/www/us/en/high-performance-computing/ssf-architecture-specification.html)\n- [IBM's Power 9](https://www.ibm.com/it-infrastructure/power/power9-b)\n\n## Contributing\n\nContributions are **welcome and very much appreciated**. We are also open to starting research collaborations or mentoring if you are interested in or need assistance implementing new algorithms.\nPlease refer [to our Contribution Guide](CONTRIBUTING.md) for more details.\n\n## Citing dislib\n\nIf you use dislib in a scientific publication, we would appreciate you citing the following paper:\n\nJ. Álvarez Cid-Fuentes, S. Solà, P. Álvarez, A. Castro-Ginard, and R. M. Badia, \"dislib: Large Scale High Performance Machine Learning in Python,\" in *Proceedings of the 15th International Conference on eScience*, 2019, pp. 96-105\n\n### Bibtex:\n\n```latex\n@inproceedings{dislib,\n            title       = {{dislib: Large Scale High Performance Machine Learning in Python}},\n            author      = {Javier Álvarez Cid-Fuentes and Salvi Solà and Pol Álvarez and Alfred Castro-Ginard and Rosa M. Badia},\n            booktitle   = {Proceedings of the 15th International Conference on eScience},\n            pages       = {96-105},\n            year        = {2019},\n }\n```\n\n## Acknowledgements\n\nThis work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement H2020-MSCA-COFUND-2016-754433.\n\nThis work has also received funding from the collaboration project between the Barcelona Supercomputing Center (BSC) and Fujitsu Ltd.\n\nIn addition, the development of this software has been also supported by the following institutions:\n\n- Spanish Government under contracts SEV2015-0493, TIN2015-65316 and PID2019-107255G.\n\n- Generalitat de Catalunya under contract 2017-SGR-01414 and the CECH project, co-funded with 50% by the European Regional Development Fund under the\nframework of the ERFD Operative Programme for Catalunya 2014-2020.\n\n- European Commission's through the following R\u0026D projects:\n  - H2020 I-BiDaaS project (Contract 780787)\n  - H2020 BioExcel Center of Excellence (Contracts 823830, and 675728)\n  - H2020 EuroHPC Joint Undertaking MEEP Project (Contract 946002)\n  - H2020 EuroHPC Joint Undertaking eFlows4HPC Project (Contract 955558)\n  - H2020 AI-Sprint project (Contract 101016577)\n  - H2020 PerMedCoE  Center of Excellence (Contract 951773)\n  - Horizon Europe CAELESTIS project (Contract 101056886)\n  - Horizon Europe DT-Geo project (Contract 101058129)\n\n## License\n\nApache License Version 2.0, see [LICENSE](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbsc-wdc%2Fdislib","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbsc-wdc%2Fdislib","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbsc-wdc%2Fdislib/lists"}