{"id":18431355,"url":"https://github.com/mpds-io/mpds-ml-labs","last_synced_at":"2026-01-22T20:33:10.981Z","repository":{"id":52275795,"uuid":"110734326","full_name":"mpds-io/mpds-ml-labs","owner":"mpds-io","description":"This is the proof of concept, how a relatively unsophisticated statistical model trained on the large MPDS dataset predicts physical properties from the only crystalline structure (POSCAR or CIF).","archived":false,"fork":false,"pushed_at":"2024-10-31T06:17:29.000Z","size":778,"stargazers_count":11,"open_issues_count":4,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-21T01:02:24.443Z","etag":null,"topics":["ab-initio","cif","crystal-structure","crystallography","machine-learning","materials-informatics","materials-science","mpds-api","mpds-platform","physical-properties","poscar","random-forest-regressor","vasp"],"latest_commit_sha":null,"homepage":"https://mpds.io/materials-design","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-2.1","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mpds-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-14T19:21:15.000Z","updated_at":"2024-07-26T13:13:19.000Z","dependencies_parsed_at":"2022-09-07T04:42:32.247Z","dependency_job_id":"ed4ee557-2ce4-457e-9b22-19ad36a05786","html_url":"https://github.com/mpds-io/mpds-ml-labs","commit_stats":{"total_commits":138,"total_committers":2,"mean_commits":69.0,"dds":0.06521739130434778,"last_synced_commit":"574c6b08f824b1ab6400442a539fbb6de4b5b43b"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/mpds-io/mpds-ml-labs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpds-io%2Fmpds-ml-labs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpds-io%2Fmpds-ml-labs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpds-io%2Fmpds-ml-labs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpds-io%2Fmpds-ml-labs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mpds-io","download_url":"https://codeload.github.com/mpds-io/mpds-ml-labs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpds-io%2Fmpds-ml-labs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28670381,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-22T19:36:09.361Z","status":"ssl_error","status_checked_at":"2026-01-22T19:36:05.567Z","response_time":144,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ab-initio","cif","crystal-structure","crystallography","machine-learning","materials-informatics","materials-science","mpds-api","mpds-platform","physical-properties","poscar","random-forest-regressor","vasp"],"created_at":"2024-11-06T05:24:33.123Z","updated_at":"2026-01-22T20:33:10.961Z","avatar_url":"https://github.com/mpds-io.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"Data-driven predictions: from crystal structure to physical properties and vice versa\n======\n\n[![DOI](https://zenodo.org/badge/110734326.svg)](https://zenodo.org/badge/latestdoi/110734326)\n[![arXiv](https://img.shields.io/badge/arXiv-1806.03553-b31b1b.svg)](https://arxiv.org/abs/1806.03553)\n\n![Materials simulations ab datum](https://raw.githubusercontent.com/mpds-io/mpds-ml-labs/master/crystallographer_mpds_cc_by_40.png \"Materials simulation ab datum\")\n\nThis is the repository for the work [\"Quantitative trends in 8 physical properties of 115000 inorganic compounds gained by machine learning\"](https://arxiv.org/abs/1806.03553) by Blokhin and Villars (**2018**).\n\n\nLive demos\n------\n\n[mpds.io/ml](https://mpds.io/ml) and [mpds.io/materials-design](https://mpds.io/materials-design)\n\n\nRationale\n------\n\nThis is the proof of concept, how a relatively unsophisticated statistical model (namely, _random forest regressor_) trained on the large MPDS dataset predicts a set of physical properties from the only crystalline structure. Similarly to _ab initio_, this method could be called _ab datum_. (Note however that the simulation of physical properties with a comparable precision normally takes days, weeks or even months, whereas the present method takes less than a second!) A crystal structure in either CIF or POSCAR format is required. The following physical properties are predicted:\n\n- isothermal bulk modulus\n- enthalpy of formation\n- heat capacity at constant pressure\n- melting temperature\n- Debye temperature\n- Seebeck coefficient\n- linear thermal expansion coefficient\n- electrical conductivity\n- thermal conductivity\n- band gap (or its absence, _i.e._ whether a crystal is conductor or insulator)\n\nFurther, a reverse task of predicting the possible crystalline structure from a set of given properties is solved. The suitable chemical elements are found, and the resulted structure is generated based on the MPDS prototype (if found).\n\n\nInstallation\n------\n\n```shell\napt-get install swig python3-dev\n```\n\n```shell\ngit clone REPO_ADDR\nvirtualenv --system-site-packages REPO_FOLDER\ncd REPO_FOLDER\n. bin/activate\npip install -r requirements.txt\n```\n\n\nPreparation for work\n------\n\nThe model is trained on the MPDS data using the MPDS API and the scripts `train_regressor.py` and `train_classifier.py`. Some subset of the full MPDS data is opened and possible to obtain using API for free (just login at the MPDS via [GitHub](https://mpds.io/github_oauth.html)). If the training is performed on the limited (_e.g._ opened) data subset, the scripts must be modified to query MPDS accordingly. The MPDS API returns an HTTP error code `402` if a user's request is authenticated, but not authorized. See a [full list of HTTP status codes](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes).\n\nThe code tries to use the settings exemplified in a template:\n\n```shell\ncp data/settings.ini.sample mpds_ml_labs.ini\n```\n\n\nArchitecture and usage\n------\n\nThe code can be used either as a *standalone command-line* application or as a *client-server* application.\n\nExamples of the *standalone command-line* architecture are the scripts `mpds_ml_labs/test_props_cmd.py` and `mpds_ml_labs/test_design_cmd.py`.\n\nIn the case of the *client-server* architecture, the client and the server communicate over HTTP using a simple API, and any client able to execute HTTP requests is supported, be it a `curl` command-line client, a Python script or the rich web-browser user interface. Examples of the Python scripts are `mpds_ml_labs/test_props_client.py` and `mpds_ml_labs/test_design_client.py`.\n\nServer part is a Flask app `mpds_ml_labs/app.py`. The simple HTML5 client apps `props.html` and `design.html`, supplied in the `webassets` folder, are served by a Flask app under `http://localhost:5000`. By default, to serve the requests the development Flask server is used. Therefore an _AS-IS_ deployment in an online environment without the suitable WSGI container is **highly discouraged**. For the production environments under the high load it is recommended to use something like [TensorFlow Serving](https://www.tensorflow.org/serving).\n\n\nUsed descriptor and model details\n------\n\nThe term _descriptor_ stands for the compact information-rich representation, allowing the convenient mathematical treatment of the encoded complex data (_i.e._ crystalline structure). Any crystalline structure is populated to a certain relatively big fixed volume of minimum one cubic nanometer. Then the descriptor is constructed using the periodic numbers of atoms and the lengths of their radius-vectors. The details are in the file `mpds_ml_labs/prediction.py`.\n\nAs a machine-learning model an ensemble of decision trees ([random forest regressor](http://scikit-learn.org/stable/modules/ensemble.html)) is used, as implemented in [scikit-learn](http://scikit-learn.org) Python machine-learning toolkit. The whole MPDS dataset can be used for training. To estimate the prediction quality of the _regressor_ model, the _mean absolute error_ and _R2 coefficient of determination_ is checked. To estimate the prediction quality of the binary _classifier_ model, the _fraction incorrect_ (_i.e._ the _error percentage_) is checked. The evaluation process is repeated at least 30 times to achieve a statistical reliability.\n\nGenerating the crystal structure from the physical properties is done as follows. The decision-tree properties predictions of nearly 115k distinct MPDS phases are used for the radius-based neighbor learning. This allows to extrapolate the possible chemical elements for almost any given combination of physical properties. The results of the neighbor learning are approximately 10M rows, stored in a Postgres table `ml_knn`:\n\n```sql\nCREATE TABLE ml_knn (\n    id  INT PRIMARY KEY,\n    z   SMALLINT NOT NULL,\n    y   SMALLINT NOT NULL,\n    x   SMALLINT NOT NULL,\n    k   SMALLINT NOT NULL,\n    w   SMALLINT NOT NULL,\n    m   SMALLINT NOT NULL,\n    d   SMALLINT NOT NULL,\n    t   SMALLINT NOT NULL,\n    i   SMALLINT NOT NULL,\n    o   SMALLINT NOT NULL,\n    els VARCHAR(19)\n);\nCREATE SEQUENCE ml_knn_id_seq START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1;\nALTER SEQUENCE ml_knn_id_seq OWNED BY ml_knn.id;\nALTER TABLE ONLY ml_knn ALTER COLUMN id SET DEFAULT nextval('ml_knn_id_seq'::regclass);\nCREATE INDEX prop_z ON ml_knn USING btree(z);\nCREATE INDEX prop_y ON ml_knn USING btree(y);\nCREATE INDEX prop_x ON ml_knn USING btree(x);\nCREATE INDEX prop_k ON ml_knn USING btree(k);\nCREATE INDEX prop_w ON ml_knn USING btree(w);\nCREATE INDEX prop_m ON ml_knn USING btree(m);\nCREATE INDEX prop_d ON ml_knn USING btree(d);\nCREATE INDEX prop_t ON ml_knn USING btree(t);\nCREATE INDEX prop_i ON ml_knn USING btree(i);\nCREATE INDEX prop_o ON ml_knn USING btree(o);\n```\n\nThe full contents of this table can be provided by request. The found elements matching the given property ranges are used to compile a crystal structure based on the available MPDS structure prototypes (via the MPDS API). See `mpds_ml_labs/test_design_cmd.py`.\n\n\nAPI\n------\n\nThese are examples of using the `curl` command-line client.\n\nFor the local server:\n\n```shell\ncurl -XPOST http://localhost:5000/predict -d \"structure=data_in_CIF_or_POSCAR\"\ncurl -XPOST http://localhost:5000/design -d \"numerics=ranges_of_values_of_8_properties_in_JSON\"\ncurl -XPOST http://localhost:5000/design -d 'numerics={\"z\":[5,119],\"y\":[-325,0],\"x\":[22,28],\"k\":[-18,114],\"w\":[0.5,3.5],\"m\":[958,1816],\"d\":[464,777],\"t\":[33,50],\"i\":[4,16],\"o\":[3,42]}'\n```\n\nFor the demonstration MPDS server:\n\n```shell\ncurl -XPOST https://labs.mpds.io/predict -d \"structure=data_in_CIF_or_POSCAR\"\ncurl -XPOST https://labs.mpds.io/design -d \"numerics=ranges_of_values_of_8_properties_in_JSON\"\ncurl -XPOST https://labs.mpds.io/design -d 'numerics={\"z\":[5,119],\"y\":[-325,0],\"x\":[22,28],\"k\":[-18,114],\"w\":[0.5,3.5],\"m\":[958,1816],\"d\":[464,777],\"t\":[33,50],\"i\":[4,16],\"o\":[3,42]}'\n```\n\n\nCredits\n------\n\nThis project is built on top of the open-source scientific software, such as:\n\n- [scikit-learn](http://scikit-learn.org)\n- [pandas](https://pandas.pydata.org)\n- [ASE](https://wiki.fysik.dtu.dk/ase)\n- [pycodcif](https://wiki.crystallography.net/cod-tools/CIF-parser)\n- [spglib](https://atztogo.github.io/spglib)\n- [cifplayer](https://tilde-lab.github.io/cifplayer)\n- [MPDS API client](https://developer.mpds.io)\n\n\nLicense\n------\n\n- The client and the server code: *LGPL-2.1+*\n- The machine-learning MPDS data generated as presented here: *CC BY 4.0*\n- The open part of the experimental MPDS data (10%): *CC BY 4.0*\n- The closed part of the experimental MPDS data (90%): *commercial*\n\n\nCitation\n------\n\n- Blokhin E, Villars P, Quantitative trends in physical properties of inorganic compounds via machine learning, [arXiv](https://arxiv.org/abs/1806.03553), **2018**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmpds-io%2Fmpds-ml-labs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmpds-io%2Fmpds-ml-labs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmpds-io%2Fmpds-ml-labs/lists"}