{"id":28183367,"url":"https://github.com/thegreatherrlebert/ionmob","last_synced_at":"2025-10-08T07:57:34.989Z","repository":{"id":68833379,"uuid":"403934608","full_name":"theGreatHerrLebert/ionmob","owner":"theGreatHerrLebert","description":"An open-source prediction framework for peptide ion collision cross section (CCS) values with python.","archived":false,"fork":false,"pushed_at":"2023-10-06T15:49:06.000Z","size":220460,"stargazers_count":15,"open_issues_count":0,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-09-04T21:48:59.074Z","etag":null,"topics":["ccs","collisional-cross-section","deep-learning","ion-mobility","ion-mobility-spectrometry","machine-learning","mass-spectrometry","proteomics","pypi","python-package"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/theGreatHerrLebert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-09-07T10:23:07.000Z","updated_at":"2025-08-07T14:55:51.000Z","dependencies_parsed_at":"2025-04-11T23:03:02.291Z","dependency_job_id":null,"html_url":"https://github.com/theGreatHerrLebert/ionmob","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/theGreatHerrLebert/ionmob","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theGreatHerrLebert%2Fionmob","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theGreatHerrLebert%2Fionmob/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theGreatHerrLebert%2Fionmob/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theGreatHerrLebert%2Fionmob/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/theGreatHerrLebert","download_url":"https://codeload.github.com/theGreatHerrLebert/ionmob/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theGreatHerrLebert%2Fionmob/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278909714,"owners_count":26066887,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-08T02:00:06.501Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ccs","collisional-cross-section","deep-learning","ion-mobility","ion-mobility-spectrometry","machine-learning","mass-spectrometry","proteomics","pypi","python-package"],"created_at":"2025-05-16T04:15:46.202Z","updated_at":"2025-10-08T07:57:34.957Z","avatar_url":"https://github.com/theGreatHerrLebert.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ionmob\n### A Framework for Predicting Collision Cross Section (CCS) Values of Peptide-Ions with Traditional and Deep Machine Learning Methods\n\n`ionmob` is a Python package designed to predict **CCS** values of peptides. Beyond offering several pre-trained regression models for this task, it provides a comprehensive pipeline that seamlessly integrates data preprocessing, model training, and CCS value inference. Models are built using up-to-date versions of either [TensorFlow](https://www.tensorflow.org/) or [scikit-learn](https://scikit-learn.org/stable/).\n\nWe encourage you to use, modify, or extend ionmob to suit your needs. It's freely available and open-source under the **GNU General Public License v3.0**. Feedback is always appreciated! Please let us know if you encounter any missing features, bugs, or if you'd like to contribute in any way.\n\nOur ionmob package is now featured in a publication in Bioinformatics. Access the fully open-access paper [here](https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btad486/7237255). For those interested in replicating our experiments, training models, or reproducing result plots from the paper, additional data and scripts not included in the ionmob package can be found on [Zenodo](https://zenodo.org/record/8091540).\n\n* [**TLDR**](#TLDR)\n* [**What is a peptide CCS value?**](#what-is-a-peptide-CCS-value)\n* [**Why do we measure CCS values of ions?**](#why-do-we-measure-CCS-values-of-ions)\n* [**Why would I want to predict CCS values of peptides in silico?**](#why-would-I-want-to-predict-CCS-values-of-peptides-in-silico)\n* [**Can I use ionmob if I am no machine learning expert?**](#can-I-use-ionmob-if-I-am-no-machine-learning-expert)\n* [**What can I do with ionmob if I am an experienced coder?**](#what-can-I-do-with-ionmob-if-I-am-an-experienced-coder)\n* [**Installation**](#installation)\n* [**A simple example of CCS prediction and performance evaluation with pre-trained models**](#a-simple-example-of-CCS-prediction-and-performance-evaluation-with-pre-trained-models)\n* [**Getting insight into driving factors of CCS**](#Getting-insight-into-driving-factors-of-CCS)\n* [**Alignments of in-house and external data**](#Alignments-of-in-house-and-external-data)\n* [**Implementing a custom deep CCS predictor**](#Implementing-a-custom-deep-CCS-predictor)\n* [**Cite ionmob**](#Cite-ionmob)\n\n---\n### TLDR\nTo simply get started, load our best performing predictor and infer CCS values on a dataset\nprovided by this repository:\n\n#### Inference on one of our provided datasets\n\n```python\nimport tensorflow as tf\nimport pandas as pd\nimport numpy as np\nfrom matplotlib import pyplot as plt\n\nfrom ionmob.utilities.tokenization import tokenizer_from_json\nfrom ionmob.preprocess.data import to_tf_dataset_inference\n\n# you will need to load the correct tokenizer to translate peptide sequences to tokens\ntokenizer = tokenizer_from_json('pretrained_models/tokenizers/tokenizer.json')\n\n# load the example_data\ndata = pd.read_parquet('example_data/Tenzer_unimod.parquet')\n\n# load the model\ndeepGRU = tf.keras.models.load_model('pretrained_models/GRUPredictor/')\n\n# create a tensorflow dataset from example_data\ntf_ds = to_tf_dataset_inference(mz=data['mz'],\n                                charge=data['charge'],\n                                sequences=[list(s) for s in data['sequence-tokenized']],\n                                tokenizer=tokenizer)\n\n# do inference\nccs_predicted, deep_residues = deepGRU.predict(tf_ds)\ndata['ccs_predicted'] = ccs_predicted\n```\n#### Inference on your own datasets\n\nInference of CCS values on custom data is already more involved as it requires you to tokenize your sequences before \nthey can be used by an `ionmob` predictor. Sequences presented to `ionmob` can be composed of all 20 amino acids and a \ngrowing number of modifications such as phosphorylation. Have a look at all known symbols:\n\n```python\nfrom ionmob.utilities.chemistry import VARIANT_DICT\n\nprint(VARIANT_DICT)\n\n{'L': ['L'], 'E': ['E'], 'S': ['S', 'S[UNIMOD:21]'], 'A': ['A'], 'V': ['V'], 'D': ['D'], 'G': ['G'],\n '\u003cEND\u003e': ['\u003cEND\u003e'], 'P': ['P'], '\u003cSTART\u003e': ['\u003cSTART\u003e', '\u003cSTART\u003e[UNIMOD:1]'], 'T': ['T', 'T[UNIMOD:21]'],\n 'I': ['I'], 'Q': ['Q'], 'K': ['K', 'K[UNIMOD:1]'], 'N': ['N'], 'R': ['R'], 'F': ['F'], 'H': ['H'],\n 'Y': ['Y', 'Y[UNIMOD:21]'], 'M': ['M', 'M[UNIMOD:35]'],\n 'W': ['W'], 'C': ['C', 'C[UNIMOD:312]', 'C[UNIMOD:4]'], 'C[UNIMOD:4]': ['C', 'C[UNIMOD:312]', 'C[UNIMOD:4]']}\n\n```\n\nUnmodified amino acids are just written as a singe character as capital letters. Modified sequences use the \n[UniMod convention](http://www.unimod.org/): A modification is noted like `[UNIMOD:X]`, where `X` is \nthe UniMod code for the respective modification. Read more about this convention [here](https://github.com/HUPO-PSI/ProForma).\nAlso, N termini are signified by a `\u003cSTART\u003e` token und C termini by an `\u003cEND\u003e` token. This additionally allows for \ntermini modification tokens as well as indication of read direction of peptide sequences.\n\nTranslating sequences to tokens from a given output file of PEAKS, DiaNN or MaxQuant is supported out-of-the-box:\n\n```python\nimport pandas as pd\nfrom ionmob.utilities.utility import preprocess_max_quant_sequence\n\nmq_data = pd.read_table('path/to/mq/evidence.txt', low_memory=False)\n\nmq_data['sequence-tokenized'] = mq_data.apply(lambda r: preprocess_max_quant_sequence(r['Modified sequence']), axis=1)\n```\nDepending on the software used for processing the raw-data, precursor mono-isotopic mz values might not be available\nin the output files. It is therefore possible to calculate them from a given tokenized sequence and charge state:\n\n```python\nfrom ionmob.utilities.chemistry import calculate_mz\n\nmq_data['mz'] = mq_data.apply(lambda r: calculate_mz(r['sequence-tokenized'], r['Charge']), axis=1)\n```\nNow, a dataset can be created for prediction:\n\n```python\nfrom ionmob.preprocess.data import to_tf_dataset_inference\nfrom ionmob.utilities.tokenization import tokenizer_from_json\n\ntokenizer = tokenizer_from_json('pretrained_models/tokenizers/tokenizer.json')\n\ntf_ds = to_tf_dataset_inference(mq_data['mz'], mq_data['Charge'], mq_data['sequence-tokenized'], tokenizer)\n```\n\n#### Calculate experiment specific shifts of CCS values\nA linear shift in calculated CCS values can often be observed between two experiments coming from different sources. You can correct\nfor this by calculating a shift factor that needs to be added to observed values. Optimally, use a set of high \nconfidence identifications that contain at least tokenized sequences, charges and CCS values. They can then be used\ntogether with one of the training datasets as reference:\n\n```python\nimport pandas as pd\nfrom ionmob.utilities.utility import get_ccs_shift\n\ntarget = pd.read_table('path/to/my/table.csv')\n\n# preprocess, select high confidence identifications, tokenize etc.\n\n# read a reference dataset predictor was trained on\nreference = pd.read_parquet('example_data/reference.parquet')\n\n# a shift factor is calculated from charge state 2, which has the lowest variance\nshift_factor = get_ccs_shift(target, reference)\n\n# optionally, apply shift to target dataset\ntarget['ccs_shifted'] = target.apply(lambda r: r['ccs'] + shift_factor, axis=1)\n```\n\n---\n### What is a peptide CCS value?\nThe rotationally-averaged collision cross section - **CCS** - is a scalar value that describes a physical property of an ion.\nIt can be directly linked to its ion mobility, meaning its interactive behaviour with respect to a charge-neutral gas.\nThe ion mobility is used as an additional separating dimension in high throughput mass spectrometry.\nIt supplements the measurements of retention times and mass-to-charge ratios and ultimately leads to improved peptide identification.\n\n---\n### Why do we measure CCS values of ions?\nThe CCS value of an ion is a coarse descriptor of its 3D structure.\nSince peptides are chains (strings) of amino acids, there exist permutations in nature that have exactly the same mass and chemical properties.\nDifferences in AA sequence will result in differences in the 3D structure though.\nDistinguishing between such peptides with conventional methods like e.g. LC-MS-MS is therefore challenging.\nFurthermore, post translational modifications (PTMs) might have only a small impact on an ion's mass but alter the functionality of a protein.\nSince both a permutation of sequence as well as PTMs have significant impact on 3D structure, one can use ion mobility separation to distinguish between them.\nCCS value calculation then gives us a measure how extensively their rotationally-averaged collision cross section differed.\n\n---\n### Why would I want to predict CCS values of peptides in silico?\nFirst, a predictor might give you insight into factors that drive ion mobility.\nThis information could then be used to optimize your laboratory workflows or uncover yet unknown relationships.\nSecond, the high reproducibility of measured CCS values in the lab make it an ideal candidate to increase confidence in peptide identifications from database searches.\nWe think, the recent triumph of ion mobility enhanced mass spectrometry paves the way for expressive predictors by providing previously unavailable amounts of training data!\n\n---\n### Can I use ionmob if I am no machine learning expert?\nDefinitely yes!\nWe implemented and pretrained models of different complexity that allow for in silico prediction of CCS values for peptide ions of different charge states out-of-the-box.\nThey are easily integratable into your existing proteomics workflows.\nAll you need is a little bit of python scripting experience.\nA short introduction can be found down below. \nAdditionally, you can have a look at our collection of example notebooks.\n\n---\n### What can I do with ionmob if I am an experienced coder?\nWe made sure that our framework provides a modular set of tools for more experienced coders that want to implement their own models, training strategies or data preprocessing pipelines.\nHave a look at our example notebooks for advanced workflow implementation.\nFeel also free to contribute any optimizations, models or ideas that you come up with.\nThis will ultimately help to push prediction accuracy to a point where it provides a huge benefit for rescoring of peptide identifications!\n\n---\n### Installation\nWe recommend to install ```ionmob``` into a separate [python virtual environment](https://docs.python.org/3/tutorial/venv.html). Once activated, you can install the ionmob package into it as follows: \n```\ngit clone https://github.com/theGreatHerrLebert/ionmob.git\ncd ionmob\npip install -e .\n```\n\n---\n### A simple example of CCS prediction and performance evaluation with pre-trained models\nLet us assume that you want to have a look at prediction performance for two different ```ionmob``` predictors on data of peptide identifications that came from some source.\nFor ```ionmob``` models, you should at least have the following information per peptide: **mz, charge, sequence, ccs**.\nCCS values are optional in the general case but are required if you want to compare CCS predictions to CCS measurements.\nWe will demonstrate how to do this with one of our provided example datasets:\n\n```python\nimport pandas as pd\n\n# read example_data and a predictor\ndata = pd.read_parquet('example_data/Tenzer.parquet')\ndata.head()\n```\n\nThis is what the data looks like:\n\n|    |       mz |   charge | sequence-tokenized                                                |     ccs |      rt | name           |\n|---:|---------:|---------:|:------------------------------------------------------------------|--------:|--------:|:---------------|\n|  2 |  478.78  |        2 | ['\\\u003cSTART\u003e' 'A' 'A' 'A' 'A' 'A' 'A' 'A' 'L' 'Q' 'A' 'K' '\\\u003cEND\u003e'] | 351.073 | 14.1374 | Tenzer-tryptic |\n|  3 |  514.317 |        2 | ['\u003c\\START\u003e' 'A' 'A' 'A' 'A' 'A' 'A' 'T' 'V' 'L' 'L' 'R' '\u003c\\END\u003e'] | 360.949 | 38.812  | Tenzer-tryptic |\n|  4 |  472.251 |        2 | ['\u003c\\START\u003e' 'A' 'A' 'A' 'A' 'A' 'D' 'L' 'A' 'N' 'R' '\u003c\\END\u003e']     | 320.562 | 14.467  | Tenzer-tryptic |\n\n\nLet's compare accuracy for two predictors.\nOne that only does a zero-information square-root fit on ion mz values and a deep model that also uses information on peptide sequences.\nThe latter also needs a so-called [tokenizer](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer): a tool that translates sequence symbols into a numerical representation.\nIt is specific for a pretrained model and therefore needs to be loaded as well:\n\n```python\nimport tensorflow as tf\nfrom matplotlib import pyplot as plt\nfrom ionmob.preprocess.data import sqrt_model_dataset\n\n# read the pretrained predictors\nsqrtModel = tf.keras.models.load_model('pretrained_models/SqrtModel')\ngruModel = tf.keras.models.load_model('pretrained_models/GRUPredictor/')\n\n# read tokenizer for deep model\ntokenizer = tokenizer_from_json('pretrained_models/tokenizer.json')\n\n# create dataset for sqrt prediction and predict\ntensorflow_ds_sqrt = sqrt_model_dataset(data.mz, data.charge, data.ccs).batch(1024)\ndata['ccs_predicted_s'] = sqrtModel.predict(tensorflow_ds_sqrt)\n\n# create dataset for deep prediction and predict\ntensorflow_ds_deep = get_tf_dataset(data.mz, data.charge, data.sequence, data.ccs, tokenizer,\n                                    drop_sequence_ends=False, add_charge=True).batch(1024)\nccs_predicted_gru, _ = gruModel.predict(tensorflow_ds_deep)\ndata['ccs_predicted_g'] = ccs_predicted_gru\n```\n\nLet's compare prediction accuracies:\n```python\nimport numpy as np\n\n# define error functions\ndef mean_abs_error(ccs, ccs_pred):\n    return np.round(np.mean([np.abs(x[0] - x[1]) for x in np.c_[ccs, ccs_pred]]), 2)\n\ndef mean_perc_error(ccs, ccs_pred):\n    return np.round(np.mean([np.abs((x[0] - x[1]) / x[0]) * 100 for x in np.c_[ccs, ccs_pred]]), 2)\n\n# show results\nprint(f\"sqrt mean absolute percent error: {mean_perc_error(data.ccs, data.ccs_predicted_s)}\")\nprint(f\"gru mean absolute percent error : {mean_perc_error(data.ccs, data.ccs_predicted_g)}\")\nprint(\"\")\nprint(f\"sqrt mean absolute error        : {mean_abs_error(data.ccs, data.ccs_predicted_s)}\")\nprint(f\"gru mean absolute error         : {mean_abs_error(data.ccs, data.ccs_predicted_g)}\")\n```\n\nThis then gives us CCS accuracies of:\n\n```python\nsqrt mean absolute percent error: 2.58\ngru  mean absolute percent error: 1.84\n\nsqrt mean absolute error        : 12.69\ngru  mean absolute error        : 9.04\n```\n\nFinally, let's visualize the predictions compared to the CCS measurements:\n\n```python\nfrom matplotlib import pyplot as plt\n\n# visualize the charge states in different colors\ncolor_dict = {2:'red', 3:'orange', 4:'lightgreen'}\n\n# create the plot\nfig, (ax1, ax2) = plt.subplots(nrows=1, ncols=2, figsize=(12,4), dpi=200, sharey=True, sharex=True)\n\nax1.set_title('sqrt fit prediction')\nax1.set_ylabel('CCS')\nax1.set_xlabel('MZ')\nax2.set_xlabel('MZ')\nax2.set_title('deep prediction')\n\nax1.scatter(data.mz, data.ccs, s=10, alpha=.5, label='ground truth')\nax1.scatter(data.mz, data.ccs_predicted_s, s=10, alpha=.5, c=[color_dict[x] for x in data.charge],\n            label='prediction')\nax2.scatter(data.mz, data.ccs, s=10, alpha=.5, label='ground truth')\nax2.scatter(data.mz, data.ccs_predicted_g, s=10, alpha=.2, c=[color_dict[x] for x in data.charge],\n            label='prediction')\nax1.legend()\nax2.legend()\nfig.show()\n```\n\nThis code will result in the following plot:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/images/sqrt_model.png\" width=\"900\" title=\"prediction vs ground truth\"\u003e\n\u003c/p\u003e\n\nYou can also try this yourself by cloning this repository and running [this notebook](/notebook/CheckAccuracy.ipynb).\n\n---\n### Getting insight into driving factors of CCS\nRecent papers that worked on CCS prediction such as Chang et al.[^fn2] and Meier et al.[^fn1] identified factors that drive differences in ion mobility.\nBy using an in silico digest of the human proteome, we can estimate the impact of two of them, namely the [GRAVY score](https://www.bioinformatics.org/sms2/protein_gravy.html) and helicality of peptides. Our modelling approach will look like this: first an initial CCS value is calculated soley on an ions mass and charge. This is done using the simple formula below (caution, equations might be barely visible if your browser uses darkmode):\n\n\u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\mathrm{CCS}_{\\mathrm{init}}(\\mathrm{mz}, c)=s_c\\times\\sqrt{\\mathrm{mz}} %2B b_c\"\u003e\n\nWhere a slope \u003cimg src=\"https://render.githubusercontent.com/render/math?math=s_c\"\u003e  and an intercept \u003cimg src=\"https://render.githubusercontent.com/render/math?math=b_c\"\u003e are fit separately for each modeled charge state \u003cimg src=\"https://render.githubusercontent.com/render/math?math=c\"\u003e.\nThe reason why ion-mobility does add an additional dimension of separation is the fact that an ion's CCS value does not always lie on that line.\nIf it did, CCS would be perfectly correlated with mz and therefore add no new information.\nWe can improve our inital CCS prediction modell by also predicting the residues with respect to the square root fit, meaning the vertical difference to our initial value.\nThese residues could be provided by any predictor but let's use our best performing model: the GRU-based predictor.\nIt uses deep [GRU-units](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) that can take into account sequence specific higher-order information derived from training data.\nWe will expand our mathematical formulation of the problem as follows:\n\n\u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\mathrm{CCS}_{\\mathrm{final}}(\\mathrm{mz}, c, s \\vert M) = \\mathrm{CCS}_{\\mathrm{init}}(\\mathrm{mz}, c) %2B M(s, \\theta)\"\u003e\n\nHere, a regressor \u003cimg src=\"https://render.githubusercontent.com/render/math?math=M\"\u003e (GRU-units) with parameter set \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\theta\"\u003e was fit to further lower the mean absolut error (MAE) of predicted CCS values compared to the experimentally observed ones.\nFor convenience, this predictor does not only return the final predicted ccs value but also the residue with respect to the initial fit, giving us an easy way to link specific features of a given sequence to its impact on ion mobility.\nAn implementation with ```ionmob``` to derive this could look like this:\n\n```python\nimport pandas as pd\nimport numpy as np\nimport tensorflow as tf\n\nfrom ionmob.utilities.utility import get_gravy_score, get_helix_score\nfrom ionmob.utilities.tokenization import tokenizer_from_json\nfrom ionmob.preprocess.data import get_tf_dataset, sqrt_model_dataset\n\n# read in silico digested human proteome to gain insight into predictors behaviour\ndata = pd.read_hdf('Synthetic.h5').sample(frac=0.25)\n\n# read predictors and tokenizer\ngruModel = tf.keras.models.load_model('pretrained_models/GRUPredictor/')\nsqrtModel = tf.keras.models.load_model('pretrained_models/SqrtModel/')\ntokenizer = tokenizer_from_json('pretrained_models/tokenizer.json')\n\n# generate tensorflow datasets for prediction\ntensorflow_ds_sqrt = sqrt_model_dataset(data.mz, data.charge, None).batch(1024)\ntensorflow_ds_deep = get_tf_dataset(data.mz, data.charge, data.sequence, None, tokenizer,\n                                    drop_sequence_ends=False, add_charge=True).batch(1024)\n\n# predict with sqrt-fit\nccs_predicted_sqrt = sqrtModel.predict(tensorflow_ds_sqrt)\n\n# predict with deep fit\nccs_predicted_gru, deep_part = gruModel.predict(tensorflow_ds_deep)\n\n# append predictions to dataframe\ndata['ccs_predicted_gru'] = ccs_predicted_gru\ndata['ccs_predicted_sqrt'] = ccs_predicted_sqrt\ndata['ccs_predicted_deep'] = deep_part\n\n# create normalized value of deep increase or decrease prediction of CCS\ndata['deep_normalized'] = data.ccs_predicted_deep / np.sqrt(data.mz.values)\n\n# calculate gravy and helix scores for each sequence\ngravy = [get_gravy_score(s, normalize=False) for s in data.sequence]\nhelix = [get_helix_score(s) for s in data.sequence]\n\n# append calculated values to dataframe\ndata['gravy'] = gravy\ndata['helix'] = helix\n\n# select a single charge state to deconvolce differences between charges\ncharge_2 = data[data['charge'] == 2]\n```\n\nWe are now ready to have a look at how both GRAVY score and helix score of a given peptide are correlated with an increase or decrease of the deep predicted CCS with respect to the initial guess. Since the impact is not equal along the mz axis, the deep residue value was normalized by dividing it by the square-root mz value of its ion. We will calculate the pearson correlation to have some objective measure how strong they are correlated:\n\n```python\nfrom scipy.stats import pearsonr\n\n# extract values to correlate\nx = charge_2.deep_normalized.values\ny_gravy = charge_2.gravy.values\ny_helix = charge_2.helix.values\n\nprint('Gravy Pearson:', np.round(pearsonr(x, y_gravy), 2))\nprint('Helix Pearson:', np.round(pearsonr(x, y_helix), 2))\n```\nThis gives us pearson correlation and p values for both gravy and helicality analysis:\n\n```python\nGravy Pearson: [0.49 0.  ]\nHelix Pearson: [0.52 0.  ]\n```\n\nOnce again, let's visualize this to get a better feel for what the numbers are telling us:\n\n```python\nfrom sklearn.linear_model import LinearRegression\nfrom mpl_toolkits.axes_grid1 import make_axes_locatable\n\ndef line(x, a, b):\n    return x * a + b\n\nreg_gravy = LinearRegression().fit(np.expand_dims(x, -1), np.expand_dims(y_gravy, -1))\nreg_helix = LinearRegression().fit(np.expand_dims(x, -1), np.expand_dims(y_helix, -1))\n\ny_line_gravy = [line(x, reg_gravy.coef_, reg_gravy.intercept_) for x in charge_2.deep_normalized.values]\ny_line_helix = [line(x, reg_helix.coef_, reg_helix.intercept_) for x in charge_2.deep_normalized.values]\n\n# create the plot\nfig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(nrows=2, ncols=2, figsize=(16,12), dpi=200)\n\nax1.set_title('linear correlation helicality, mobility')\nax1.set_ylabel('helix score')\nax1.set_xlabel('relative mobility trend')\nax2.set_xlabel('MZ')\nax2.set_title('Deep vs Sqrt prediction')\n\nim1 = ax1.scatter(charge_2.deep_normalized, charge_2.helix, c=charge_2.helix, alpha=.3, s=10, label='example_data points')\nim1 = ax1.scatter(charge_2.deep_normalized, y_line_helix, s=10, c='red', label='linear trend')\n\nim2 = ax2.scatter(charge_2.mz, charge_2.ccs_predicted_gru, s=10, c=charge_2.helix - np.mean(data.gravy), alpha=.3, label='example_data points')\nim2 = ax2.scatter(charge_2.mz, charge_2.ccs_predicted_sqrt, s=2, c='red', alpha=.3, label='sqrt prediction')\nax1.legend()\nax2.legend()\n\ndivider = make_axes_locatable(ax2)\ncax = divider.append_axes('right', size='2%', pad=0.05)\ncbar = fig.colorbar(im1, cax=cax, orientation='vertical', ticks=[0, 0.5, 1])\ncbar.ax.set_yticklabels(['0', '0.5', '1'])\n\nax3.set_title('linear correlation gravy, mobility')\nax3.set_ylabel('gravy score')\nax3.set_xlabel('relative mobility trend')\nax4.set_xlabel('MZ')\nax4.set_title('Deep vs Sqrt prediction')\n\nim3 = ax3.scatter(charge_2.deep_normalized, charge_2.gravy, c=charge_2.gravy, alpha=.3, s=10, label='example_data points')\nim3 = ax3.scatter(charge_2.deep_normalized, y_line_gravy, s=10, c='red', label='linear trend')\n\nim4 = ax4.scatter(charge_2.mz, charge_2.ccs_predicted_gru, s=10, c=charge_2.gravy, alpha=.3, label='example_data points')\nim4 = ax4.scatter(charge_2.mz, charge_2.ccs_predicted_sqrt, s=2, c='red', alpha=.3, label='sqrt prediction')\nax3.legend()\nax4.legend()\n\ndivider = make_axes_locatable(ax4)\ncax = divider.append_axes('right', size='2%', pad=0.05)\ncbar = fig.colorbar(im3, cax=cax, orientation='vertical', ticks=[0, 0.5, 1])\ncbar.ax.set_yticklabels(['\u003c -4', '0', '\u003e 4'])\n\nfig.show()\n```\nThis code then creates:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/images/gravy_helix_linear_correlation.png\" width=\"900\" title=\"prediction vs ground truth\"\u003e\n\u003c/p\u003e\n\nAs we can observe, our predictor is able to reproduce findings that were already postulated by Chang et al. as well as Meier et al.: Higher GRAVY and helicality values indeed lead to higher CCS values (at least with respect to our trained predictor). \nThis correlation is by no means perfect, but it lies in the nature of complex interactions that lead to a peptide's 3D structure that they cannot easily be modelled by a simple set of descriptors. \nUltimately, this is why a complex function modelling technique like Deep Learning can add something new!\nImplement your own ideas to uncover driving factors like amino acid counts or specific amino acid positions by altering [this notebook](/notebook/MobilityDrivingFactors.ipynb).\n\n---\n### Alignments of in-house and external data\n\n```python\nimport pandas as pd\nfrom ionmob.preprocess import experiment as exp\nfrom ionmob.preprocess import alignment as alig\n\ndata_dir = \"example_data/raw_data/\"\nfname = \"M210115_00[1,2,3]_HeLa_grad110_ramp100__evidence.txt\"\n\npath = data_dir + fname\ndf = pd.read_csv(path, sep=\"\\t\")\n```\n\n2 ways to construct an Experiment object are provided.\n1st method: isolate columns needed from df (see below) as numpy arrays and pass to constructor\n\n```python\nseq, charge, ccs, intensity, mz, raw_file, evidence_id = df[\"Modified sequence\"].values, df[\"Charge\"].values, df[\n    \"CCS\"].values, df[\"Intensity\"].values, df[\"m/z\"].values, df[\"Raw file\"].values, df[\"id\"].values\n# give your experiment instance a name. ideally a short uinique version of fname\nex_name = \"HeLa_grad110\"\nex1 = exp.Experiment(ex_name, seq, charge, ccs,\n                     intensity, mz, raw_file, evidence_id)\n```\n\nor rather like this. be aware of the order of args!\n```python\nargs = df[\"Modified sequence\"].values, df[\"Charge\"].values, df[\"CCS\"].values, df[\n    \"Intensity\"].values, df[\"m/z\"].values, df[\"Raw file\"].values, df[\"id\"].values\nex1 = exp.Experiment(ex_name, *args)\n```\n\n2nd method: if you are sure that the output table contains the columns \"Modified sequence\",\n\"Charge\", \"CCS\", \"Intensity\", \"m/z\", \"Raw file\", \"id\", \"Mass\", \"Number of isotopic peaks\",\n\"Retention time\", \"Retention length\" ( which is usually the case for MaxQuant evidence.txt),\nthen you can also use this method\n\n```python\nex1 = exp.Experiment.from_MaxQuant_DataFrame(df, \"HeLa_grad110\")\n```\n\naccess the name and data of Experiment like this\n```python\nprint(\"name of your experiment: \", ex1.name)\nprint(\"example_data of your experiment: \", ex1.data)\n```\nRegardless of the initialization method the provided data is cleaned of NaN values in any of the essential columns and of the singly charged ions. \nFurthermore, the entries in the .data attribute are aggregated upon initialization of duplicate features (duplicates of sequence-charge-ccs entries), making those unique.\nAdditionally, entries that matched with reversed sequences of the decoy database (depicted as '+' entry in 'reverse' column) are removed.\n\nSince the .data attribute itself is a pd.DataFrame you can use the pandas library to work on it or isolate information from\n```python\nex1.data.loc[ex1.data.charge == 2]\n```\n\n\nto further aggregate rows and thereby getting rid of possible feature divergence, assign a modality class to each feature\n```python\nex2 = ex1.assign_modalities()\n```\nfrom this point on you can proceed with the inter-experimental CCS alignment of experiment\ndata aquired by the same device\n\n```python\ndata_dir = \"example_data/raw_data/\"\nfile_names = [\"M210115_00[1,2,3]_HeLa_grad110_ramp100__evidence.txt\",\n              \"M210115_00[4,5,6]_HeLa_grad47_ramp100__evidence.txt\",\n              \"M210115_00[7,8,9]_HeLa_grad20_ramp100__evidence.txt\"]\nexp_names = [\"HeLa_grad110\", \"HeLa_grad47\", \"HeLa_grad20\"]\npaths = [data_dir + fname for fname in file_names]\ndfs = [pd.read_csv(path, sep=\"\\t\") for path in paths]\nexs = [exp.Experiment.from_MaxQuant_DataFrame(\n    df, exp_name) for exp_name, df in zip(exp_names, dfs)]\nexs = [ex.assign_modalities() for ex in exs]\n\n# perform the ccs alignment of the experiments to each other\naligned_exs = alig.align_experiments(exs)\n# merge the aligned\naligned_ex = alig.merge_experiments(aligned_exs, \"our_experiments\")\n```\nif you want to expand your aquired data you can align a dataset aquired by another lab to the first one\nfirst read and intrinsically align the experiments of the other dataset like you did above\n```python\ndata_dir2 = \"example_data/mann_data/\"\nfile_names2 = [\"Results_evidence_mann_Drosophila.txt\",\n               \"Results_evidence_mann_HeLaTryp.txt\",\n               \"Results_evidence_mann_Celegans.txt\"]\nexp_names2 = [\"mann_Drosophila\", \"mann_HeLaTryp\", \"mann_Celegans\"]\npaths2 = [data_dir2 + fname for fname in file_names2]\ndfs2 = [pd.read_csv(path, sep=\"\\t\") for path in paths]\nexs2 = [exp.Experiment.from_MaxQuant_DataFrame(\n    df, exp_name) for exp_name, df in zip(exp_names2, dfs2)]\nexs2 = [ex.assign_modalities() for ex in exs2]\n\naligned_exs2 = alig.align_experiments(exs2)\n\naligned_ex2 = alig.merge_experiments(aligned_exs2, \"mann_experiments\")\n```\n\nin order to merge experiments from 2 different labs you firstly have to perform a linear shift on the data (in this case a copy of aligned_ex2 is returned with the additional column \"shifted_ccs\")\n```python\naligned_ex2_mean_shifted = alig.apply_mean_shift(aligned_ex, aligned_ex2)\n```\nat this point you can compare the original ccs values to the shifted ones within aligned_ex2_mean_shifted by accessing aligned_ex2_mean_shifted.data.ccs and aligned_ex2_mean_shifted.data.shifted_ccs\n\nif you are satisified with the shifted data you can proceed with adopting the shifted ccs values and then merge them with the previous method\n```python\naligned_ex2_mean_shifted = alig.adopt_shifted_ccs(aligned_ex2_mean_shifted)\n\nexs_of_labs = [aligned_ex, aligned_ex2_mean_shifted]\nbig_dataset = alig.merge_experiments(exs_of_labs, \"our_and_mann_dataset\")\n```\n\n---\n### Implementing a custom deep CCS predictor\nSay you come up with your very own idea for a deep CCS predictor architecture and want to build on top of ```ionmob```.\nIt is recomended that you have a NVIDIA CUDA enabled GPU with cuDNN bianries available in your working environment,\notherwise training may take quite some time.\nWe  will assume that a dataset for training was already generated, including all necesarry steps for preprocessing.\nFor this demonstration, we can use ```ionmob``` datasets. \nLet's use sets from different sources for training, validation and test.\nThis way, we make sure that we do not overestimate model performace.\nWe will start our model implementation by fitting a tokenizer.\n\n```python\nimport pandas as pd\nimport numpy as np\nimport tensorflow as tf\nimport os\nfrom datetime import datetime\n\nimport os\n\n# suppress CUDA specific logs \nos.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'\n\ngpus = tf.config.experimental.list_physical_devices('GPU')\n\ntf.config.experimental.set_virtual_device_configuration(gpus[0],\n                                                        [tf.config.experimental.VirtualDeviceConfiguration(\n                                                            memory_limit=2048)])\n\nfrom matplotlib import pyplot as plt\nfrom ionmob.alignment.experiment import Experiment\n\nfrom ionmob.models.deep_models import ProjectToInitialSqrtCCS\nfrom ionmob.preprocess.data import get_tf_dataset\nfrom ionmob.utilities.utility import get_sqrt_slopes_and_intercepts, sequence_to_tokens, sequence_with_charge,\n\nfit_tokenizer\n\ndata_train = pd.read_hdf('example_data/Meier.h5')\ndata_valid = pd.read_hdf('example_data/Tenzer.h5')\ndata_test = pd.read_hdf('example_data/Chang.h5')\n\n# tokenize sequences \nseq_tokenized = [sequence_to_tokens(s, drop_ends=True) for s in data_train.sequence.values]\n# fit a tokenizer\ntokenizer = fit_tokenizer(seq_tokenized)\n# have a look at tokens\n```\n\nThe tokenizer now knows 41 tokens, 20 of which are amino acids and 21 are PTMs.\n\nIt has proven to be a very efficient way to build on top of a simple square-root fit to help a deep predictor reach high accuracy as well as fast convergence. \n```ionmob``` implements its own layer that is able to project all charge states at the same time, making it very convenient to add it to your own predictor.\nIt is done in two steps: First, fit slopes and intercepts for the initial prediction separately. \nSecond, use the gained values to initialize a first projection layer.\n```ionmob``` makes use of charge state one-hot encoding to gate the prediction based on a given charge state.\nIf you are interested in the intrinsics, [have a look at the implementation](https://github.com/theGreatHerrLebert/ionmob/blob/8f9378c51149d9e1df89fc4550baeebed2176a22/ionmob/models/deep_models.py#L20).\n\n```python\nslopes, intercepts = get_sqrt_slopes_and_intercepts(data_train.mz, data_train.charge, data_train.ccs)\ninitial_layer = ProjectToInitialSqrtCCS(slopes, intercepts)\n\n# just make sure that everything worked by testing the projection\ninitial_ccs = initial_layer([np.expand_dims(data_train.mz, 1), tf.one_hot(data_train.charge - 1, 4)]).numpy()\n\n# visualize to make sure all went as intended\nplt.figure(figsize=(8, 4), dpi=120)\nplt.scatter(data_train.mz, initial_ccs, s=10, label='sqrt projection')\nplt.xlabel('Mz')\nplt.ylabel('CCS')\nplt.legend()\nplt.show()\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/images/sqrt_fit.png\" width=\"500\" title=\"sqrt fit\"\u003e\n\u003c/p\u003e\n\nThe most flexible way to implement a new predictor is to subclass a [tensorflow module or keras model](https://www.tensorflow.org/guide/keras/custom_layers_and_models). \nWe will do the latter, as it is the prominent way to generate new predictors for ```ionmob```. \nLet's set up a predictor that uses 1D convolutions to extract additional information from the sequence of an ion. \nAll layers that should be part of the model are defined in the constructor, the execution is defined by specifying the call method.\n\n```python\nclass ConvolutionalCCSPredictor(tf.keras.models.Model):\n    \n    def __init__(self, slopes, intercepts, num_tokens=41, seq_len=50):\n        super(ConvolutionalCCSPredictor, self).__init__()\n        # the inital sqrt projection\n        self.initial = ProjectToInitialSqrtCCS(slopes, intercepts)\n        \n        # the deep sequence processor\n        self.embedding = tf.keras.layers.Embedding(input_dim=num_tokens + 1, output_dim=128, input_length=seq_len)\n        self.conv1d = tf.keras.layers.Conv1D(filters=32, kernel_size=8, activation='relu')\n        self.mp1d = tf.keras.layers.MaxPool1D(pool_size=2)\n        self.conv1d_2 = tf.keras.layers.Conv1D(filters=64, kernel_size=8, activation='relu')\n        \n        # the deep regression tail\n        self.dense = tf.keras.layers.Dense(128, activation='relu')\n        self.dropout = tf.keras.layers.Dropout(0.5)\n        self.dense_2 = tf.keras.layers.Dense(64, activation='relu')\n        self.out = tf.keras.layers.Dense(1, activation=None)\n\n    def call(self, inputs):\n        # read inputs\n        mz, charge, sequence, _, _ = inputs\n        \n        # calculate sequence part\n        deep = self.conv1d_2(self.mp1d(self.conv1d(self.embedding(sequence))))\n        \n        # concat with mz and charge\n        concat = tf.keras.layers.Concatenate()([tf.keras.layers.Flatten()(deep), tf.sqrt(mz), charge])\n        \n        # deep regression\n        dense = self.dense_2(self.dropout(self.dense(concat)))\n        \n        # output is sqrt-fit + deep-regression\n        return self.initial([mz, charge]) + self.out(dense)\n```\n\nCallbacks are a convenient way to further automate your training procedure. \nWe will use two different callbacks that observe model performance on validation data.\nThe first one is a learning rate reducer: Should the loss not go down after three consecutive epochs on the validation set, the reducer is going to reduce the learning rate by an order of magnitude.\nIf there is still no improvement on performance, the early stopper will stop the training procedure after another 2 epochs.\n\n```python\nearly_stopper = tf.keras.callbacks.EarlyStopping(\n    monitor='val_loss',\n    patience=5\n)\n\nreduce_lr = tf.keras.callbacks.ReduceLROnPlateau(\n    monitor='val_loss', \n    factor=1e-1,\n    patience=2,\n    monde='auto',\n    min_delta=1e-5,\n    cooldown=0,\n    min_lr=1e-7\n)\n\ncbs = [early_stopper, reduce_lr]\n```\n\nWe are now ready to instanciate our predictor, build it and then compile it with a desired objective function and optimizer. \nThe model's summary tells us that it has a total of 178,785 trainable parameters.\n\n```python\n# create a recurrent predictor\nmodel = ConvolutionalCCSPredictor(slopes, intercepts)\n\n# set input shapes: mz, charge_one_hot, max_seq_len, helix_score, gravy_score\nmodel.build([(None, 1), (None, 4), (None, 50), (None, 1), (None, 1)])\n\nmodel.compile(loss=tf.keras.losses.MeanAbsoluteError(),\n              optimizer=tf.keras.optimizers.Adam(1e-2), metrics=['mae'])\n\ntf_train = get_tf_dataset(data_train.mz, data_train.charge, data_train.sequence, \n                          data_train.ccs, tokenizer, \n                          drop_sequence_ends=True, add_charge=False).shuffle(int(1e7)).batch(1024)\n\ntf_valid = get_tf_dataset(data_valid.mz, data_valid.charge, data_valid.sequence, \n                          data_valid.ccs, tokenizer, \n                          drop_sequence_ends=True, add_charge=False).shuffle(int(1e7)).batch(1024)\n\ntf_test = get_tf_dataset(data_test.mz, data_test.charge, data_test.sequence, \n                          data_test.ccs, tokenizer, drop_sequence_ends=True, add_charge=False).batch(1024)\n\nhistory = model.fit(tf_train, validation_data=tf_valid, \n                    epochs=50, verbose=False, callbacks=cbs)\n\n# plot training and validation loss \nplt.figure(figsize=(8, 4), dpi=120)\nplt.plot(history.history['loss'], label='training')\nplt.plot(history.history['val_loss'], label='validation')\nplt.xlabel('epoch')\nplt.ylabel('loss')\nplt.legend()\nplt.show()\n```\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/images/loss_train_valid.png\" width=\"500\" title=\"training and validation loss\"\u003e\n\u003c/p\u003e\n\nAs we can see from the plot above, loss quickly stops to improve on validation data while it is still falling on training data. The reduction of the learning rate is clearly visible after epoch 9. We can now have a look at test performance and report our CCS prediction accuracy.\n\n```pytho\nmodel.evaluate(tf_test)\n\n4/4 [==============================] - 0s 16ms/step - loss: 11.5374 - mae: 11.5374\n\n[11.537385940551758, 11.537385940551758]\n```\nIt is arround 11.5. Not too bad compared to the naive approach which gave us a value of arround 13. Want to try it yourself? Use [this notebook](notebook/DeepModelTraining.ipynb).\n\n# Cite ionmob\nIf you use `ionmob` for your own work, we only ask you to give credit by citing:\n\n```\n@article{10.1093/bioinformatics/btad486,\n    author = {Teschner, David and Gomez-Zepeda, David and Declercq, Arthur and Łącki, Mateusz K and Avci, Seymen and Bob, Konstantin and Distler, Ute and Michna, Thomas and Martens, Lennart and Tenzer, Stefan and Hildebrandt, Andreas},\n    title = \"{Ionmob: A Python Package for Prediction of Peptide Collisional Cross-Section Values}\",\n    journal = {Bioinformatics},\n    pages = {btad486},\n    year = {2023},\n    month = {08},\n    issn = {1367-4811},\n    doi = {10.1093/bioinformatics/btad486},\n    url = {https://doi.org/10.1093/bioinformatics/btad486},\n    eprint = {https://academic.oup.com/bioinformatics/advance-article-pdf/doi/10.1093/bioinformatics/btad486/51038853/btad486.pdf},\n}\n```\n\n[^fn1]: Deep learning the collisional cross-sections of the peptide universe from a million experimental values. Nat Commun, 2021. https://doi.org/10.1038/s41467-021-21352-8\n[^fn2]: Sequence-Specific Model for Predicting Peptide Collision Cross Section Values in Proteomic Ion Mobility Spectrometry. Journal of Proteome Research, 2021. https://doi.org/10.1021/acs.jproteome.1c00185\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthegreatherrlebert%2Fionmob","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthegreatherrlebert%2Fionmob","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthegreatherrlebert%2Fionmob/lists"}