{"id":15547680,"url":"https://github.com/lucaangioloni/proteinsecondarystructure-cnn","last_synced_at":"2025-04-14T19:44:28.993Z","repository":{"id":37602023,"uuid":"115259827","full_name":"LucaAngioloni/ProteinSecondaryStructure-CNN","owner":"LucaAngioloni","description":"Protein Secondary Structure predictor using Convolutional Neural Networks","archived":false,"fork":false,"pushed_at":"2023-07-15T11:02:13.000Z","size":46087,"stargazers_count":114,"open_issues_count":2,"forks_count":31,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-28T08:11:09.642Z","etag":null,"topics":["chain","cnn","cullpdb","deep-learning","keras","machine-learning","pdb","protein-prediction","protein-structure","structure-prediction"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LucaAngioloni.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-12-24T11:55:30.000Z","updated_at":"2025-03-21T16:06:59.000Z","dependencies_parsed_at":"2023-01-21T12:49:11.999Z","dependency_job_id":"f160e439-b122-4dc1-ab5d-787be05990e2","html_url":"https://github.com/LucaAngioloni/ProteinSecondaryStructure-CNN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LucaAngioloni%2FProteinSecondaryStructure-CNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LucaAngioloni%2FProteinSecondaryStructure-CNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LucaAngioloni%2FProteinSecondaryStructure-CNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LucaAngioloni%2FProteinSecondaryStructure-CNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LucaAngioloni","download_url":"https://codeload.github.com/LucaAngioloni/ProteinSecondaryStructure-CNN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248949982,"owners_count":21188183,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chain","cnn","cullpdb","deep-learning","keras","machine-learning","pdb","protein-prediction","protein-structure","structure-prediction"],"created_at":"2024-10-02T13:10:01.880Z","updated_at":"2025-04-14T19:44:28.970Z","avatar_url":"https://github.com/LucaAngioloni.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/protein-secondary-structure-prediction-using-2/protein-secondary-structure-prediction-on-1)](https://paperswithcode.com/sota/protein-secondary-structure-prediction-on-1?p=protein-secondary-structure-prediction-using-2)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/protein-secondary-structure-prediction-using-2/protein-secondary-structure-prediction-on)](https://paperswithcode.com/sota/protein-secondary-structure-prediction-on?p=protein-secondary-structure-prediction-using-2)\n\n# Protein Secondary Structure Predictor\nProtein secondary structure predictor using CNN\n![protein_banner](images/protein_banner.jpg)\n___\n\n## Introduction\nProteins are chains of amino acids joined together by peptide bonds. Many conformations of this chains are possible due to the many possible combinations of amino acids and rotation of the chain in multiple positions along the chain. It is these conformation changes that are responsible for differences in the three dimensional structure of proteins.\n\n![protein-molecule-structure](images/protein-molecule-structure.jpg)\n\nProtein structure prediction is one of the most important goals pursued by bioinformatics and theoretical chemistry; it is highly important in medicine (for example, in drug design) and biotechnology (for example, in the design of novel enzymes). [[1]](#references)\n\nWhen we talk about the structure of proteins, four different structure levels are mentioned: the primary, secondary, tertiary and quaternary structure.\n\n![Proteine_structures](images/Proteine_structures.jpg)\n\nProtein primary structure is the linear sequence of amino acids in a peptide or protein.\n\nProtein secondary structure is the three dimensional form of local segments of proteins. Secondary structure elements typically spontaneously form as an intermediate before the protein folds into its three dimensional tertiary structure.\nBoth protein and nucleic acid secondary structures can be used to aid in [multiple sequence alignment](https://en.wikipedia.org/wiki/Multiple_sequence_alignment).\n\nThe tertiary structure is however particularly interesting as it describes the 3D structure of the protein molecule, which reveals very important functional and chemical properties, such as which chemical bindings the protein can take part in.\n\nPredicting protein tertiary structure from only its amino acid sequence is a very challenging problem, but using the simpler secondary structure definitions is becomes more tractable. [[2]](#references)\n\nI focused on the primary and secondary structure (SS), more specifically on using Convolutional Neural Networks (CNNs) for predicting the secondary structure of proteins given their primary structure.\n\n## Protein Structures and Protein Data\nThe primary structure of proteins are described by the sequence of amino acids on their polypeptide chain.\n\n![Aminoacids](images/Aminoacids.png)\n\nThere are 20 natural occurring amino acids in the human body which, in a one letter notation, are denoted by: ’A’, ’C’, ’D’, ’E’, ’F’, ’G’, ’H’, ’I’, ’K’, ’L’, ’M’, ’N’, ’P’, ’Q’, ’R’, ’S’, ’T’, ’V’, ’W’, ’Y’. ’A’ standing for Alanine, ’C’ for Cysteine, ’D’ for Aspartic Acid etc. A 21st letter, ’X’, is sometimes used for denoting an unknown or any amino acid.\n\nInstead of using the  primary structure as a simple indicator for the presence of one of the amino acids, a more powerful primary structure representation has been used: **Protein Profiles**.\nThese are used to take into account evolutionary neighborhoods and are used to model protein families and domains. They are built by converting multiple sequence alignments into position-specific scoring systems (PSSMs). Amino acids at each position in the alignment are scored according to the frequency with which they occur at that position. [[3]](#references)\n\n![Proteine_Profile](images/Proteine_Profile.png)\n\nA protein’s polypeptide chain typically consist of around 200-300 amino acids, but it can consist of far less or far more. The amino acids can occure at any position in a chain, meaning that even for a chain consisting of 4 amino acids, there are 204 possible distinct combinations. In the used [dataset](#the-dataset) the average protein chain consists of 208 amino acids.\n\nProteins’ secondary structure determines structural states of local segments of amino acid residues in the protein. The alpha-helix state for instance forms a coiled up shape and the beta-strand forms a zig-zag like shape etc. The secondary structure of the protein is interesting because it, as mentioned in the introduction, reveals important chemical properties of the protein and because it can be used for further predicting it’s tertiary structure. When predicting protein's secondary structure we distinguish between **3-state SS** prediction and **8-state SS** prediction.\n\nFor 3-state prediction the goal is to classify each amino acid into either:\n- alpha-helix, which is a regular state denoted by an ’H’.\n- beta-strand, which is a regular state denoted by an ’E’.\n- coil region, which is an irregular state denoted by a ’C’.\n\nThe letters which denotes the above secondary structures are not to be confused with those which denotes the amino acids.\n\nFor 8-state prediction, Alpha-helix is further sub-divided into three states: alpha-helix (’H’), 310 helix (’G’) and pi-helix (’I’). Beta-strand is sub-divided into: beta-strand (’E’) and beta-bride (’B’) and coil region is sub-divided into: high curvature loop (’S’), beta-turn (’T’) and irregular (’L’). [[2]](#references)\n\n    E = extended strand, participates in β ladder\n    B = residue in isolated β-bridge\n    H = α-helix\n    G = 3-helix (3-10 helix)\n    I = 5-helix (π-helix)\n    T = hydrogen bonded turn\n    S = bend\n    _ = loop (any other type)\n\nFor the scope of this project the more challenging 8-state prediction problem has been chosen.\n\n## Dataset\nThe dataset used is CullPDB data set, consisting of 6133 proteins each of 39900 features.\nThe 6133 proteins × 39900 features can be reshaped into 6133 proteins × 700 amino acids × 57 features.\n\nThe amino acid chains are described by a 700 × 57 matrix to keep the data size consistent. The 700 denotes the peptide chain and the 57 denotes the number of features in each amino acid. When the end of a chain is reached the rest of the vector will simply be labeled as ’No Seq’ (a padding is applied).\n\nAmong the 57 features, 22 represent the primary structure (20 amino acids, 1 unknown or any amino acid, 1 'No Seq' -padding-), 22 the Protein Profiles (same as primary structure) and 9 are the secondary structure (8 possible states, 1 'No Seq' -padding-).\n\nThe Protein profiles where used instead of the amino acids residues.\n\nFor a more detailed description of the dataset and for download see [[4]](#references).\n\nIn a first phase of research the whole amino acid sequence was used as an example (700 x 22) to predict the whole secondary structure (label) (700 x 9).\n\nIn the second phase, local windows of a limited number of elements, shifted along the sequence, were used as examples (`cnn_width` x 21) to predict the secondary structure (8 classes) in a single location in the center of each window (The 'No Seq' and padding were removed and ignored in this phase because it wasn't necessary anymore for the sequences to be of the same length).\n\nThe Dataset (of 6133 proteins) was divided randomly into training (5600), validation (256) and testing (272) sets, as suggested by [[5]](#references) for the results shown below.\n\nHowever different splits of the dataset with different sizes have been tested with equal results.\n\n## Implementation\nThis project was implemented using the **Keras** framework with the **Tensorflow** backend.\n\nTwo main approaches have been explored:\n1. Use the whole protein sequence (primary structure) as an example for the CNN, with an output of dimension 700 x 9, the sequence of the predicted secondary structure.\n2. Use local windows of a limited number of elements as an example for the CNN which is shifted along the sequences, predicting for each window the secondary structure in a single location (8 classes), in the center of each window.\n\n### 1) Whole protein prediction\nThis simple model consists of 3 main 1D Convolutional Layers:\n\n```Python\nLR = 0.0005\ndrop_out = 0.3\nbatch_dim = 64\n\nloss = 'categorical_crossentropy'\n\n# We fix the window size to 11 because the average length of an alpha helix is around eleven residues\n# and that of a beta strand is around six.\n# See references [6].\nm = Sequential()\nm.add(Conv1D(128, 11, padding='same', activation='relu', input_shape=(dataset.sequence_len, dataset.amino_acid_residues)))\nm.add(Dropout(drop_out))\nm.add(Conv1D(64, 11, padding='same', activation='relu'))\nm.add(Dropout(drop_out))\nm.add(Conv1D(dataset.num_classes, 11, padding='same', activation='softmax'))\nopt = optimizers.Adam(lr=LR)\nm.compile(optimizer=opt,\n          loss=loss,\n          metrics=['accuracy', 'mae'])\n```\nThe resulting computation graph (from tensorboard):\n\n![whole_graph](images/whole_graph.png)\n\nThis was a first prototype, with a low number of parameters (125.512 trainable parameters). A major problem with this approach, was the fact that the padding added to shorter sequences, still influenced the loss, calculated on the whole output sequence ('categorical_crossentropy' loss from tensorwlow was used).\n\nThis required the creation of a custom loss to take into account the outputs from the padding region, which is of different shape for each example.\n\nSoon this approach was abandoned.\n\n### 2) Window CNN\nThis model implementation:\n```Python\ncnn_width = 17\n\nLR = 0.0009 # maybe after some (10-15) epochs reduce it to 0.0008-0.0007\ndrop_out = 0.38\nbatch_dim = 64\n\nloss = 'categorical_crossentropy'\n\nm = Sequential()\nm.add(Conv1D(128, 5, padding='same', activation='relu', input_shape=(cnn_width, dataset.amino_acid_residues)))\nm.add(BatchNormalization())\nm.add(Dropout(drop_out))\nm.add(Conv1D(128, 3, padding='same', activation='relu'))\nm.add(BatchNormalization())\nm.add(Dropout(drop_out))\nm.add(Conv1D(64, 3, padding='same', activation='relu'))\nm.add(BatchNormalization())\nm.add(Dropout(drop_out))\nm.add(Flatten())\nm.add(Dense(128, activation='relu'))\nm.add(Dense(32, activation='relu'))\nm.add(Dense(dataset.num_classes, activation = 'softmax'))\nopt = optimizers.Adam(lr=LR)\nm.compile(optimizer=opt,\n          loss=loss,\n          metrics=['accuracy', 'mae'])\n```\n\nThe resulting computation graph (from tensorboard):\n\n![window_graph](images/window_graph.png)\n\nThe size of the window has been chosen to be bigger than 11 because the average length of an alpha helix is around eleven residues and that of a beta strand is around six (See references [6]). Multiple even sizes from 11 to 23 were tested, with 17 yielding the best results (performance/training time trade off).\n\nThis model has 232.552 parameters (Trainable params: 231.912) and was trained on 946494 samples, validated on 120704 samples (windows).\n\n## Results\nThe Window CNN has been trained with the CullPDB dataset split like described in [Dataset section](#Dataset) for 35 epochs (on CPU in approximately 6 hours).\n\nThe learning curves are shown below:\n\n![Window_Acc](images/WindowCNN_Accuracy.png) | ![Window_Loss](images/WindowCNN_Loss.png)\n|:---:|:---:|\n| Window CNN Accuracy (Q8 Accuracy)| Window CNN Loss |\n\nThe accuracy on the test set achieved with this model is equal to `0.721522` (Q8 Accuracy), which is comparable to the results obtained in [[5]](#references) and [[6]](#references) using different techniques.\n\nThe model has also been trained with the filtered version of the dataset: `CullPDB6133+filtered` available at [[4]](#references) and tested with the public benchmark **CB513**. The accuracy obtained is equal to `0.6833` (Q8 Accuracy), again comparable with [[5]](#references) and [[6]](#references).\n\n#### Whole protein prediction\nThis model has been trained with the CullPDB dataset split like described in [Dataset section](#Dataset) for just 20 epochs (on CPU in approximately 25 minutes).\n\nThe learning curves are shown below (in this model the loss is calculated without taking the padding into account, so the resulting values are biased):\n\n![Window_Acc](images/WholeCNN_Accuracy.png) | ![Window_Loss](images/WholeCNN_Loss.png)\n|:---:|:---:|\n| Whole protein CNN Accuracy | Whole protein CNN Loss |\n\nThe accuracy on the test set achieved with this model is equal to `0.6966` (Q8 Accuracy), which is pretty close to the results obtained with the Window CNN in a small fraction of the time required for the Window CNN.\n\nMoreover the accuracy obtained training on the filtered dataset and testing on the CB513 dataset, is equal to `0.6557` (Q8 Accuracy).\n\n## References\n\\[1\\]: https://en.wikipedia.org/wiki/Protein_structure_prediction\n\n\\[2\\]: https://en.wikipedia.org/wiki/Protein_secondary_structure\n\n\\[3\\]: https://www.ebi.ac.uk/training/online/course/introduction-protein-classification-ebi/what-are-protein-signatures/signature-types/what-are-\n\n\\[4\\]: http://www.princeton.edu/%7Ejzthree/datasets/ICML2014/\n\n\\[5\\]: Jian Zhou and Olga G. Troyanskaya (2014) - \"Deep Supervised and Convolutional Generative Stochastic Network for Protein Secondary Structure Prediction\" - https://arxiv.org/pdf/1403.1347.pdf\n\n\\[6\\]: Sheng Wang et al. (2016) - \"Protein Secondary Structure Prediction Using Deep Convolutional Neural Fields\" - https://arxiv.org/pdf/1512.00843.pdf\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucaangioloni%2Fproteinsecondarystructure-cnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucaangioloni%2Fproteinsecondarystructure-cnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucaangioloni%2Fproteinsecondarystructure-cnn/lists"}