{"id":20650874,"url":"https://github.com/nlpatvcu/relex-cnn","last_synced_at":"2025-06-25T09:36:27.557Z","repository":{"id":96598397,"uuid":"408616765","full_name":"NLPatVCU/RelEx-CNN","owner":"NLPatVCU","description":null,"archived":false,"fork":false,"pushed_at":"2022-02-15T19:47:03.000Z","size":213,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-16T21:49:41.728Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NLPatVCU.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-20T22:21:20.000Z","updated_at":"2021-10-22T19:46:53.000Z","dependencies_parsed_at":"2023-04-25T21:25:13.808Z","dependency_job_id":null,"html_url":"https://github.com/NLPatVCU/RelEx-CNN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NLPatVCU/RelEx-CNN","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NLPatVCU%2FRelEx-CNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NLPatVCU%2FRelEx-CNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NLPatVCU%2FRelEx-CNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NLPatVCU%2FRelEx-CNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NLPatVCU","download_url":"https://codeload.github.com/NLPatVCU/RelEx-CNN/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NLPatVCU%2FRelEx-CNN/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261845359,"owners_count":23218546,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T17:24:00.360Z","updated_at":"2025-06-25T09:36:27.537Z","avatar_url":"https://github.com/NLPatVCU.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RelEx CNN documentation \nThis is a deep learning-based approach to extract and classify clinical relations. This approach introduces 2 Convolutional Neural Network (CNN) models. \nConvolutional neural  networks  (CNNs)  have  been trending  due  to their  strong  learning  ability of features without manual feature engineering. Initially the convolution layer is a filter which is a set of learnable weights learned using the backpropagation algorithm and it extracts features from the input text. Maxpooling operations use the position information of local features relative to the entity pair and helps to extract the most significant feature from the output of the convolution filter. These advantages of the CNN can be utilized to reduce the dependency on manual feature engineering and learn the features automatically. \n\nEntity pairs of a relation are normally located in a sentence and we can represent the context of each relation by extracting  the sentence. But one sentence can include multiple distinct mentions of relations, therefore learning the entire sentence at once would not help in determining different relation classes. The sentence can be explicitly divided into segments based on the location and the context of the entities and these segments play different roles in determining the class.\n\nOur system mainly consists  of  two  components:  Single label Sentence-CNN  and Segment-CNN.\nIn the following, the algorithm is explained in detail and a walk thorugh guide is provided to run the package.\n\n## Table of Contents\n1. [Installation](#installation)\n   1. [Deployment](#deployment)\n2. [Algorithm](#algorithm)\n   1. [Data Segmentation](#data_segmentation)\n   2. [Pre-processing](#pre-processing)\n      1. [Vectorization](#Vectorization)\n      2. [Label Binarization](#binarizer)\n   3. [Word Embeddings](#word_embeddings)\n   4. [CNN Models](#models)\n      1. [Sentence CNN](#sen_cnn)\n      2. [Segment Cnn](#seg_cnn)\n   5. [Regularization](#Regularization)\n\n## Installation\n\nCreate a python 3.6 virtual environment:``` python3 -m venv \u003cname_of_virtualenv\u003e```\nActivate the virtual environment: ```source new_env/bin/activate```\n\nInstall manually\n```\npip install numpy\npip install cython\npip install -U pip setuptools wheel\npip install -U spacy\npython -m spacy download en_core_web_sm\n```\nInstall the packages given in the *requirements.txt*\n```\npip install -r requirements.txt\n```\n\n### Deployment\nSample dataset (from n2c2-2018 corpus, i2b2-2010 corpus) is given. External embeddings needed to downloaded. \n\nEdit the configs file to set the paths and parameters \n```relex/configs/n2c2.ini```\n\nEdit the path to the config file in the run.py:\n```config.read('configs/n2c2.ini')```\n\nRun the following program: \n```\npython relex/run.py\n```\n\n## Algorithm \n### Data segmentation \u003ca name=\"data_segmentation\"\u003e\u003c/a\u003e\nText and annotation files (in BRAT format) of a dataset are filtered and passed into a dataset object which is read in along with the entity mentions of each relation category. The dataset is prepossessed to convert it to the standard format before segmentation by removing punctuation, accent marks and converting all letters to lowercase.\n\nThe segmentation module, identifies and extracts sentences where entities are located. Sentences are divided into the following segments and wrapped into a segmentation object:\n-   preceding segment\n-   concept1 segment\n-   middle segment\n-   concept2 segment\n-   succeeding segment\n\nWhen extracting sentences, it checks whether the annotated relation type already exists, if not the sentences are labeled as a no-relation pair.\n\n### Pre-processing\n#### Vectorization \n\nNeural  networks  learn  information  through  numerical representation of the data. We need to convert the text into real number vectors. We cannot feed lists of integers into a neural network, therefore, we have to turn our lists into tensors. There are two ways to vectorize the words.\n\n-Pad lists to have same length, and turn them into an integer tensor of shape (samples, word_indices). We used Keras tokenizer to take into account only the top given number of the most common words in data and builds a word index. We used multiple methods to vectorize the words. Maximum length of a sequence is determined and the output sequence is padded according to it. Sequences that are shorter than determined length are padded with value at the end whereas sequences longer are truncated so that they fit the desired length. Position of the padding is controlled by the arguments.\n\n#### Label Binarization\u003ca name=\"binarizer\"\u003e\u003c/a\u003e\nBinarizes labels in a one-vs-all fashion. Several regression and binary classification algorithms are available in scikit-learn can be utilized for this. It converts multi-class labels to binary labels (belong or does not belong to the class) by assigning a unique value or number to each label in a categorical feature.\n\n*Eg: labels - TrCP, TrIP, TeRP*\n\n*TrCP \t\t1 0 0 0 0*\n\n*TrIP\t\t\t0 1 0 0 0*\n\n*TeRP\t\t\t0 0 1 0 0*\n\nIf the multilabel flag is set to true, then the binarization is done in the following manner:\nTrCP, TrIP, TeRP\t1 1 1 0 0\n\n### Word embeddings\u003ca name=\"word_embeddings\"\u003e\u003c/a\u003e\nThe word embeddings map a set of words or phrases in a vocabulary to real-valued vectors which helps to reduce the dimensionality and learn linguistic patterns in the data. Given a batch of vector sequences as input, the embedding layer converts the sequence into real-valued embedding vectors. Initially the weights are assigned randomly and gradually they are adjusted through backpropagation.\n\nUsing pre-trained word embeddings as features on CNN based methods have helped to achieve better performance in previous NLP related studies. We applied both Word2Vec and GloVe representations to train word embeddings in the experiments.\n\n### CNN Models\u003ca name=\"models\"\u003e\u003c/a\u003e\n#### Sentence CNN \u003ca name=\"sen_cnn\"\u003e\u003c/a\u003e\n\nEach relation consists of a pair of entities and the Sentence CNN learns the relation representation for the entire sentence as a whole. First, we identify the sentence where each relation is located and extract the sentence and we feed it into a CNN for learning.\n\nFollowing figure shows the function of a single label sentence CNN.\n\n![](https://lh6.googleusercontent.com/VzMboSkKWKdFSI3E66RiL_s0NLlLJDEGQhbEywKXEIqOnWTHm39w1vPiqy3EUr5NdxRh4q375ejzX-K-znAEifHd-UZnG517UGX11G0y7j2sBb5TD4s-SWWJ2Ptq9GqK1nEZP33c)\n\n#### Segment CNN \u003ca name=\"seg_cnn\"\u003e\u003c/a\u003e\nBased on where the entities are located in the sentence we can divide the sentence into segments. Different segments play different roles in determining the relation class. But each relation in Sentence-CNN is represented by an entire sentence and does not capture the positional information of the entity pairs, therefore when a sentence is divided into segments and trained by separate convolutional units.\n\nA Sentence is explicitly segmented into 5 segments:\n-   preceding - tokenized words before the first concept\n-   concept 1 - tokenized words in the first concept\n-   middle - tokenized words between the 2 concepts\n-   concept 2 - tokenized words in the second concept\n-   succeeding - tokenized words after the second concept\n\n![](https://lh5.googleusercontent.com/_eS0O7NU9XaTM8NoO0-6ETLMF379pv25M0K22PLtni0mX5eskWrQuy196S4RA9gajiZ9zuUVIolVgO-y_iAl6hp-01jBM856rojESO1YwWIJA3oZfygQ3y5DwmdPoDdG04pMWoeD)\n\nAs the figure above shows, we construct separate convolution units for each segment and concatenate before the fixed length vector is fed to the dense layer that performs the classification.\n\nWe experiment with different sliding window sizes, filter sizes, word embeddings, loss functions to fine tune the above three models.\n\n### Regularization\n\nSentence CNN and Segment CNN perform well with small filter sizes while Segment CNN performs large filter sizes. Both single and multi label Sentence CNN performed well with GloVe word embeddings whereas Segment CNN with MIMIC word embeddings.\n\nFor regularization of the model we use dropout technique on the output of convolution layer. Dropout randomly drops few nodes to prevent co-adaptation of hidden units and we set this value to 0.5 while training. We use Adam and rmsprop techniques to optimize our loss function.\n\n# Citation\nIf you use this model in your work please cite as follows:\n```\n@article{mahendran2021extracting,\n  title={Extracting Adverse Drug Events from Clinical Notes},\n  author={Mahendran, Darshini and McInnes, Bridget T},\n  journal={arXiv e-prints},\n  pages={arXiv--2104},\n  year={2021}\n}\n We will update the BibTeX with the conference publication soon.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnlpatvcu%2Frelex-cnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnlpatvcu%2Frelex-cnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnlpatvcu%2Frelex-cnn/lists"}