{"id":18729579,"url":"https://github.com/pbteja1998/ire_project_18","last_synced_at":"2025-11-12T07:30:14.923Z","repository":{"id":89716262,"uuid":"157237433","full_name":"pbteja1998/ire_project_18","owner":"pbteja1998","description":"Scientific Paper Summarization","archived":false,"fork":false,"pushed_at":"2018-11-17T15:41:48.000Z","size":12124,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-12-28T14:37:14.213Z","etag":null,"topics":["naive-bayes-classifier","nltk","sckiit-learn","summarization"],"latest_commit_sha":null,"homepage":"https://ire-project-18.herokuapp.com/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pbteja1998.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-12T15:48:21.000Z","updated_at":"2018-11-17T15:41:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"c45d3992-924e-409f-8d98-b7369b58d6de","html_url":"https://github.com/pbteja1998/ire_project_18","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbteja1998%2Fire_project_18","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbteja1998%2Fire_project_18/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbteja1998%2Fire_project_18/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pbteja1998%2Fire_project_18/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pbteja1998","download_url":"https://codeload.github.com/pbteja1998/ire_project_18/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239599040,"owners_count":19665911,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["naive-bayes-classifier","nltk","sckiit-learn","summarization"],"created_at":"2024-11-07T14:27:37.384Z","updated_at":"2025-11-12T07:30:14.855Z","avatar_url":"https://github.com/pbteja1998.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Summarization of Scientific Texts: A Rhetorical Approach\n\n- This project is based on [Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status](https://www.mitpressjournals.org/doi/pdf/10.1162/089120102762671936)\n\n- The main idea behind this paper is to use a rhetorical approach for classifying different statements present in a scientific paper on basis of argumentative zoning.\n\n- This project builds towards automatic summarisation of scientific papers. We aim to classify each sentence within the research paper as one of the 7 rhetorical categories as mentioned below.\n\n### Annotation Based on Argumentative Zoning\n\nEach of the statement in the paper is divided into following different categories\n- **Aim** - Specific research goal of the current paper  \n- **Textual** - Makes reference to the structure of the current paper    \n- **Own** - (Neutral) description of own work presented in current paper: Methodology, results, discussion  \n- **Background** - Generally accepted scientific background  \n- **Contrast** - Statements of comparison with or contrast to other work; weaknesses of other work  \n- **Basis** - Statements of agreement with other work or continuation of other work  \n- **Other** - (Neutral) description of other researchers’ work  \n\nOn basis of the above rhetorical categories we do the argumentative zoning of the sentences present in the papers. \n\n### Features of a sentence\n\n- **Location** - Where in the document the sentence occurs  \n- **Section Structure** - Where in the section does the sentence occurs, i.e. if a sentence is a first line of the section and so on.  \n- **Paragraph Structure** - Whether a sentence occurs in the start, middle or the end of a paragraph.  \n- **Headline**\n- **Length** - Whether the given line is a long line or not.  \n- **Title** - If the words in the sentence occur in the title or not.  \n- **TF IDF Score** - Whether the sentence consists of significant words or not.  \n- **Voice** - What is the voice of the main verb of the sentence.  \n- **Tense** - Tense of the main verb or aux verb of the sentence.  \n- **Modal** - Just using the above concept we find whether there is an auxiliary verb with the main verb. If yes we give the corresponding values.  \n\n\n### Approaches and Tools Used\nWe used existing argumentative zoning dataset and on that we created different feature vectors corresponding to each sentence, and then we trained a ***Naive Bayes*** classifier on the dataset. We did a test-train split of **0.8**\n\nWe used ***NLTK*** and ***Scikit*** for writing the classifier. Since we used scikit learn we were able to test our model with multiple distributions. \n\nWe have used ***Naive Bayes*** with the following distributions: \n- ***Bernoulli Distribution***\n- ***Gaussian Distribution***\n- ***Multinomial Distribution***\n- ***Complement Distribution***\n\n### Results \n\n| Type      | Number of papers |\n|-----------| -------------    |\n| Train Dataset  | ***64 ( 80 % )***      |\n| Test Dataset   | ***15 ( 20 % )***      |\n\n\n\n| Distribution      | Accuracy |\n|-----------| -------------    |\n| Bernoulli  | ***84.64***      |\n| Gaussian   | ***100***      |\n| Multinomial  | ***80.89***      |\n| Complement   | ***81.28***      |\n\n\n### Plots:\n\n---\n\n#### Bernoulli Distribution Confusion Matrix\n![Bernoulli Distribution Confusion Matrix](plots/bernouli_cf_matrix.png)\n#### Bernoulli Distribution Histogram\n![Bernoulli Distribution Histogram](plots/bernouli_hist.png)\n\n---\n\n#### Complement Distribution Confusion Matrix\n![Complement Distribution Confusion Matrix](plots/complement_cf_matrix.png)\n#### Complement Distribution Histogram\n![Complement Distribution Histogram](plots/complement_hist.png)\n\n---\n\n#### Gaussian Distribution Confusion Matrix\n![Gaussian Distribution Confusion Matrix](plots/guassian_cf_matrix.png)\n#### Gaussian Distribution Histogram\n![Gaussian Distribution Histogram](plots/guassian_hist.png)\n\n---\n\n#### Multinomial Distribution Confusion Matrix\n![Multinomial Distribution Confusion Matrix](plots/multinomial_cf_matrix.png)\n#### Multinomial Distribution Histogram\n![Multinomial Distribution Histogram](plots/multinomail_hist.png)\n\n---\n\n### Running the code\n\n\u003cpre\u003e\nTo Generate summary of a given file\n$ \u003cb\u003e python src/summary.py {relative_path_of_file_from_summary.py} \u003c/b\u003e\n\nExample:\n$ \u003cb\u003e python src/summary.py ../data/tagged/9405001.az-scixml \u003c/b\u003e\n\u003c/pre\u003e\n\n\u003cpre\u003e\nTo Train, Test and get accuracy of the classification of sentences\n$ \u003cb\u003e python src/naive_bayes.py \u003c/b\u003e\n\u003c/pre\u003e\n\n\u003cpre\u003e\nRunning Flask app Locally\n\n$ \u003cb\u003e sudo apt-get install python-pip \u003c/b\u003e\n$ \u003cb\u003e sudo pip install virtualenv \u003c/b\u003e\n$ \u003cb\u003e virtualenv -p python venv \u003c/b\u003e\n$ \u003cb\u003e source venv/bin/activate \u003c/b\u003e\n$ \u003cb\u003e pip install -r requirements.txt \u003c/b\u003e\n$ \u003cb\u003e export PORT=5000 \u003c/b\u003e\n$ \u003cb\u003e gunicorn -b :$PORT --chdir src app:app \u003c/b\u003e\n\n\u003c/pre\u003e\n\n\u003cpre\u003e\n\nAfter running the above commands, go the the following url\n \u003cb\u003e \u003ca href=\"http://0.0.0.0:5000/\"\u003e http://0.0.0.0:5000/ \u003c/a\u003e \u003c/b\u003e\n\n\u003c/pre\u003e\n\n### References\n\n- [Automatic Argumentative-Zoning Using Word2vec](https://arxiv.org/abs/1703.10152)\n- [Argumentative Zoning](https://www.cl.cam.ac.uk/~sht25/az.html)\n- [Tools for automated rhetorical analysis of academic writing](http://antonetteshibani.com/tag/argumentative-zoning/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpbteja1998%2Fire_project_18","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpbteja1998%2Fire_project_18","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpbteja1998%2Fire_project_18/lists"}