{"id":28025623,"url":"https://github.com/aksherwal/project_nlp","last_synced_at":"2026-02-21T04:01:54.576Z","repository":{"id":284420049,"uuid":"954848491","full_name":"Aksherwal/Project_NLP","owner":"Aksherwal","description":"A classification project to distinguish fiction and non-fiction texts using linguistic features derived from POS tagging. Inspired by research, it explores additional features for improved classification accuracy.","archived":false,"fork":false,"pushed_at":"2025-03-26T14:55:47.000Z","size":5430,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-05T07:06:15.670Z","etag":null,"topics":["data-science","logistic-regression","machine-learning","natural-language-processing","nlp","nltk","python","text-classification"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Aksherwal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-25T17:55:30.000Z","updated_at":"2025-03-26T14:55:51.000Z","dependencies_parsed_at":"2025-07-05T07:06:16.450Z","dependency_job_id":"41705466-098c-4538-a991-322a25fa93a4","html_url":"https://github.com/Aksherwal/Project_NLP","commit_stats":null,"previous_names":["aksherwal/project_nlp"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Aksherwal/Project_NLP","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aksherwal%2FProject_NLP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aksherwal%2FProject_NLP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aksherwal%2FProject_NLP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aksherwal%2FProject_NLP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Aksherwal","download_url":"https://codeload.github.com/Aksherwal/Project_NLP/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aksherwal%2FProject_NLP/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29672755,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-21T03:11:15.450Z","status":"ssl_error","status_checked_at":"2026-02-21T03:10:34.920Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","logistic-regression","machine-learning","natural-language-processing","nlp","nltk","python","text-classification"],"created_at":"2025-05-11T04:26:27.407Z","updated_at":"2026-02-21T04:01:54.558Z","avatar_url":"https://github.com/Aksherwal.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fiction and Non-Fiction Text Classification\n\nThis repository contains the implementation of a classification model that distinguishes between fiction and non-fiction texts using linguistic features derived from part-of-speech (POS) tagging. Inspired by the research paper [\"A Simple Approach to Classify Fictional and Non-Fictional Genres\"](./A%20Simple%20Approach%20to%20Classify%20Fictional%20and%20Non-Fictional%20Genres.pdf), we replicate the results with a slight modification by utilizing the NLTK POS tagger instead of the one mentioned in the paper. The results demonstrate the robustness of the study. We further explore additional POS-based features for genre classification.\n\n## Table of Contents\n\n- [Introduction](#introduction)\n- [Features](#features)\n- [Data Sources](#data-sources)\n- [Dependencies](#dependencies)\n- [Usage](#usage)\n- [Results](#results)\n- [Further Work](#further-work)\n- [References](#references)\n\n## Introduction\n\nThe primary goal of this project is to classify text as fiction or non-fiction based on POS-based features. Initially, the study focuses on two key features:\n1. Adverb-to-Adjective Ratio\n2. Adjective-to-Pronoun Ratio\n\nThe classification is done using a logistic regression model. Additional POS-based features are also explored to test their efficacy in genre classification. Read the [original paper](./A%20Simple%20Approach%20to%20Classify%20Fictional%20and%20Non-Fictional%20Genres.pdf) for more details.\n\n## Features\n\n- **Adverb-to-Adjective Ratio**: Measures the prevalence of descriptive adverbs relative to adjectives.\n- **Adjective-to-Pronoun Ratio**: Measures the descriptive richness of text in relation to pronouns.\n- **Custom POS-Based Features**: Additional features derived from linguistic analysis are being evaluated for performance improvement.\n\n## Data Sources\n\n1. **Brown Corpus**: A collection of texts categorized into fiction and non-fiction, provided by the NLTK library.\n2. **Baby BNC (British National Corpus)**: Fictional and non-fictional texts, sourced from the `baby_bnc.csv` file in the repository.\n\n## Dependencies\n\nEnsure the following libraries are installed:\n- Python 3.7+\n- NLTK\n- pandas\n- scikit-learn\n\nInstall dependencies using:\n```bash\npip install -r requirements.txt\n```\n\n## Usage\n\n1. **Prepare the Data**:\n   - Place the `baby_bnc.csv` file in the repository root.\n   - The Brown Corpus is automatically loaded from NLTK.\n\n2. **Run the Notebook**:\n   - Open and execute the Jupyter Notebook [`similar_results.ipynb`](./similar_results.ipynb) to reproduce results or experiment with additional features.\n\n3. **Generate Features**:\n   - Modify the feature extraction logic in the `extract_two_features` function or extend it to include new features.\n\n4. **Train and Test**:\n   - Execute the classification pipeline in the notebook to test the logistic regression model with various feature combinations.\n\n## Results\n\n- Using the NLTK POS tagger, the model achieves results comparable to the original study, validating its robustness.\n- Preliminary experiments with additional POS-based features show promising directions for improving classification accuracy.\n\n## Further Work\n\n- Exploring additional POS-based ratios to improve classification accuracy.\n- Testing the model on a broader set of corpora.\n- Applying other machine learning algorithms to evaluate performance enhancements.\n\n## References\n\n1. Mohammed Rameez Qureshi, Sidharth Ranjan, Rajakrishnan P. Rajkumar, and Kushal Shah. [\"A Simple Approach to Classify Fictional and Non-Fictional Genres\"](./A%20Simple%20Approach%20to%20Classify%20Fictional%20and%20Non-Fictional%20Genres.pdf). *Proceedings of the Second Storytelling Workshop, Florence, Italy, August 1, 2019.*\n2. NLTK documentation: [https://www.nltk.org/](https://www.nltk.org/)\n3. Scikit-learn documentation: [https://scikit-learn.org/](https://scikit-learn.org/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faksherwal%2Fproject_nlp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faksherwal%2Fproject_nlp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faksherwal%2Fproject_nlp/lists"}