{"id":29458306,"url":"https://github.com/anthippi/naive-bayes-imdb-classification","last_synced_at":"2026-04-09T20:37:11.943Z","repository":{"id":303694917,"uuid":"912440459","full_name":"Anthippi/Naive-Bayes-IMDB-Classification","owner":"Anthippi","description":"A custom Naive Bayes classifier for sentiment analysis of movie reviews from the IMDb dataset, utilizing feature selection based on Information Gain and comparing its performance with scikit-learn's BernoulliNB.","archived":false,"fork":false,"pushed_at":"2025-07-08T23:06:19.000Z","size":146,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-09T00:23:21.234Z","etag":null,"topics":["classification","imdb","matplotlib","naive-bayes-classifier","numpy","pandas","scikit-learn","sklearn"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Anthippi.png","metadata":{"files":{"readme":"README.en.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-01-05T15:29:44.000Z","updated_at":"2025-07-08T23:06:22.000Z","dependencies_parsed_at":"2025-07-09T00:23:25.583Z","dependency_job_id":"5ac1a39d-914a-4de9-8241-64aeae076e78","html_url":"https://github.com/Anthippi/Naive-Bayes-IMDB-Classification","commit_stats":null,"previous_names":["anthippi/naive-bayes-imdb-classification"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Anthippi/Naive-Bayes-IMDB-Classification","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anthippi%2FNaive-Bayes-IMDB-Classification","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anthippi%2FNaive-Bayes-IMDB-Classification/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anthippi%2FNaive-Bayes-IMDB-Classification/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anthippi%2FNaive-Bayes-IMDB-Classification/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Anthippi","download_url":"https://codeload.github.com/Anthippi/Naive-Bayes-IMDB-Classification/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anthippi%2FNaive-Bayes-IMDB-Classification/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265222978,"owners_count":23730327,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classification","imdb","matplotlib","naive-bayes-classifier","numpy","pandas","scikit-learn","sklearn"],"created_at":"2025-07-13T23:37:10.373Z","updated_at":"2026-04-09T20:37:06.897Z","avatar_url":"https://github.com/Anthippi.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Naive Bayes Classifier for Sentiment Analysis (IMDB Dataset)\n\nThis project implements a Naive Bayes classifier for sentiment analysis on movie reviews from the IMDb dataset. The classifier is built from scratch to demonstrate fundamental concepts of natural language processing (NLP), feature selection, and probabilistic modeling.\n\n---\n\n## Contents\n\n- Vocabulary creation using `Information Gain`  \n- Text-to-feature vector conversion (bag-of-words)  \n- Custom `Naive Bayes` classifier implementation  \n- Comparison with `Scikit-learn BernoulliNB`  \n- Model evaluation with `precision`, `recall`, `F1`, and `accuracy metrics`  \n\n---\n\n## Code Structure\n\n### `WordStat`\nTracks word statistics: counts in positive/negative documents.\n\n### `VocabularyBuilder`\nImplements vocabulary creation:\n- Filters words (removes most frequent and least frequent)\n- Calculates Information Gain (IG) for feature selection\n- Generates final vocabulary with `m` features\n\n### `FeatureVector`\nConverts documents to feature vectors:\n- Binary features (1 = word present, 0 = absent)\n- Appends sentiment label (1 = positive, 0 = negative)\n\n### `NaiveBayes`\nCustom Naive Bayes implementation:\n- Uses log probabilities with Laplace smoothing\n- Calculates log-likelihood for each feature\n- Predicts most probable class for new documents\n\n---\n\n## Customization\n\nModify these parameters in the notebook:\n\n```python\n# Vocabulary parameters\nn = 50    # Ignore top-n frequent words\nk = 80    # Ignore bottom-k rare words\nm = 500   # Select top-m features\n\n# Training parameters\ntrain_size = 25000  # Training samples per class\ntest_size = 25000   # Test samples per class\n```\n---\n\n## Execution Instructions\n\n1. Download the IMDB dataset  \n2. Unzip the dataset to your project directory  \n3. Run the Jupyter notebook:\n\n```bash\njupyter notebook NaiveBayesClassifier.ipynb\n```\n## Execution Pipeline\n\n### Vocabulary Creation\n Parameters: `n=50`, `k=80`, `m=500`\n- Removes top-50 most frequent words and 80 least frequent words\n- Selects top-500 words based on Information Gain\n\n### Feature Vector Creation\n- Converts all reviews into binary feature vectors\n\n### Training \u0026 Evaluation\n- Trains custom Naive Bayes classifier\n- Trains Scikit-learn BernoulliNB for comparison\n- Prints precision, recall, F1-score, and accuracy metrics\n\n## Model Performance Comparison\n |Model                 | Accuracy | Precision | Recall | F1 Score |\n|----------------------------|----------|-----------|--------|----------|\n| Custom NaiveBayes          | 0.85     | 0.84      | 0.85   | 0.84     |\n| Scikit-Learn NaiveBayes    | 0.86     | 0.85      | 0.86   | 0.85     |\n\n## Requirements\n- Python 3.7+\n- Libraries:\n    - `numpy`\n    - `pandas`\n    - `matplotlib`\n    - `sklearn`\n \nInstall dependencies:\n```bash\npip install numpy pandas matplotlib scikit-learn seaborn jupyter\n```\n--- \n## Key Features\n### Information Gain Calculation\nStatistical method for selecting the most discriminative features\n\n### Laplace Smoothing\nHandles unseen words during classification\n\n### Efficient Vectorization\nBinary features optimized for memory and performance\n\n### Visual Analytics\nLearning curves, confusion matrices, and feature importance plots\n\n---\n\n## Sample Output\n\n```text\nVocabulary creation completed (500 words)  \nFirst 10 vocabulary words: ['excellent', 'wonderful', 'best', 'perfect', 'great', ...]\n\nTraining set size: 25,000 vectors  \nTest set size: 25,000 vectors\n\nCustom NaiveBayes Results:\n              precision    recall  f1-score   support\n\n    Negative       0.84      0.85      0.84     12500\n    Positive       0.85      0.84      0.84     12500\n\n    accuracy                           0.85     25000\n\nScikit-Learn Results:\n              precision    recall  f1-score   support\n\n    Negative       0.85      0.86      0.85     12500\n    Positive       0.86      0.85      0.85     12500\n\n    accuracy                           0.86     25000\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanthippi%2Fnaive-bayes-imdb-classification","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanthippi%2Fnaive-bayes-imdb-classification","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanthippi%2Fnaive-bayes-imdb-classification/lists"}