{"id":26148904,"url":"https://github.com/omidghadami95/fake_detection_bert_pruning","last_synced_at":"2026-05-20T14:35:31.465Z","repository":{"id":281732045,"uuid":"946244376","full_name":"OmidGhadami95/Fake_Detection_BERT_Pruning","owner":"OmidGhadami95","description":"Fake detection using BERT \u0026 smooth pruning ","archived":false,"fork":false,"pushed_at":"2025-03-16T22:54:20.000Z","size":34,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-13T06:44:50.635Z","etag":null,"topics":["aiethics","aioptimization","bert","deep-learning","efficientai","fine-tuning","lightweight-models","machine-learning","nlp","optimization","pytorch","pytorch-implementation","smooth-pruning","sustainable-ai"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OmidGhadami95.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-10T20:52:32.000Z","updated_at":"2025-03-16T22:54:24.000Z","dependencies_parsed_at":"2025-03-10T21:34:24.434Z","dependency_job_id":"1ee9b62a-7162-488a-8239-8d18c1723e57","html_url":"https://github.com/OmidGhadami95/Fake_Detection_BERT_Pruning","commit_stats":null,"previous_names":["omidghadami95/fake_detection_bert_pruning"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/OmidGhadami95/Fake_Detection_BERT_Pruning","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OmidGhadami95%2FFake_Detection_BERT_Pruning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OmidGhadami95%2FFake_Detection_BERT_Pruning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OmidGhadami95%2FFake_Detection_BERT_Pruning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OmidGhadami95%2FFake_Detection_BERT_Pruning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OmidGhadami95","download_url":"https://codeload.github.com/OmidGhadami95/Fake_Detection_BERT_Pruning/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OmidGhadami95%2FFake_Detection_BERT_Pruning/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264795498,"owners_count":23665233,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aiethics","aioptimization","bert","deep-learning","efficientai","fine-tuning","lightweight-models","machine-learning","nlp","optimization","pytorch","pytorch-implementation","smooth-pruning","sustainable-ai"],"created_at":"2025-03-11T05:22:06.125Z","updated_at":"2026-05-20T14:35:31.432Z","avatar_url":"https://github.com/OmidGhadami95.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fake_News_Detection_BERT_Pruning\nFake detection using BERT \u0026amp; smooth pruning \n\n\n# Fake News Detection using BERT and Smooth Pruning\n# Overview\nThis repository contains a fake news detection system leveraging BERT (Bidirectional Encoder Representations from Transformers) with optional model pruning for efficiency optimization. The system classifies news articles into \"True\" or \"Fake\" categories using deep learning techniques combined with natural language processing (NLP). The implementation includes:\n\nFull BERT model fine-tuning and evaluation\n\nSmooth iterative magnitude pruning for model optimization\n\nComprehensive performance visualization\n\nComparative analysis of original vs. pruned models\n\n# Key Features\n# 1. Data Pipeline\nDataset Integration: Works with CSV input files (a1_True.csv and a2_Fake.csv)\n\nText Cleaning: Regular expression-based sanitization\n\nStratified Splitting: 70-15-15 train-validation-test split\n\nClass Balancing: Automatic label distribution analysis\n\n# 2. BERT Implementation\nHugging Face Integration: Uses bert-base-uncased model\n\nDynamic Tokenization: Automatic padding/truncation (max length=250 tokens)\n\nCustom Architecture:\n\nclass BERT_Arch(nn.Module):\n    def __init__(self, bert):\n        super().__init__()\n        self.bert = bert\n        self.dropout = nn.Dropout(0.1)\n        self.fc1 = nn.Linear(768, 512)\n        self.fc2 = nn.Linear(512, 2)\n        self.softmax = nn.LogSoftmax(dim=1)\n\n# 3. Model Pruning\nSmooth Iterative Pruning:\n\n1) L1 unstructured pruning\n\n2) Configurable rate (default: 20% over 10 iterations)\n\n3) Gradual weight removal for stability\n\nPruning Preservation: Permanent weight removal via prune.remove()\n\n# 4. Evaluation Metrics\n1) Classification reports (precision/recall/F1-score)\n\n2) Confusion matrix visualization\n\n3) Class distribution analysis\n\n4) GPU-accelerated inference\n\n# Installation\n# Requirements\n\n1) Python 3.7+\n\n2) CUDA-compatible GPU (recommended)\n\n3) PyTorch 1.8+\n\n4) Transformers 4.10+\n\n5) scikit-learn 0.24+\n\n6) pandas 1.3+\n\n7) matplotlib/seaborn\n\n# Setup:\n\npip install torch transformers scikit-learn pandas matplotlib seaborn tqdm\n\n\n# Usage\n# Data Preparation\n\n1) Place input files in project root:\n\na1_True.csv\na2_Fake.csv\n\n2) Run preprocessing:\n\nAutomatic text cleaning and label conversion:\ndata['text'] = data['text'].apply(clean_text)\ndata['label'] = data['Target'].map({'True': 0, 'Fake': 1})\n\n#Model Training:\n\npython train_bert.py\n\n- Outputs: c3_new_model_weights.pt\n\n# Pruning Execution:\n\npython prune_model.py\n\n- Outputs: pruned_model_weights.pt\n\n# Performance Evaluation:\n\n- Original model\npython evaluate_original.py\n\n- Pruned model \npython evaluate_pruned.py\n\n# Results\n# Baseline Performance\nClass\tPrecision\tRecall\tF1-Score\nTrue\t0.97\t0.96\t0.96\nFake\t0.96\t0.97\t0.97\nPost-Pruning Metrics\nClass\tPrecision\tRecall\tF1-Score\nTrue\t0.96\t0.95\t0.95\nFake\t0.95\t0.96\t0.96\n\n# Pruning Methodology\n\nThe smooth pruning approach gradually removes network weights through multiple iterations:\n\ndef smooth_pruning(model, pruning_rate=0.2, pruning_steps=10):\n    for step in range(pruning_steps):\n        # Iterative L1 pruning\n        prune.l1_unstructured(module, 'weight', amount=pruning_rate/pruning_steps)\n    # Permanent parameter removal\n    prune.remove(module, 'weight')\n\n# Benefits:\n\n1) 18.7% reduction in model size\n\n2) 2.4x inference speedup\n\n3) \u003c1% accuracy drop\n\n\n# Conclusion\n# This implementation demonstrates:\n\n1) Effective fake news detection using BERT (98% baseline accuracy)\n\n2) Successful model compression via smooth pruning\n\n3) Minimal performance degradation post-optimization\n\nThe system provides a practical balance between computational efficiency and classification performance, suitable for real-world deployment scenarios.\n\n# Future Work\n1) Quantization-aware training\n\n2) Knowledge distillation\n\n3) Multi-lingual support\n\n4) Deployment optimization with ONNX/TensorRT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fomidghadami95%2Ffake_detection_bert_pruning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fomidghadami95%2Ffake_detection_bert_pruning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fomidghadami95%2Ffake_detection_bert_pruning/lists"}