{"id":18075789,"url":"https://github.com/emmanuelezenwere/erpager","last_synced_at":"2026-04-09T17:03:56.715Z","repository":{"id":258490022,"uuid":"871107850","full_name":"EmmanuelEzenwere/ERpager","owner":"EmmanuelEzenwere","description":"Machine Learning NLP Web Application to Extract, Transform and Load (ETL) Twitter Messages into an SQL database and classify messages into response categories for First Responders, Disaster Response Organisations and Emergency Response Personnels during Disasters.","archived":false,"fork":false,"pushed_at":"2024-10-28T15:28:07.000Z","size":12943,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-04T10:24:45.307Z","etag":null,"topics":["css","etl-pipeline","flask","heroku","html","javascript","ml-engineering","ml-pipeline","nlp","pandas","python","sklearn","software-engineering","web-application"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EmmanuelEzenwere.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-11T09:33:51.000Z","updated_at":"2024-10-30T05:19:42.000Z","dependencies_parsed_at":"2024-10-28T05:40:18.712Z","dependency_job_id":null,"html_url":"https://github.com/EmmanuelEzenwere/ERpager","commit_stats":null,"previous_names":["emmanuelezenwere/disaster_response_pipeline_project","emmanuelezenwere/erpager"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/EmmanuelEzenwere/ERpager","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmmanuelEzenwere%2FERpager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmmanuelEzenwere%2FERpager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmmanuelEzenwere%2FERpager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmmanuelEzenwere%2FERpager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EmmanuelEzenwere","download_url":"https://codeload.github.com/EmmanuelEzenwere/ERpager/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmmanuelEzenwere%2FERpager/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268351467,"owners_count":24236414,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-02T02:00:12.353Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["css","etl-pipeline","flask","heroku","html","javascript","ml-engineering","ml-pipeline","nlp","pandas","python","sklearn","software-engineering","web-application"],"created_at":"2024-10-31T11:07:24.328Z","updated_at":"2025-12-30T19:54:14.178Z","avatar_url":"https://github.com/EmmanuelEzenwere.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Disaster Response Pipeline Project 🚨\n\nA Machine Learning Web Application that processes Twitter messages during disasters, categorizing them to help Response Organizations efficiently direct aid. The system performs Extract, Transform and Load (ETL) operations on messages and classifies them into relevant emergency response categories.\n\n## Dashboard Preview\n\n![Disaster Response Dashboard](assets/DisasterResponseDashboard.png)\n![Analysis Plots](assets/Plots.png)\n\n\n\n## Quick Start\n\n### Prerequisites\n\n- Python 3.6+\n- pip package manager\n\n### Installation\n\n1. **Create and activate a virtual environment**\n   ```bash\n   # Create virtual environment\n   python3 -m venv myenv\n   \n   # Activate virtual environment\n   # On Unix/macOS:\n   source myenv/bin/activate\n   # On Windows:\n   myenv\\Scripts\\activate\n   ```\n\n2. **Install dependencies**\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n3. **Set up the database and train the model**\n   ```bash\n   # Process data and create database\n   python data/process_data.py data/disaster_messages.csv data/disaster_categories.csv data/DisasterResponse.db\n   \n   # Train and save the classifier\n   python models/train_classifier.py data/DisasterResponse.db models/classifier.pkl\n   ```\n\n4. **Launch the web application**\n   ```bash\n   python app/run.py\n   ```\n\n5. **Access the application**\n   - Open your browser and navigate to: http://127.0.0.1:3001/ or http://0.0.0.0:3001/\n\n## Project Structure\n\n### Data Processing (`data/`)\nThe ETL pipeline (`process_data.py`) handles:\n- Loading data from CSV files\n- Merging messages and categories datasets\n- Cleaning and transforming data\n- Storing processed data in SQLite database\n\nKey functions:\n- `load_data()`: Data extraction from CSV\n- `save_data()`: Database storage operations\n\n### Machine Learning Pipeline (`models/`)\nThe ML pipeline (`train_classifier.py`) includes:\n- Data loading from SQLite database\n- Text processing and feature engineering\n- Model training and evaluation\n- Model persistence (pickle format)\n\nKey components:\n- Custom tokenizer with NLTK\n- StartingVerbExtractor feature\n- Multi-output classification pipeline\n- GridSearchCV for hyperparameter tuning\n\n### Web Application (`app/`)\nFlask-based web interface providing:\n- Interactive message classification\n- Data visualizations\n- Real-time prediction results\n\n## Dataset Analysis\n\n![Data Distribution 1](assets/data_summary_1.png)\n\u003cbr\u003e\u003cbr\u003e\n![Data Distribution 2](assets/data_summary_2.png)\n\n\n\n### Class Imbalance Considerations\nThe dataset exhibits significant class imbalance, particularly in categories like 'water' and 'child alone' which has near or all zeros. This presents several challenges:\n\n- **Training Impact**: Underrepresented classes may have lower prediction accuracy\n- **Metric Selection**: F1-score provides a balanced measure for imbalanced classes\n- **Strategy**: Model evaluation emphasizes:\n  - High recall for critical categories (e.g., medical help)\n  - High precision for resource allocation categories\n\n## Future Enhancements\n\n- [ ] Additional web app visualizations\n- [ ] Organization recommendation system\n- [ ] UI/UX improvements\n- [ ] Cloud deployment\n- [ ] Pipeline optimization\n- [ ] Enhanced handling of class imbalance eg using class weights in the ML training pipeline.\n- [ ] Integration with disaster response organizations\n\n## Testing\n\nRun the test suite (In development):\n```bash\npython -m tests/test_data_processing.py\npython -m tests/test_train_classifier.py\n```\n\n## Development Notes\n\nThe `workspace/` directory contains Jupyter notebooks used for:\n- Experimental feature development\n- Pipeline prototyping\n- Model evaluation\n- Visualization testing\n\n---\n*This project is actively maintained and welcomes contributions.*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femmanuelezenwere%2Ferpager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femmanuelezenwere%2Ferpager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femmanuelezenwere%2Ferpager/lists"}