{"id":18268483,"url":"https://github.com/anibalsanchez/disaster-response-pipeline-project","last_synced_at":"2025-10-04T05:10:18.537Z","repository":{"id":260090529,"uuid":"879799847","full_name":"anibalsanchez/disaster-response-pipeline-project","owner":"anibalsanchez","description":"DisasterResponseAI: Crisis management with NLP-powered message classification for swift, targeted relief efforts. #DataScience #MachineLearning #EmergencyResponse","archived":false,"fork":false,"pushed_at":"2024-11-01T13:54:52.000Z","size":622,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-09T02:47:04.763Z","etag":null,"topics":["datascience","datascience-machinelearning","disasterresponse","disasterresponsepipeline","emergencyresponse","nlp-machine-learning"],"latest_commit_sha":null,"homepage":"https://blog.anibalhsanchez.com/en/blogging/87-data-driven-disaster-response-smart-message-classification-system.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc-by-4.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anibalsanchez.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-28T15:11:41.000Z","updated_at":"2024-11-11T10:47:32.000Z","dependencies_parsed_at":"2024-10-29T13:51:15.599Z","dependency_job_id":null,"html_url":"https://github.com/anibalsanchez/disaster-response-pipeline-project","commit_stats":null,"previous_names":["anibalsanchez/disaster-response-pipeline-project"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/anibalsanchez/disaster-response-pipeline-project","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anibalsanchez%2Fdisaster-response-pipeline-project","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anibalsanchez%2Fdisaster-response-pipeline-project/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anibalsanchez%2Fdisaster-response-pipeline-project/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anibalsanchez%2Fdisaster-response-pipeline-project/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anibalsanchez","download_url":"https://codeload.github.com/anibalsanchez/disaster-response-pipeline-project/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anibalsanchez%2Fdisaster-response-pipeline-project/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278266896,"owners_count":25958733,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datascience","datascience-machinelearning","disasterresponse","disasterresponsepipeline","emergencyresponse","nlp-machine-learning"],"created_at":"2024-11-05T11:31:48.392Z","updated_at":"2025-10-04T05:10:18.516Z","avatar_url":"https://github.com/anibalsanchez.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Disaster Response Pipeline Project\n\nData-Driven Disaster Response: Smart Message Classification System.\n\n## Introduction\n\nAs a Udemy [Data Scientist](https://www.udacity.com/enrollment/nd025) Nanodegree Program student, I'm tasked with solving the **Disaster Response Pipeline Project** and publishing the results.\n\n_This project aims to revolutionize disaster response by developing an intelligent system that rapidly categorizes and routes incoming messages to appropriate relief agencies. Using advanced NLP and machine learning, it provides instant multi-category classification through a user-friendly web interface, enabling swift and efficient resource allocation. The goal is to significantly improve disaster management effectiveness, ultimately saving more lives and minimizing crisis impact through data-driven response strategies._\n\n![Disaster Response Pipeline Project](images/disaster-response-pipeline-project_kelly-sikkema-_whs7FPfkwQ-unsplash.jpg)\n\nThis project applies data engineering skills to analyze disaster data from Appen and build a model for an API that classifies disaster messages. The main components include:\n\n- **ETL Pipeline**: Processes and cleans disaster message data, storing it in a SQLite database. (data/process_data.py)\n- **ML Pipeline**: Develops a machine learning model to categorize disaster messages (models/train_classifier.py).\n- **Flask Web App**: Provides an interface for emergency workers to input new messages and receive classification results (app/run.py).\n\nKey features:\n\n- Real-time classification of disaster messages\n- Data visualizations of the disaster response data\n- Utilizes NLP and machine learning techniques\n\nThe project showcases:\n\n- Data pipeline development\n- Machine learning model creation\n- Web application deployment\n- Clean, organized code structure\n\nThe notebook and source code are available here:\n\n- Blog post: \u003chttps://blog.anibalhsanchez.com/en/blogging/87-data-driven-disaster-response-smart-message-classification-system.html\u003e\n- Repository: \u003chttps://github.com/anibalsanchez/disaster-response-pipeline-project\u003e\n\n## Quick Start Guide - Setup and Execution\n\n**Prepare the Environment**\n\n- Clone the repository\n\n**Set up Database and Model**\n\nRun these commands in the project's root directory:\n\n- ETL Pipeline (data cleaning and database storage):\n  ```sh\n  python data/process_data.py data/disaster_messages.csv data/disaster_categories.csv data/DisasterResponse.db \u003e process_data.log\n  ```\n- ML Pipeline (model training and saving):\n  ```sh\n  python models/train_classifier.py data/DisasterResponse.db models/classifier.pkl \u003e train_classifier.log\n  ```\n\n**Launch the Web Application**\n\n- Navigate to the app directory: `cd app`\n- Start the Flask server: `python run.py`\n\n**Access the Web Interface**\n\n- Open your browser and go to: `http://localhost:3001`\n\nEnsure all file paths are correct before running scripts. The web app may take a moment to load due to data processing.\n\n![Access the Web Interface](images/disaster-response-project-web-app.png)\n\n### Lando Container Development\n\nFor convenience, the project contains the Lando development container definition in the `.lando.yml` file. To start the environment, execute:\n\n```sh\nlando start\n\n ✔ Container disasterresponsepipelineproject_python_1  Started                                                                 0.1s\n * Serving Flask app 'run'\n * Debug mode: on\nWARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.\n * Running on all addresses (0.0.0.0)\n * Running on http://127.0.0.1:3001\n * Running on http://192.168.96.2:3001\nPress CTRL+C to quit\n * Restarting with stat\n * Debugger is active!\n * Debugger PIN: 139-837-717\n192.168.96.1 - - [29/Oct/2024 11:05:28] \"GET / HTTP/1.1\" 200 -\n```\n\n### Production Cloud Environments\n\nThe project also includes the basic configuration for Apache Python WSGI, `disaster-response-pipeline-project.wsgi`, and the Amazon AWS AppRunner, `apprunner.yaml`. The Amazon AWS AppRunner was an attempt to run the project in this context, but it fell short due to the limited resources. The project's proper cloud configuration is a classic setup in **Amazon EC2, a c5.xlarge instance**.\n\n## Exploring the Data\n\nTo start the project, I imported the disaster messages and categories and proceeded with the ETL and cleanup.\n\nThe ETL (Extract, Transform, Load) process described in the code can be explained in simpler terms as follows:\n\n### Data Loading\n\nThe process begins by reading two separate CSV files: one containing messages and another with categories. These two datasets are merged based on a common identifier, creating a comprehensive dataset.\n\n### Data Cleaning\n\nThe cleaning process starts by removing duplicate messages and filtering out entries that begin with 'NOTES:' (these messages are manually identified as unrelated to disasters). The text of each message is then preprocessed by removing URLs (replacing them with a \"urlplaceholder\" label) and special characters and converting everything to lowercase for consistency.\n\n### Data Restructuring\n\nSince the classification is based on the English language, the 'original' message column is eliminated.\n\n#### Category Handling\n\nThe 'categories' column initially contains multiple categories in a single string and is split into separate columns. Each category is converted from a text label to a binary value, making it easier to work with in data analysis or machine learning tasks. The newly created category columns are added to the primary dataset.\n\nAs mentioned in the project definition, the dataset is imbalanced. There are categories with few samples—for instance, the `water` column. The small number of samples of these categories affects the model training. To test different scenarios, I defined a threshold (based on the `water` column) of 0.07 to remove the columns in which the ratio is lower and developed the routine `clean_data_full`. This process keeps only the most relevant columns and removes any rows where all category values are zero, ensuring that each remaining entry has at least some categorization.\n\nGiven the exercise scope, the project trains the final model with all 36 categories using the `clean_data_base` method.\n\n### Final Cleanup\n\nThis ETL process essentially transforms the data from two sources into a clean, structured format ready for further analysis or modeling.\n\n## Feature set exploration\n\nThis is the list of categories filtered above the 0.07 threshold:\n\n| Name |\n|------|\n| request |\n| aid_related |\n| medical_help |\n| child_alone |\n| food |\n| shelter |\n| other_aid |\n| weather_related |\n| floods |\n| storm |\n| earthquake |\n| direct_report |\n\nThe whole list of the 36 categories are:\n\n| Name |\n|------|\n| related |\n| request |\n| offer |\n| aid_related |\n| medical_help |\n| medical_products |\n| search_and_rescue |\n| security |\n| military |\n| child_alone |\n| water |\n| food |\n| shelter |\n| clothing |\n| money |\n| missing_people |\n| refugees |\n| death |\n| other_aid |\n| infrastructure_related |\n| transport |\n| buildings |\n| electricity |\n| tools |\n| hospitals |\n| shops |\n| aid_centers |\n| other_infrastructure |\n| weather_related |\n| floods |\n| storm |\n| fire |\n| earthquake |\n| cold |\n| other_weather |\n| direct_report |\n\n## Disaster Response Classifier\n\nThe Disaster Response Classifier implements a machine-learning process for classifying disaster-related messages.\n\nThe text processing processes the text by breaking it into individual words and simplifying them to their base form, **lemmatization** via WordNet Lemmatizer.\n\nThe model is built using the **TF-IDF vectorization** and the **Random Forest** method to learn how to simultaneously categorize messages into multiple disaster-related categories (via the Multi-Output Classifier). To compensate the possible imbalance in the dataset, the **balanced class weight** and the **balanced_subsample class weight** have been tested in the optimization search. The parameter can help improve the model's performance on the minority classes without the need for additional data preprocessing or sampling techniques. The **Grid Search** technique optimizes the model and finds the best parameters.\n\nFinally, the data is split into training and testing sets for evaluation. The confusion matrix and the f1 scores are produced to analyze the results.\n\nThe optimized model is saved for later use in the web app.\n\n## Findings in the Disaster Response Dataset\n\nI trained the model in the fully cleaned categories and the full list of categories. The model performs well across most categories, with accuracies ranging from 0.72 to 1.00. This indicates that the classifier is effective in categorizing disaster-related messages.\n\nThe following sorted tables allow easier identification of the best and worst performing categories based on their F1-scores.\n\n- High-Performing: F1-score ≥ 0.90\n- Moderate-Performing: 0.80 ≤ F1-score \u003c 0.90\n- Lower-Performing: F1-score \u003c 0.80\n\n### Confusion Matrix Table For Fully Cleaned Categories\n\n| Category | Accuracy | Precision | Recall | F1-score | Performance |\n|----------|----------|-----------|--------|----------|-------------|\n| earthquake | 0.97 | 0.96 | 0.83 | 0.96 | High-Performing |\n| food | 0.95 | 0.92 | 0.82 | 0.95 | High-Performing |\n| storm | 0.94 | 0.86 | 0.79 | 0.94 | High-Performing |\n| floods | 0.93 | 0.91 | 0.61 | 0.93 | High-Performing |\n| shelter | 0.93 | 0.84 | 0.66 | 0.92 | High-Performing |\n| medical_help | 0.89 | 0.68 | 0.53 | 0.89 | Moderate-Performing |\n| weather_related | 0.89 | 0.93 | 0.85 | 0.89 | Moderate-Performing |\n| request | 0.86 | 0.79 | 0.74 | 0.86 | Moderate-Performing |\n| aid_related | 0.85 | 0.89 | 0.91 | 0.85 | Moderate-Performing |\n| direct_report | 0.81 | 0.70 | 0.72 | 0.81 | Moderate-Performing |\n| other_aid | 0.78 | 0.56 | 0.31 | 0.75 | Lower-Performing |\n\nThese are **High-Performing Categories**:\n\n- **Earthquake, Food, Storm, Floods, and Shelter**: These categories show excellent performance with F1-scores above 0.90, indicating high accuracy in identifying these specific disaster-related messages.\n\nThese are the **Moderate-Performing Categories**:\n\n- **Medical Help, Weather Related, Request, Aid Related, and Direct Report**: These categories show good performance with F1-scores around 0.85 (+-0.05), suggesting reliable classification for these broader categories.\n- Concerning to **Medical Help**: Despite the accuracy (0.89), it has a low recall (0.53) for positive cases, suggesting _the model might be missing many true medical help requests_.\n\nThese are the **Lower-Performing Categories**:\n\n- **Other Aid**: This category has the lowest F1-score (0.75), indicating difficulty in accurately classifying messages related to miscellaneous aid requests.\n\n**Imbalanced Data Considerations**: Several categories (e.g., Medical Help, Floods) show high accuracy but lower recall for the positive class, indicating a potential class imbalance. This suggests the model might be biased towards the majority class in these categories.\n\n### Confusion Matrix Table For All 36 Categories\n\n| Category | Accuracy | Precision | Recall | F1-score | Performance |\n|----------|----------|-----------|--------|----------|-------------|\n| child_alone | 1.00 | 0.00 | 0.00 | 1.00 | High-Performing |\n| related | 0.99 | 1.00 | 1.00 | 0.99 | High-Performing |\n| offer | 0.99 | 0.00 | 0.00 | 0.99 | High-Performing |\n| tools | 0.99 | 0.00 | 0.00 | 0.99 | High-Performing |\n| shops | 0.99 | 0.00 | 0.00 | 0.99 | High-Performing |\n| missing_people | 0.99 | 0.78 | 0.22 | 0.98 | High-Performing |\n| fire | 0.99 | 0.57 | 0.15 | 0.98 | High-Performing |\n| clothing | 0.98 | 0.58 | 0.53 | 0.98 | High-Performing |\n| electricity | 0.98 | 0.59 | 0.50 | 0.98 | High-Performing |\n| hospitals | 0.98 | 0.30 | 0.18 | 0.98 | High-Performing |\n| cold | 0.98 | 0.68 | 0.54 | 0.98 | High-Performing |\n| money | 0.97 | 0.51 | 0.41 | 0.97 | High-Performing |\n| security | 0.97 | 0.27 | 0.15 | 0.97 | High-Performing |\n| aid_centers | 0.98 | 0.40 | 0.06 | 0.97 | High-Performing |\n| military | 0.96 | 0.55 | 0.63 | 0.97 | High-Performing |\n| search_and_rescue | 0.96 | 0.53 | 0.26 | 0.96 | High-Performing |\n| water | 0.96 | 0.72 | 0.82 | 0.96 | High-Performing |\n| death | 0.96 | 0.67 | 0.56 | 0.96 | High-Performing |\n| earthquake | 0.96 | 0.88 | 0.82 | 0.96 | High-Performing |\n| refugees | 0.95 | 0.41 | 0.43 | 0.95 | High-Performing |\n| transport | 0.95 | 0.51 | 0.39 | 0.95 | High-Performing |\n| food | 0.94 | 0.80 | 0.82 | 0.94 | High-Performing |\n| buildings | 0.94 | 0.56 | 0.67 | 0.94 | High-Performing |\n| storm | 0.93 | 0.70 | 0.77 | 0.94 | High-Performing |\n| medical_products | 0.93 | 0.43 | 0.50 | 0.93 | High-Performing |\n| floods | 0.93 | 0.66 | 0.72 | 0.93 | High-Performing |\n| shelter | 0.92 | 0.68 | 0.71 | 0.92 | High-Performing |\n| other_weather | 0.92 | 0.38 | 0.41 | 0.92 | High-Performing |\n| medical_help | 0.90 | 0.50 | 0.56 | 0.91 | High-Performing |\n| other_infrastructure | 0.91 | 0.27 | 0.31 | 0.92 | High-Performing |\n| request | 0.87 | 0.71 | 0.72 | 0.87 | Moderate-Performing |\n| weather_related | 0.86 | 0.85 | 0.76 | 0.86 | Moderate-Performing |\n| infrastructure_related | 0.88 | 0.33 | 0.36 | 0.88 | Moderate-Performing |\n| other_aid | 0.80 | 0.42 | 0.44 | 0.80 | Moderate-Performing |\n| direct_report | 0.81 | 0.64 | 0.67 | 0.81 | Moderate-Performing |\n| aid_related | 0.72 | 0.80 | 0.66 | 0.72 | Lower-Performing |\n\nThe model shows varying performance across the 36 different categories. High accuracy scores are across most categories (\u003e0.90 in many cases). The precision, recall, and F1-scores vary significantly between categories.\n\nThe **best-performing categories** are Related (Accuracy: 0.99, Precision: 1.00, Recall: 1.00, F1: 0.99), Food (Accuracy: 0.94, Precision: 0.80, Recall: 0.82, F1: 0.94) and Earthquake Good performance (Accuracy: 0.96, Precision: 0.88, Recall: 0.82, F1: 0.96).\n\nThe **worst-performing** are Offer (Precision: 0.00, Recall: 0.00), Child_alone (Precision: 0.00, Recall: 0.00), Poor performance (Precision: 0.00, Recall: 0.00) and Shops (Precision: 0.00, Recall: 0.00).\n\n**Imbalanced Data Considerations**: There are categories with **a high accuracy but low precision/recall**.\nCategories like \"fire, \"security\", \"missing_people\", and \"hospitals\" show this pattern and suggest potential class imbalance issues.\n\nThere are categories with **zero precision and recall**. Categories like \"offer\", \"child_alone\", \"tools\", and \"shops\". It indicates severe class imbalance or a complete lack of positive examples.\n\nOther categories like \"aid_related\" and \"weather_related\" show balanced metrics, suggesting a more balanced data distribution.\n\nTo address class imbalance, techniques like oversampling, undersampling, or SMOTE for imbalanced categories are recommended to improve poorly performing categories. Additionally, the categories with zero precision/recall require to gather more data to overcome this issue.\n\nBy addressing these issues, the model's performance can be improved across all categories, especially those currently underperforming.\n\n## Conclusion\n\nThe classifier performs well overall, especially in specific disaster types like earthquakes and storms. Data cleaning significantly affects the model's results. There's room for improvement in the poorly performing categories. Further analysis and potential model adjustments could enhance performance in these areas.\n\n## LICENSE\n\nAttribution 4.0 International - CC BY 4.0 \u003chttps://creativecommons.org/licenses/by/4.0/\u003e\n\n\u003cp xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dct=\"http://purl.org/dc/terms/\"\u003e\u003ca property=\"dct:title\" rel=\"cc:attributionURL\" href=\"https://github.com/anibalsanchez/answering-house-prices-questions-based-on-advanced-regression-techniques\"\u003eDisaster Response Pipeline Project\u003c/a\u003e by \u003ca rel=\"cc:attributionURL dct:creator\" property=\"cc:attributionName\" href=\"https://www.linkedin.com/in/anibalsanchez/\"\u003eAnibal H. Sanchez Perez\u003c/a\u003e is licensed under \u003ca href=\"https://creativecommons.org/licenses/by/4.0/?ref=chooser-v1\" target=\"_blank\" rel=\"license noopener noreferrer\" style=\"display:inline-block;\"\u003eCreative Commons Attribution 4.0 International\u003cimg style=\"height:22px!important;margin-left:3px;vertical-align:text-bottom;\" src=\"https://mirrors.creativecommons.org/presskit/icons/cc.svg?ref=chooser-v1\" alt=\"\"\u003e\u003cimg style=\"height:22px!important;margin-left:3px;vertical-align:text-bottom;\" src=\"https://mirrors.creativecommons.org/presskit/icons/by.svg?ref=chooser-v1\" alt=\"\"\u003e\u003c/a\u003e\u003c/p\u003e\n\nPhoto by \u003ca href=\"https://unsplash.com/@kellysikkema?utm_content=creditCopyText\u0026utm_medium=referral\u0026utm_source=unsplash\"\u003eKelly Sikkema\u003c/a\u003e on \u003ca href=\"https://unsplash.com/photos/traffic-light-sign-underwater-_whs7FPfkwQ?utm_content=creditCopyText\u0026utm_medium=referral\u0026utm_source=unsplash\"\u003eUnsplash\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanibalsanchez%2Fdisaster-response-pipeline-project","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanibalsanchez%2Fdisaster-response-pipeline-project","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanibalsanchez%2Fdisaster-response-pipeline-project/lists"}