{"id":20619853,"url":"https://github.com/mafda/disaster_response_pipeline","last_synced_at":"2026-04-22T07:35:19.656Z","repository":{"id":250235029,"uuid":"829203017","full_name":"mafda/disaster_response_pipeline","owner":"mafda","description":"This project involves developing an ETL pipeline integrated with supervised learning and grid search to classify text messages sent during disaster events. It includes an ML pipeline and a web app designed to categorize disaster response messages in real time using NLP techniques","archived":false,"fork":false,"pushed_at":"2024-08-01T05:09:45.000Z","size":1975,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-17T05:09:16.087Z","etag":null,"topics":["data-engineering","data-science","etl-pipeline","flask-application","grid-search-hyperparameters","nlp-keywords-extraction","nlp-machine-learning","supervised-machine-learning"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mafda.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-16T01:09:59.000Z","updated_at":"2024-08-01T05:09:48.000Z","dependencies_parsed_at":"2024-11-16T12:12:49.539Z","dependency_job_id":null,"html_url":"https://github.com/mafda/disaster_response_pipeline","commit_stats":null,"previous_names":["mafda/disaster_response_pipeline"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafda%2Fdisaster_response_pipeline","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafda%2Fdisaster_response_pipeline/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafda%2Fdisaster_response_pipeline/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafda%2Fdisaster_response_pipeline/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mafda","download_url":"https://codeload.github.com/mafda/disaster_response_pipeline/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242277684,"owners_count":20101542,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-engineering","data-science","etl-pipeline","flask-application","grid-search-hyperparameters","nlp-keywords-extraction","nlp-machine-learning","supervised-machine-learning"],"created_at":"2024-11-16T12:12:41.685Z","updated_at":"2026-04-22T07:35:14.620Z","avatar_url":"https://github.com/mafda.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Disaster Response Project\n\nThis project involves developing an ETL pipeline integrated with supervised\nlearning and grid search to classify text messages sent during disaster events. \n\nIt includes an ML pipeline and a web app designed to categorize disaster\nresponse messages in real time using Natural Language Processing (NLP)\ntechniques.\n\n In this project, I'll apply data engineering skills to analyze disaster data\n from [Appen](https://www.figure-eight.com/) to build a model for an API that\n classifies disaster messages.\n\n## Web App\n\n\n| home page                          | results page                            |\n| ---------------------------------- | --------------------------------------- |\n| ![](assets/Disasters_graphics.jpg) | ![](assets/Disasters_model_results.jpg) |\n\n## Project Setup\n\n### Clone this repository\n\n```shell\n(base)$: git clone git@github.com:mafda/disaster_response_project.git\n(base)$: cd disaster_response_project\n```\n\n### Configure environment\n\n- Create the conda environment\n\n    ```shell\n    (base)$: conda env create -f environment.yml\n    ```\n\n- Activate the environment\n\n    ```shell\n    (base)$: conda activate disaster_response\n    ```\n\n- Download the dataset and model from [this\n  link](https://drive.google.com/drive/folders/1uNqCHmE__m9tEV-pgvkBcu4iJFfkaSbN?usp=share_link),\n  create `data` folder and copy the `categories.csv` and `messages.csv` datasets\n  here.\n\n    ```shell\n    (disaster_response)$: mkdir data\n    ```\n\n## Project Structure\n\n```shell\n.\n├── README.md\n├── app\n│   ├── run.py\n│   └── templates\n│       ├── go.html\n│       └── master.html\n├── data\n│   ├── DisasterResponse.db\n│   ├── categories.csv\n│   └── messages.csv\n├── environment.yml\n├── models\n│   └── classifier_model.pkl\n├── notebooks\n│   ├── ETL_Pipeline_Preparation.ipynb\n│   └── ML_Pipeline_Preparation.ipynb\n└── src\n    ├── process_data.py\n    └── train_classifier.py\n```\n\n## Project Components\n\n**1. ETL Pipeline**\n\nThe first part of the data pipeline is the Extract, Transform, and Load process.\nI will read the dataset, clean the data, and then store it in a SQLite database.\n\nFile `src/process_data.py`, contains a data cleaning pipeline that:\n* Loads the `messages.csv` and `categoriescsv` datasets\n* Merges the two datasets\n* Cleans the data\n* Stores it in a SQLite database\n\n**2. ML Pipeline**\n\nThe second part, I will create a machine learning pipeline that uses NLTK, as\nwell as scikit-learn's Pipeline and GridSearchCV to output a final model that\nuses the `message` column to predict classifications for 36 categories\n(multi-output classification). Finally, I will export your model to a pickle\nfile. \n\nFile `src/train_classifier.py`, contains a machine learning pipeline that:\n* Loads data from the SQLite database\n* Splits the dataset into training and test sets\n* Builds a text processing and machine learning pipeline\n* Trains and tunes a model using GridSearchCV\n* Outputs results on the test set\n* Exports the final model as a pickle file\n\n**3. Flask Web App**\n\nIn the last step, I'll display your results in a Flask web app.\n\nFile `app/run.py` will start the web app where users can enter their query and\nthe machine learning model will categorize this event.\n* Show data visualizations using Plotly in the web app.\n\n## Executing Program\n\nThere are 3 steps to follow, from cleaning the data, through training the model,\nand ending with the web app.\n\n### 1. Cleaning data\n\nIn the project directory, run:\n\n```shell\n(disaster_response)$: python src/process_data.py data/messages.csv data/categories.csv data/DisasterResponse.db\n```\n\n![](assets/clean_data.png)\n\n### 2. Training model\n\nAfter the data cleaning process, in the project directory, run:\n\n```shell\n(disaster_response)$: python src/train_classifier.py data/DisasterResponse.db models/classifier_model.pkl\n```\n\n![](assets/training_data.png)\n\n### 3. Web App\n\nGo the `app` directory and run:\n\n```shell\n(disaster_response)$: cd app\n(disaster_response) app $: python run.py\n```\n![](assets/web_app.png)\n\nAnd go to http://localhost:3000\n\n## Model Results\n\n* **Model 0: Basic Pipeline**: This model sets up a basic text classification\n  pipeline using Random Forest for multi-output classification.\n* **Model 1: Grid Search with Hyperparameter Tuning**: This model aims to\n  improve performance by tuning hyperparameters through cross-validation.\n* **Model 2: Feature Union Pipeline**: Purpose: This model allows for more\n  complex feature engineering by using FeatureUnion to potentially combine text\n  features with other types of features in the future.\n\nIn summary, **Model 0** is a straightforward pipeline for text classification.\n**Model 1** enhances this by performing hyperparameter tuning to optimize the\nmodel. **Model 2** introduces a more flexible architecture, paving the way for\nincorporating additional feature sets beyond text.\n\nBased on the provided results, here's a detailed analysis to determine which\nmodel has the best performance:\n\n| Model                                           | F1 Score Mean | Standard Deviation |\n| ----------------------------------------------- | ------------- | ------------------ |\n| Model 0: Basic Pipeline                         | 0.935306      | 0.056039           |\n| Model 1: Grid Search with Hyperparameter Tuning | 0.931143      | 0.061321           |\n| Model 2: Feature Union Pipeline                 | 0.935085      | 0.056090           |\n  \n### Analysis\n\nF1 Score Mean:\n\n* Model 0 has the highest mean F1 score (0.935306), closely followed by Model 2\n  (0.935085).\n* Model 1 has the lowest mean F1 score (0.931143).\n\nStandard Deviation:\n\n* Model 0 has a standard deviation of 0.056039.\n* Model 2 has a slightly higher standard deviation of 0.056090.\n* Model 1 has the highest standard deviation (0.061321), indicating more\n  variability in performance.\n\n* **Best Performance:** Model 0 has the best performance with the highest mean\n  F1 score (0.935306) and a relatively low standard deviation (0.056039),\n  indicating both strong and consistent performance.\n* **Second Best:** Model 2 is very close in performance to Model 0, with a\n  nearly identical mean F1 score (0.935085) and a slightly higher standard\n  deviation (0.056090).\n* **Least Preferred:** Model 1 has the lowest mean F1 score (0.931143) and the\n  highest standard deviation (0.061321), making it the least preferred model in\n  terms of both average performance and consistency.\n\n## Conclusion\n\nI have chosen **Model 1** as the best model for the following reasons:\n\n* Comparable Performance: The performance metrics are very similar across all\n  three models. **Model 1** achieves a mean F1 score of 0.931143 with a standard\n  deviation of 0.061321, which is very close to the scores of Model 1 (mean F1\n  score of 0.935306 and standard deviation of 0.056039) and Model 3 (mean F1\n  score of 0.935085 and standard deviation of 0.056090). This indicates that\n  **Model 1** performs at a similar level of accuracy and consistency as the other\n  models.\n* Grid Search Optimization: **Model 1** utilizes GridSearchCV for hyperparameter\n  tuning, which systematically searches for the best combination of\n  hyperparameters to optimize model performance. This thorough approach ensures\n  that the model is well-tuned and can potentially achieve better performance in\n  different scenarios. The use of grid search demonstrates a more rigorous and\n  comprehensive method for model optimization.\n  \nGiven the similar performance across the models and the added advantage of grid\nsearch optimization, **Model 1** stands out as the most robust and well-tuned choice\nfor this project.\n\n## References\n\n- [Data Scientist Nanodegree\n  Program](https://www.udacity.com/course/data-scientist-nanodegree--nd025)\n\n---\n\nmade with 💙 by [mafda](https://mafda.github.io/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmafda%2Fdisaster_response_pipeline","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmafda%2Fdisaster_response_pipeline","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmafda%2Fdisaster_response_pipeline/lists"}