{"id":19693389,"url":"https://github.com/semasuka/income-classification","last_synced_at":"2025-07-14T16:06:03.563Z","repository":{"id":129548403,"uuid":"443140396","full_name":"semasuka/Income-classification","owner":"semasuka","description":"Predicting if an individual make more than 50K using different features","archived":false,"fork":false,"pushed_at":"2024-10-19T04:34:08.000Z","size":189457,"stargazers_count":5,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-05T06:08:27.386Z","etag":null,"topics":["aws-s3","binary-classification","data-analysis","data-science","data-visualization","eda","finance-analytics","machine-learning","precision","python","random-forest-classifier","scikit-learn","streamlit"],"latest_commit_sha":null,"homepage":"https://share.streamlit.io/semasuka/Income-classification/income_class_st.py","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/semasuka.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-12-30T17:22:14.000Z","updated_at":"2024-10-19T04:34:12.000Z","dependencies_parsed_at":"2025-05-07T04:42:07.640Z","dependency_job_id":"9a9cea74-3457-4c50-9288-1e7365e873f3","html_url":"https://github.com/semasuka/Income-classification","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/semasuka/Income-classification","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semasuka%2FIncome-classification","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semasuka%2FIncome-classification/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semasuka%2FIncome-classification/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semasuka%2FIncome-classification/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/semasuka","download_url":"https://codeload.github.com/semasuka/Income-classification/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semasuka%2FIncome-classification/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265314363,"owners_count":23745254,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-s3","binary-classification","data-analysis","data-science","data-visualization","eda","finance-analytics","machine-learning","precision","python","random-forest-classifier","scikit-learn","streamlit"],"created_at":"2024-11-11T19:16:43.291Z","updated_at":"2025-07-14T16:06:03.551Z","avatar_url":"https://github.com/semasuka.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"![banner](assets/Income_classification_banner.png)\n\n![Python version](https://img.shields.io/badge/Python%20version-3.10%2B-lightgrey)\n![GitHub last commit](https://img.shields.io/github/last-commit/semasuka/Income-classification)\n![GitHub repo size](https://img.shields.io/github/repo-size/semasuka/Income-classification)\n![Type of ML](https://img.shields.io/badge/Type%20of%20ML-Binary%20Classification-red)\n![Licebse](https://img.shields.io/badge/License-MIT-green)\n[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1wKNNC5ZIEXEWMbgiw-knBXLznTRX6xYH)\n[![Streamlit App](https://static.streamlit.io/badges/streamlit_badge_black_white.svg)](https://share.streamlit.io/semasuka/income-classification/income_class_st.py)\n[![Open Source Love svg1](https://badges.frapsoft.com/os/v1/open-source.svg?v=103)](https://github.com/ellerbrock/open-source-badges/)\n\nBadge [source](https://shields.io/)\n\n# People with the highest education level, and who are either husbands or wifes make more money\n\n\n## Authors\n\n- [@semasuka](https://www.github.com/semasuka)\n\n## Table of Contents\n\n- [Business problem](#business-problem)\n- [Data source](#data-source)\n- [Methods](#methods)\n- [Tech Stack](#tech-stack)\n- [Quick glance at the results](#quick-glance-at-the-results)\n- [Lessons learned and recommendation](#lessons-learned-and-recommendation)\n- [Limitation and what can be improved](#limitation-and-what-can-be-improved)\n- [Run Locally](#run-locally)\n- [Explore the notebook](#explore-the-notebook)\n- [Deployment on streamlit](#deployment-on-streamlit)\n- [App deployed on Streamlit](#app-deployed-on-streamlit)\n- [Repository structure](#repository-structure)\n- [Contribution](#contribution)\n- [License](#license)\n\n\n\n\n## Business problem\n\nThis an app to predict if someone make more or less than 50k/year using different features. \nThis app can be used when that information is not available or is confidential during a loan application at any financial institution or car financing application to have a better financial picture of the applicant.\n## Data source\n\n- [Kaggle Income classification](https://www.kaggle.com/lodetomasi1995/income-classification) (Main dataset)\n- [GDP group dataset](https://www.kaggle.com/nitishabharathi/gdp-per-capita-all-countries) (Dataset used to enriched the main dataset with the countrie's GDP grouping)\n## Methods\n\n- Exploratory data analysis\n- Bivariate analysis\n- Multivariate correlation\n- Feature engineering\n- Feature selection\n- S3 bucket model hosting\n- Model deployment\n## Tech Stack\n\n- Python (refer to requirement.txt for the packages used in this project)\n- Streamlit (interface for the model)\n- AWS S3 (model storage)\n\n\n## Quick glance at the results\n\nMost correlated features to the target.\n\n![heatmap](assets/heatmap.png)\n\nConfusion matrix of random forest (Best estimator with the best parameters)\n\n![Confusion matrix](assets/confusion_matrix.png)\n\nROC curve of random forest (Best estimator with the best parameters)\n\n![ROC curve](assets/roc.png)\n\nTop 5 models after hyper parameter tuning\n\n| Model     \t        | Precision score \t|\n|-------------------\t|------------------\t|\n| Random Forest     \t| 87% \t            |\n| Neural Network    \t| 81% \t            |\n| KNN               \t| 83% \t            |\n| Gradient Boosting \t| 90% \t            |\n| Bagging           \t| 87% \t            |\n\n- ***The final model used is: Random forest classifier***\n- ***Metrics used: Precision (87%)***\n- ***Why choosing random forest yet gradient boosting had 90%, well because Gradient boosting was overfitting***\n- ***Why choose precision as metrics: Because a financial instituation would rather get make sure that the people the loan is given do actually make more than 50k. Even though that it means approving fewer applicants. Thus prioritizing precision over recall.***\n\n\n## Lessons learned and recommendation\n\n- Based on the analysis on this project, we found out that the education level and type of relationship are the most predictive features to determine if someone makes more or less than 50K. Other features like Capital gain, hours work and age are also usefull. The least usefull features are: their occupation and the workclass they belong to.\n- Recommendation would be to focus more on the most predictive feature when looking at the applicant profile, and pay less attention on their occupation and workclass.\n## Limitation and what can be improved\n\n- Speed: since the model is stored on AWS S3, it can take some few seconds to load. Solution: cache the model with the Streamlit @st.experimental_singleton for faster reload.\n- Dataset used: the dataset used is from 1990, inflation has not been taken into consideration and the countries's economies have changed since then. Solution: retrain with a more recent dataset.\n- Hyperparameter tuning: I used RandomeSearchCV to save time but could be improved by couple of % with GridSearchCV.\n\n\n## Run Locally\nInitialize git\n\n```bash\ngit init\n```\n\n\nClone the project\n\n```bash\ngit clone https://github.com/semasuka/Income-classification.git\n```\n\nenter the project directory\n\n```bash\ncd Income-classification\n```\n\nCreate a conda virtual environment and install all the packages from the environment.yml (recommended)\n\n```bash\nconda env create --prefix \u003cenv_name\u003e --file assets/environment.yml\n```\n\nActivate the conda environment\n\n```bash\nconda activate \u003cenv_name\u003e\n```\n\nList all the packages installed\n\n```bash\nconda list\n```\n\nStart the streamlit server locally\n\n```bash\nstreamlit run income_class_st.py\n```\nIf you are having issue with streamlit, please follow [this tutorial on how to set up streamlit](https://docs.streamlit.io/library/get-started/installation)\n\n## Explore the notebook\n\nTo explore the notebook file [here](https://nbviewer.org/github/semasuka/Income-classification/blob/master/Income_Classification.ipynb)\n\n## Deployment on streamlit\n\nTo deploy this project on streamlit share, follow these steps:\n\n- first, make sure you upload your files on Github, including a requirements.txt file\n- go to [streamlit share](https://share.streamlit.io/)\n- login with Github, Google, etc.\n- click on new app button\n- select the Github repo name, branch, python file with the streamlit codes\n- click advanced settings, select python version 3.9 and add the secret keys if your model is stored on AWS or GCP bucket\n- then save and deploy!\n\n## App deployed on Streamlit\n\n![Streamlit GIF](assets/gif_streamlit.gif)\n\nVideo to gif [tool](https://ezgif.com/)\n## Repository structure\n\n\n```\n\n├── assets\n│   ├── confusion_matrix.png        \u003c- confusion matrix image used in the README.\n│   ├── gif_streamlit.gif           \u003c- gif file used in the README.\n│   ├── heatmap.png                 \u003c- heatmap image used in the README.\n│   ├── Income_classification_banner.png   \u003c- banner image used in the README.\n│   ├── environment.yml             \u003c- list of all the dependencies with their versions(for conda environment).\n│   ├── roc.png                     \u003c- ROC image used in the README.\n│\n│\n├── datasets\n│   ├── GDP.csv                     \u003c- the data used to feature engineering/enriched the original data.\n│   ├── test.csv                    \u003c- the test data.\n│   ├── train.csv                   \u003c- the train data.\n│\n│\n├── pandas_profile_file\n│   ├── income_class_profile.html   \u003c- exported panda profile html file.\n│\n│\n├── .gitignore                      \u003c- used to ignore certain folder and files that won't be commit to git.\n│\n│\n├── Income_Classification.ipynb     \u003c- main python notebook where all the analysis and modeling are done.\n│\n│\n├── LICENSE                         \u003c- license file.\n│\n│\n├── income_class_st.py              \u003c- file with the best model and best hyperparameter with streamlit component for rendering the interface.\n│\n│\n├── README.md                       \u003c- this readme file.\n│\n│\n├── requirements.txt                \u003c- list of all the dependencies with their versions(used for Streamlit ).\n\n```\n## Contribution\n\nPull requests are welcome! For major changes, please open an issue first to discuss what you would like to change or contribute.\n\n## License\n\nMIT License\n\nCopyright (c) 2022 Stern Semasuka\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n\nLearn more about [MIT](https://choosealicense.com/licenses/mit/) license\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemasuka%2Fincome-classification","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsemasuka%2Fincome-classification","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemasuka%2Fincome-classification/lists"}