{"id":21089625,"url":"https://github.com/coolmunzi/credit_card_default_prediction","last_synced_at":"2026-04-22T08:31:49.856Z","repository":{"id":202322817,"uuid":"467379593","full_name":"coolmunzi/credit_card_default_prediction","owner":"coolmunzi","description":"End to End Machine Learning project for prediction credit card default in next month. ","archived":false,"fork":false,"pushed_at":"2022-03-08T07:45:52.000Z","size":8380,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-01T02:39:05.279Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/coolmunzi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-03-08T05:43:24.000Z","updated_at":"2022-08-18T03:59:51.000Z","dependencies_parsed_at":"2023-10-20T12:38:04.337Z","dependency_job_id":null,"html_url":"https://github.com/coolmunzi/credit_card_default_prediction","commit_stats":null,"previous_names":["coolmunzi/credit_card_default_prediction"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/coolmunzi/credit_card_default_prediction","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coolmunzi%2Fcredit_card_default_prediction","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coolmunzi%2Fcredit_card_default_prediction/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coolmunzi%2Fcredit_card_default_prediction/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coolmunzi%2Fcredit_card_default_prediction/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/coolmunzi","download_url":"https://codeload.github.com/coolmunzi/credit_card_default_prediction/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coolmunzi%2Fcredit_card_default_prediction/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32127854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-22T07:37:52.372Z","status":"ssl_error","status_checked_at":"2026-04-22T07:37:51.635Z","response_time":58,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-19T21:29:55.873Z","updated_at":"2026-04-22T08:31:49.833Z","avatar_url":"https://github.com/coolmunzi.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Problem Statement\nTo build a classification methodology to determine whether a person defaults the credit card payment for the next month. \n\n## Architecture\n![Application Flow](reference_images/application_flow.png)\n\n## Data Description\nThe client will send data in multiple sets of files in batches at a given location. The data has been extracted from the census bureau. \nThe data contains 32561 instances with the following attributes:\n### Independent Features:\n1. **LIMIT_BAL**: continuous.Credit Limit of the person.\n2. **SEX**: Categorical: 1 = male; 2 = female\n3. **EDUCATION**: Categorical: 1 = graduate school; 2 = university; 3 = high school; 4 = others\n4. **MARRIAGE**: 1 = married; 2 = single; 3 = others\n5. **AGE-num**: continuous. \n6. **PAY_0** to **PAY_6**: History of past payment. We tracked the past monthly payment records (from April to September, 2005)\n7. **BILL_AMT1** to **BILL_AMT6**: Amount of bill statements.\n8. **PAY_AMT1** to **PAY_AMT6**: Amount of previous payments. \n\n### Dependent Feature:\n1. **default payment next month**: Whether a person shall default in the credit card payment or not. Yes = 1, No = 0.\n\nApart from training files, we also require a \"schema\" file from the client, which contains all the relevant information about the training files such as:\nName of the files, Length of Date value in FileName, Length of Time value in FileName, Number of Columns, Name of the Columns, and their datatype.\n\n## Data Validation \nIn this step, we perform different sets of validation on the given set of training files.  \n1. **Name Validation**- We validate the name of the files based on the given name in the schema file. We have created a regex pattern as per the name given in the schema file to use for validation. After validating the pattern in the name, we check for the length of date in the file name as well as the length of time in the file name. If all the values are as per requirement, we move such files to \"Good_Data_Folder\" else we move such files to \"Bad_Data_Folder.\"\n2. **Number of Columns** - We validate the number of columns present in the files, and if it doesn't match with the value given in the schema file, then the file is moved to \"Bad_Data_Folder.\"\n3. **Name of Columns** - The name of the columns is validated and should be the same as given in the schema file. If not, then the file is moved to \"Bad_Data_Folder\".\n4. **The datatype of columns** - The datatype of columns is given in the schema file. It is validated when we insert the files into Database. If the datatype is wrong, then the file is moved to \"Bad_Data_Folder\".\n5. **Null values in columns** - If any of the columns in a file have all the values as NULL or missing, we discard such a file and move it to \"Bad_Data_Folder\".\n\n## Data Insertion in Database\n \n1) **Database Creation and connection** - Create a database with the given name passed. If the database has already been created, open a connection to the database. \n2) **Table creation in the database** - Table with name - \"Good_Data\", is created in the database for inserting the files in the \"Good_Data_Folder\" based on given column names and datatype in the schema file. If the table is already present, then the new table is not created, and new files are inserted in the already present table as we want training to be done on new as well as old training files.     \n3) **Insertion of files in the table** - All the files in the \"Good_Data_Folder\" are inserted in the above-created table. If any file has invalid data type in any of the columns, the file is not loaded in the table and is moved to \"Bad_Data_Folder\".\n\n## Model Training \n1) **Data Export from Db** - The data in a stored database is exported as a CSV file to be used for model training.\n2) **Data Preprocessing**   \n    a) Check for null values in the columns. If present, impute the null values using the categorical imputer.\n    \n    b) Scale the numeric values using the standard scaler.\n\n    c) Check for  correlation.\n3) **Clustering** - KMeans algorithm is used to create clusters in the preprocessed data. The optimum number of clusters is selected by plotting the elbow plot, and for the dynamic selection of the number of clusters, we are using \"KneeLocator\" function. The idea behind clustering is to implement different algorithms. \n   \n    The Kmeans model is trained over preprocessed data, and the model is saved for further use in prediction.\n5) **Model Selection** – After the clusters have been created, we find the best model for each cluster. We are using two algorithms, “Naïve Bayes” and \"XGBoost\". For each cluster, both the algorithms are passed with the best parameters derived from GridSearch. We calculate the AUC scores for both models and select the model with the best score. Similarly, the model is selected for each cluster. All the models for every cluster are saved for use in prediction.\n\n\n\n\n## Prediction Data Description\n The Client will send the data in multiple sets of files in batches at a given location. Data will contain the credit and payment record of customers. \nApart from prediction files, we also require a \"schema\" file from the client, which contains all the relevant information about the training files such as:\nName of the files, Length of Date value in FileName, Length of Time value in FileName, Number of Columns, Name of the Columns and their datatype.\n\n## Data Validation  \nIn this step, we perform different sets of validation on the given set of training files.  \n1) **Name Validation**- We validate the name of the files based on given Name in the schema file. We have created a regex pattern as per the name given in the schema file, to use for validation. After validating the pattern in the name, we check for the length of date in the file name as well as the length of the timestamp in the file name. If all the values are as per requirement, we move such files to \"Good_Data_Folder\" else we move such files to \"Bad_Data_Folder\". \n2) **Number of Columns** - We validate the number of columns present in the files, and if it doesn't match with the value given in the schema file, then the file is moved to \"Bad_Data_Folder\". \n3) **Name of Columns** - The name of the columns is validated and should be same as given in the schema file. If not, then the file is moved to \"Bad_Data_Folder\". \n4) **Datatype of columns** - The datatype of columns is given in the schema file. This is validated when we insert the files into Database. If the datatype is incorrect, then the file is moved to \"Bad_Data_Folder\". \n5) **Null values in columns** - If any of the columns in a file has all the values as NULL or missing, we discard such file and move it to \"Bad_Data_Folder\". \n\n## Data Insertion in Database \n1) **Database Creation and connection** - Create a database with the given name passed. If the database is already created, open the connection to the database. \n2) **Table creation in the database** - Table with name - \"Good_Data\", is created in the database for inserting the files in the \"Good_Data_Folder\" based on given column names and datatype in the schema file. If the table is already present, then a new table is not created, and new files are inserted into the already present table as we want training to be done on new as well old training files.     \n3) **Insertion of files in the table** - All the files in the \"Good_Data_Folder\" are inserted in the above-created table. If any file has invalid data type in any of the columns, the file is not loaded in the table and is moved to \"Bad_Data_Folder\".\n\n## Prediction flow \n1) **Data Export from Db** - The data in the stored database is exported as a CSV file to be used for prediction.\n2) **Data Preprocessing**:  \n    a) Check for null values in the columns. If present, impute the null values using the categorical imputer.\n    b) Scale the numeric values using the standard scaler.\n    c) Check for  correlation.\n3) **Clustering** - KMeans model created during training is loaded, and clusters for the preprocessed prediction data is predicted.\n4) **Prediction** - Based on the cluster number, the respective model is loaded and is used to predict the data for that cluster.\n5) Once the prediction is made for all the clusters, the predictions along with the Wafer names are saved in a CSV file at a given location, and the location is returned to the client.\n\n## Deployment\n\nWe will be deploying the model to the Heroku cloud platform. \nNow let’s look at the project folder structure:\n![Application Flow](reference_images/project_structure.png)\n\nSteps before cloud deployment:\nWe need to change our code a bit so that it works unhindered on the cloud, as well.\n\na) Add a file called ‘Procfile’ inside the ‘reviewScrapper’ folder. This folder contains the command to run the flask application once deployed on the server:\n    \n    web: gunicorn app:app\nHere, the keyword ‘web’ specifies that the application is a web application. And the part ‘app:app’ instructs the program to look for a flask application called ‘app’ inside the ‘app.py’ file. Gunicorn is a Web Server Gateway Interface (WSGI) HTTP server for Python.\n\nb) Open a command prompt window and navigate to your ‘reviewScrapper’ folder. Enter the command ‘pip freeze \u003e requirements.txt’. This command generates the ‘requirements.txt’ file. \nrequirements.txt helps the Heroku cloud app to install all the dependencies before starting the webserver.\n\nc) Deployment to Heroku:\n    \n    • Go to Heroku.com and create an account and login.\n    \n    • Click on new(inside the red box) to create a new app.\n    \n    • Give the name of the app and click ‘create app’.\n    \n    • After app creation, the ‘deploy’ section has all the deployment steps mentioned.\n    \n    • We’ll download and install the Heroku CLI from the Heroku website: https://devcenter.heroku.com/articles/heroku-cli.\n    \n    • Git also needs to be installed in your computer.\n    \n    • After installing the Heroku CLI, Open a command prompt window and navigate to your project folder.\n    \n    • Type the command ‘heroku login’ to login to your heroku account as shown  below:\n\nIt opens up a webpage to login to Heroku.\n \n    • Before deploying the code to the Heroku cloud, we need to commit the changes to the local git repository.\n    \n    • Type the command ‘git init to initialize a local git repository  as shown below:\n    \n    • Enter the command heroku git:remote \u003cremote repo name\u003e to connect local git repo to the remote repo.\n    \n    • Enter the command ‘git status’ to see the uncommitted changes\n    \n    • Enter the command ‘git add .’ to add the uncommitted changes to the local repository.\n    \n    • Enter the command ‘git commit -am \"make it better\"’ to commit the changes to the local repository.\n    \n    • Enter the command ‘git push heroku master’ to push the code to the heroku cloud.\n    \n    • After deployment, heroku gives you the URL to hit the web API.\n\nOnce your application is deployed successfully, enter the command ‘heroku logs --tail’ to see the logs.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoolmunzi%2Fcredit_card_default_prediction","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcoolmunzi%2Fcredit_card_default_prediction","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoolmunzi%2Fcredit_card_default_prediction/lists"}