{"id":22647771,"url":"https://github.com/davidhintelmann/heart-disease","last_synced_at":"2025-06-30T08:06:10.422Z","repository":{"id":156030363,"uuid":"295596779","full_name":"davidhintelmann/Heart-Disease","owner":"davidhintelmann","description":"UCI Dataset, Machine Learning, predict if patient has heart disease","archived":false,"fork":false,"pushed_at":"2020-10-01T03:25:38.000Z","size":1641,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-03T20:03:20.446Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/davidhintelmann.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-15T02:49:43.000Z","updated_at":"2020-10-01T03:25:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"1e02ea68-f621-4798-b997-0dcbfc9da546","html_url":"https://github.com/davidhintelmann/Heart-Disease","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidhintelmann%2FHeart-Disease","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidhintelmann%2FHeart-Disease/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidhintelmann%2FHeart-Disease/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidhintelmann%2FHeart-Disease/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/davidhintelmann","download_url":"https://codeload.github.com/davidhintelmann/Heart-Disease/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246150409,"owners_count":20731419,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-09T07:34:33.746Z","updated_at":"2025-03-29T06:48:15.537Z","avatar_url":"https://github.com/davidhintelmann.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Supervised Machine Learning Project: Predicting Heart Disease\n\n# Summary\n\nCardiovascular diseases (CVDs) or heart diseases are the most prevalent disease globally. According to the World Health Organization (WHO), heart disease is the primary cause of death around the world and is estimated to cause about 17.9 million deaths a year, with four out of five deaths due to heart attacks and strokes\n\nThis is notebook was created investigate heart disease using a heart disease dataset from UC Irvine Machine Learning Repository. I will be using the processed Cleveland dataset since it has already been recduced from the original 76 attributes to 14. The goal is to predict if a patient has heart disease or not based on 13 features by creating multiple supervised machine learning models. Before this is done the data will be cleaned to find any missing values and potentially scaled or transformed to improved the ML model.\n\n# Objective\n\nGiven the prevalence of heart disease and its impact on global health, this report intends to examine data collected as part of the Cleveland dataset and explored the following:\n\n* What are the distributions of the variables such as sex, age, blood pressure, etc. - also explore how meaningful these factors are by applying and interpreting the results of statistical tests.  \n* Is there a difference in presence of heart disease in men and women? The null hypothesis is that there is no difference between sexes.  \n* What are the relationships between the variables and the prevalence of heart disease? Through our analysis, we intend to discuss these variables and draw conclusions on their impact on the presence of heart disease.  \n\nFinally to develop various machine learning models to predict what patient has heart disease, and then compare different models to one another while checking individual models scoring metrics to comfirm the quality of the model created.\n\n# Import all Python Libraries\n\n\n```python\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nimport statsmodels.api as sm\nfrom statsmodels.formula.api import ols\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.svm import SVC\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.model_selection import GridSearchCV\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.preprocessing import PowerTransformer\nfrom sklearn.preprocessing import MinMaxScaler\nfrom sklearn.preprocessing import RobustScaler\nfrom scipy import stats\n\nfrom matplotlib import rcParams\nrcParams['figure.figsize'] = 10,8\nsns.set_theme()\n```\n\n## Read csv file into Pandas Dataframe\n\n\n```python\ncolumns = ['age','sex','cp','trestbps','chol','fbs','restecg','thalach','exang','oldpeak','slope','ca','thal','num']\ndf = pd.read_csv('data.csv', names=columns)\n```\n\n\n```python\ndf.head()\n```\n\n\n\n\n\u003cdiv\u003e\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eage\u003c/th\u003e\n      \u003cth\u003esex\u003c/th\u003e\n      \u003cth\u003ecp\u003c/th\u003e\n      \u003cth\u003etrestbps\u003c/th\u003e\n      \u003cth\u003echol\u003c/th\u003e\n      \u003cth\u003efbs\u003c/th\u003e\n      \u003cth\u003erestecg\u003c/th\u003e\n      \u003cth\u003ethalach\u003c/th\u003e\n      \u003cth\u003eexang\u003c/th\u003e\n      \u003cth\u003eoldpeak\u003c/th\u003e\n      \u003cth\u003eslope\u003c/th\u003e\n      \u003cth\u003eca\u003c/th\u003e\n      \u003cth\u003ethal\u003c/th\u003e\n      \u003cth\u003enum\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003e63.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e145.0\u003c/td\u003e\n      \u003ctd\u003e233.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e150.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e2.3\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e6.0\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003e67.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e160.0\u003c/td\u003e\n      \u003ctd\u003e286.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e108.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e1.5\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e2\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003e67.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e120.0\u003c/td\u003e\n      \u003ctd\u003e229.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e129.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e2.6\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e7.0\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003e37.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e130.0\u003c/td\u003e\n      \u003ctd\u003e250.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e187.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e3.5\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e4\u003c/th\u003e\n      \u003ctd\u003e41.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e130.0\u003c/td\u003e\n      \u003ctd\u003e204.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e172.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e1.4\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n# Data Preperation\n\nA key rationale for the choice of the dataset (Cleveland rather than the other three) was the cleanliness of the original dataset as it would better lend to initial exploratory analysis and the development of a potential model.\nThe data preparation process was as follows:\n\n* Acquire data: Reading data from the aforementioned sources;\n\n* Set up the environment: In Pandas, libraries were imported with a comprehensive setup to run data frames and visualisation as well as statistics analysis;\n\n* Prepare data for analysis:  \n    -Imported the data  \n    -Identified variables  \n    -Cleaned undesired columns, checked for nulls  \n    -Changed datatypes to numeric  \n    \n    \n* Develop more variables for analysis: Created new data frame or changed existed ones as necessary for more in-depth analysis if needed:  \n-Mapped multiple categories of disease to a old feature, 'num', which is either healthy (0) or has signs of heart disease (1). Previously this has a value from zero to four, though we are changing the problem to a binary classification problem   \n\n\n* Perform analysis: Performed quantitative data analysis (Discrete) or Exploratory Data Analysis (EDA) for the data that covers heart disease.\n\n\n```python\ndf.info()\n```\n\n    \u003cclass 'pandas.core.frame.DataFrame'\u003e\n    RangeIndex: 303 entries, 0 to 302\n    Data columns (total 14 columns):\n     #   Column    Non-Null Count  Dtype  \n    ---  ------    --------------  -----  \n     0   age       303 non-null    float64\n     1   sex       303 non-null    float64\n     2   cp        303 non-null    float64\n     3   trestbps  303 non-null    float64\n     4   chol      303 non-null    float64\n     5   fbs       303 non-null    float64\n     6   restecg   303 non-null    float64\n     7   thalach   303 non-null    float64\n     8   exang     303 non-null    float64\n     9   oldpeak   303 non-null    float64\n     10  slope     303 non-null    float64\n     11  ca        303 non-null    object \n     12  thal      303 non-null    object \n     13  num       303 non-null    int64  \n    dtypes: float64(11), int64(1), object(2)\n    memory usage: 33.3+ KB\n\n\n---\n\nColumns 'ca' and 'thal' have dtype object but appear to be numerical values from the third cell above.\n\nWe notice below that there are rows with values filled in with '?', specifically 'ca' and 'thal' columns.\n\n\n```python\ndf[df['thal'] == '?']\n```\n\n\n\n\n\u003cdiv\u003e\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eage\u003c/th\u003e\n      \u003cth\u003esex\u003c/th\u003e\n      \u003cth\u003ecp\u003c/th\u003e\n      \u003cth\u003etrestbps\u003c/th\u003e\n      \u003cth\u003echol\u003c/th\u003e\n      \u003cth\u003efbs\u003c/th\u003e\n      \u003cth\u003erestecg\u003c/th\u003e\n      \u003cth\u003ethalach\u003c/th\u003e\n      \u003cth\u003eexang\u003c/th\u003e\n      \u003cth\u003eoldpeak\u003c/th\u003e\n      \u003cth\u003eslope\u003c/th\u003e\n      \u003cth\u003eca\u003c/th\u003e\n      \u003cth\u003ethal\u003c/th\u003e\n      \u003cth\u003enum\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e87\u003c/th\u003e\n      \u003ctd\u003e53.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e128.0\u003c/td\u003e\n      \u003ctd\u003e216.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e115.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e?\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e266\u003c/th\u003e\n      \u003ctd\u003e52.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e128.0\u003c/td\u003e\n      \u003ctd\u003e204.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e156.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e?\u003c/td\u003e\n      \u003ctd\u003e2\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\ndf[df['ca'] == '?']\n```\n\n\n\n\n\u003cdiv\u003e\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003eage\u003c/th\u003e\n      \u003cth\u003esex\u003c/th\u003e\n      \u003cth\u003ecp\u003c/th\u003e\n      \u003cth\u003etrestbps\u003c/th\u003e\n      \u003cth\u003echol\u003c/th\u003e\n      \u003cth\u003efbs\u003c/th\u003e\n      \u003cth\u003erestecg\u003c/th\u003e\n      \u003cth\u003ethalach\u003c/th\u003e\n      \u003cth\u003eexang\u003c/th\u003e\n      \u003cth\u003eoldpeak\u003c/th\u003e\n      \u003cth\u003eslope\u003c/th\u003e\n      \u003cth\u003eca\u003c/th\u003e\n      \u003cth\u003ethal\u003c/th\u003e\n      \u003cth\u003enum\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e166\u003c/th\u003e\n      \u003ctd\u003e52.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e138.0\u003c/td\u003e\n      \u003ctd\u003e223.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e169.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e?\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e192\u003c/th\u003e\n      \u003ctd\u003e43.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e4.0\u003c/td\u003e\n      \u003ctd\u003e132.0\u003c/td\u003e\n      \u003ctd\u003e247.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e143.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e0.1\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e?\u003c/td\u003e\n      \u003ctd\u003e7.0\u003c/td\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e287\u003c/th\u003e\n      \u003ctd\u003e58.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e125.0\u003c/td\u003e\n      \u003ctd\u003e220.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e144.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e0.4\u003c/td\u003e\n      \u003ctd\u003e2.0\u003c/td\u003e\n      \u003ctd\u003e?\u003c/td\u003e\n      \u003ctd\u003e7.0\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e302\u003c/th\u003e\n      \u003ctd\u003e38.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e138.0\u003c/td\u003e\n      \u003ctd\u003e175.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e173.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e0.0\u003c/td\u003e\n      \u003ctd\u003e1.0\u003c/td\u003e\n      \u003ctd\u003e?\u003c/td\u003e\n      \u003ctd\u003e3.0\u003c/td\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\nWe have two rows with unknown 'thal' values and four rows with unknown 'ca' values. These rows will be dropped for a cleaner dataset.\n\n\n```python\ndrop_rows = [*df[df['thal'] == '?'].index.values, *df[df['ca'] == '?'].index.values]\ndf.drop(drop_rows, axis=0, inplace=True)\n```\n\nWe also change the dtype of columns 'thal' and 'ca' to 'float64'.\n\n\n```python\ndf[['thal','ca']] = df[['thal','ca']].astype('float64')\n```\n\nSince we are only trying to predict if a patient has heart disease we will change the values in the 'num' column to a binary value of 0 or 1. Since this column represents the diagnosis of heart disease (angiographic disease status) where a value 0: \u003c 50% diameter narrowing and a value greater than 1: \u003e 50% diameter narrowing then any number great than zero will be changed to a one.\n\nWe also create a copy of the Dataframe below so we can test to see if our transformations improve our models prediction ability.\n\n\n```python\ndf.loc[(df.num \u003e 0),'num']=1\ndf_T = df.copy()\n```\n\n# Exploratory Data Analysis\n\nThe 14 attributes in this dataset are:\n1. age - numerical \n2. sex - cateforical 0-female, 1-male\n3. cp - chest pain, categorical 1-typical angina, 2-atypical angina, 3-non-anginal pain, 4-asymptomatic \n4. trestbps - resting blood pressure (in mm Hg on admission to the hospital), numerical\n5. chol - serum cholestorel in mg/dl, numerical \n6. fbs - fasting blood sugar \u003e 120 mg/dl, binary 0-false, 1-true \n7. restecg - resting electrocardiographic results, categorical 0-normal, 1-having ST-T wave abnormality (T wave inversions and/or ST elevation or depression of \u003e 0.05 mV), 2-showing probable or definite left ventricular hypertrophy by Estes' criteria \n8. thalach - maximum heart rate achieved, numerical\n9. exang - exercise induced angina, binary 0-false, 1-true\n10. oldpeak - ST depression induced by exercise relative to rest, numerical\n11. slope - the slope of the peak exercise ST segment, categorical 0-upsloping, 1-flat, 2-downsloping\n12. ca - number of major vessels (0-3) colored by flourosopy, categorical \n13. thal - categorical  3-normal, 6-fixed defect, 7-reversable defect \n14. num - target variable, originally had value 0-4 but now only 0 or 1 as explained in section above.\n\n\n\n```python\nsns.displot(df, x=\"age\", hue='sex', height=8, aspect=10/8)\nplt.title('Figure 1. Distribution of Ages for each Sex');\n```\n\n\n![png](plot_imgs/output_25_0.png)\n\n\n\n```python\nnum_m = len(df[df['sex'] == 1.0]) # number of males\nnum_f = len(df[df['sex'] == 0.0]) # number of females\nprint('Number of males in study are {} out of {}'.format(num_m,len(df)))\nprint('Number of females in study are {} out of {}'.format(num_f,len(df)))\n```\n\n    Number of males in study are 201 out of 297\n    Number of females in study are 96 out of 297\n\n\nWe can see most of the patients in this dataset are males. It is also a small dataset with only 297 patients, which is too small by modern 'big data' standards.\n\nIn figure 2 we note there is a much higher chance of males having heart disease at 55.72%, and lower for females at 26.04%.\n\n\n```python\nsns.countplot(x='sex', hue=\"num\", data=df)\nplt.title('Figure 2. Ratio for each sex having signs of Heart Disease');\n```\n\n\n![png](plot_imgs/output_28_0.png)\n\n\n\n```python\nnum_md = len(df[(df['num'] \u003e 0) \u0026 (df['sex'] == 1.0)]) # males with heart disease\nnum_fd = len(df[(df['num'] \u003e 0) \u0026 (df['sex'] == 0.0)]) # females with heart disease\nprint('Percentage of males in study with heart disease is {}%'.format(round(num_md/num_m*100,2)))\nprint('Percentage of females in study with heart disease is {}%'.format(round(num_fd/num_f*100,2)))\n```\n\n    Percentage of males in study with heart disease is 55.72%\n    Percentage of females in study with heart disease is 26.04%\n\n\n\n```python\ndf.corr()[['num']].sort_values(by='num',ascending=False)\n```\n\n\n\n\n\u003cdiv\u003e\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003enum\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003enum\u003c/th\u003e\n      \u003ctd\u003e1.000000\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003ethal\u003c/th\u003e\n      \u003ctd\u003e0.526640\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003eca\u003c/th\u003e\n      \u003ctd\u003e0.463189\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003eoldpeak\u003c/th\u003e\n      \u003ctd\u003e0.424052\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003eexang\u003c/th\u003e\n      \u003ctd\u003e0.421355\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003ecp\u003c/th\u003e\n      \u003ctd\u003e0.408945\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003eslope\u003c/th\u003e\n      \u003ctd\u003e0.333049\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003esex\u003c/th\u003e\n      \u003ctd\u003e0.278467\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003eage\u003c/th\u003e\n      \u003ctd\u003e0.227075\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003erestecg\u003c/th\u003e\n      \u003ctd\u003e0.166343\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003etrestbps\u003c/th\u003e\n      \u003ctd\u003e0.153490\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003echol\u003c/th\u003e\n      \u003ctd\u003e0.080285\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003efbs\u003c/th\u003e\n      \u003ctd\u003e0.003167\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003ethalach\u003c/th\u003e\n      \u003ctd\u003e-0.423817\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\n\n\n```python\nsns.set_palette(\"viridis\")\nsns.displot(x=\"trestbps\", hue='sex', data=df, height=8, aspect=10/8)\nplt.title('Figure 3. Distribution of Resting Blood Pressure for each sex');\n```\n\n\n![png](plot_imgs/output_31_0.png)\n\n\n\n```python\nsns.set_palette(\"cubehelix\")\ndf[(df['trestbps'] \u003e 0) \u0026 (df['sex'] == 1.0)].trestbps.plot(kind='hist')\nplt.title('Figure 4. Distribution of Resting Blood Pressure for Men')\nplt.xlabel('Resting Blood Pressure');\n```\n\n\n![png](plot_imgs/output_32_0.png)\n\n\n\n```python\nsns.set_palette(\"flare_r\")\ndf[(df['trestbps'] \u003e 0) \u0026 (df['sex'] == 0.0)].trestbps.plot(kind='hist')\nplt.title('Figure 5. Distribution of Resting Blood Pressure for Women')\nplt.xlabel('Resting Blood Pressure');\n```\n\n\n![png](plot_imgs/output_33_0.png)\n\n\n\n```python\nsns.set_palette(\"cubehelix\")\ndf[(df['chol'] \u003e 0) \u0026 (df['sex'] == 1.0)].trestbps.plot(kind='hist', bins=11)\nplt.title('Figure 6. Distribution of Serum Cholesterol for Men')\nplt.xlabel('Serum Cholesterol in mg/dl');\n```\n\n\n![png](plot_imgs/output_34_0.png)\n\n\n\n```python\nsns.set_palette(\"flare_r\")\ndf[(df['chol'] \u003e 0) \u0026 (df['sex'] == 0.0)].trestbps.plot(kind='hist', bins=11)\nplt.title('Figure 7. Distribution of Serum Cholesterol for Women')\nplt.xlabel('Serum Cholesterol in mg/dl');\n```\n\n\n![png](plot_imgs/output_35_0.png)\n\n\n\n```python\nsns.set_palette(\"cubehelix\")\ndf[(df['thalach'] \u003e 0) \u0026 (df['sex'] == 1.0)].trestbps.plot(kind='hist', bins=14)\nplt.title('Figure 8. Distribution of Maximum Heart Rate Achieved for Men')\nplt.xlabel('Maximum Heart Rate Achieved');\n```\n\n\n![png](plot_imgs/output_36_0.png)\n\n\n\n```python\nsns.set_palette(\"flare_r\")\ndf[(df['thalach'] \u003e 0) \u0026 (df['sex'] == 0.0)].trestbps.plot(kind='hist', bins=14)\nplt.title('Figure 9. Distribution of Maximum Heart Rate Achieved for Women')\nplt.xlabel('Maximum Heart Rate Achieved');\n```\n\n\n![png](plot_imgs/output_37_0.png)\n\n\n# Statistics and Transformations\n\n\n```python\nstat = ols(formula='trestbps ~ age', data=df).fit()\n```\n\n\n```python\nstat.summary()\n```\n\n\n\n\n\u003ctable class=\"simpletable\"\u003e\n\u003ccaption\u003eOLS Regression Results\u003c/caption\u003e\n\u003ctr\u003e\n  \u003cth\u003eDep. Variable:\u003c/th\u003e        \u003ctd\u003etrestbps\u003c/td\u003e     \u003cth\u003e  R-squared:         \u003c/th\u003e \u003ctd\u003e   0.084\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eModel:\u003c/th\u003e                   \u003ctd\u003eOLS\u003c/td\u003e       \u003cth\u003e  Adj. R-squared:    \u003c/th\u003e \u003ctd\u003e   0.081\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eMethod:\u003c/th\u003e             \u003ctd\u003eLeast Squares\u003c/td\u003e  \u003cth\u003e  F-statistic:       \u003c/th\u003e \u003ctd\u003e   27.18\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eDate:\u003c/th\u003e             \u003ctd\u003eSat, 19 Sep 2020\u003c/td\u003e \u003cth\u003e  Prob (F-statistic):\u003c/th\u003e \u003ctd\u003e3.48e-07\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eTime:\u003c/th\u003e                 \u003ctd\u003e17:51:06\u003c/td\u003e     \u003cth\u003e  Log-Likelihood:    \u003c/th\u003e \u003ctd\u003e -1262.3\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eNo. Observations:\u003c/th\u003e      \u003ctd\u003e   297\u003c/td\u003e      \u003cth\u003e  AIC:               \u003c/th\u003e \u003ctd\u003e   2529.\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eDf Residuals:\u003c/th\u003e          \u003ctd\u003e   295\u003c/td\u003e      \u003cth\u003e  BIC:               \u003c/th\u003e \u003ctd\u003e   2536.\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eDf Model:\u003c/th\u003e              \u003ctd\u003e     1\u003c/td\u003e      \u003cth\u003e                     \u003c/th\u003e     \u003ctd\u003e \u003c/td\u003e   \n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eCovariance Type:\u003c/th\u003e      \u003ctd\u003enonrobust\u003c/td\u003e    \u003cth\u003e                     \u003c/th\u003e     \u003ctd\u003e \u003c/td\u003e   \n\u003c/tr\u003e\n\u003c/table\u003e\n\u003ctable class=\"simpletable\"\u003e\n\u003ctr\u003e\n      \u003ctd\u003e\u003c/td\u003e         \u003cth\u003ecoef\u003c/th\u003e     \u003cth\u003estd err\u003c/th\u003e      \u003cth\u003et\u003c/th\u003e      \u003cth\u003eP\u003e|t|\u003c/th\u003e  \u003cth\u003e[0.025\u003c/th\u003e    \u003cth\u003e0.975]\u003c/th\u003e  \n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eIntercept\u003c/th\u003e \u003ctd\u003e  100.5966\u003c/td\u003e \u003ctd\u003e    6.046\u003c/td\u003e \u003ctd\u003e   16.640\u003c/td\u003e \u003ctd\u003e 0.000\u003c/td\u003e \u003ctd\u003e   88.699\u003c/td\u003e \u003ctd\u003e  112.494\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eage\u003c/th\u003e       \u003ctd\u003e    0.5701\u003c/td\u003e \u003ctd\u003e    0.109\u003c/td\u003e \u003ctd\u003e    5.214\u003c/td\u003e \u003ctd\u003e 0.000\u003c/td\u003e \u003ctd\u003e    0.355\u003c/td\u003e \u003ctd\u003e    0.785\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\u003ctable class=\"simpletable\"\u003e\n\u003ctr\u003e\n  \u003cth\u003eOmnibus:\u003c/th\u003e       \u003ctd\u003e21.250\u003c/td\u003e \u003cth\u003e  Durbin-Watson:     \u003c/th\u003e \u003ctd\u003e   1.894\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eProb(Omnibus):\u003c/th\u003e \u003ctd\u003e 0.000\u003c/td\u003e \u003cth\u003e  Jarque-Bera (JB):  \u003c/th\u003e \u003ctd\u003e  26.106\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eSkew:\u003c/th\u003e          \u003ctd\u003e 0.569\u003c/td\u003e \u003cth\u003e  Prob(JB):          \u003c/th\u003e \u003ctd\u003e2.14e-06\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003cth\u003eKurtosis:\u003c/th\u003e      \u003ctd\u003e 3.902\u003c/td\u003e \u003cth\u003e  Cond. No.          \u003c/th\u003e \u003ctd\u003e    338.\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\u003cbr/\u003e\u003cbr/\u003eNotes:\u003cbr/\u003e[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n\n\n\n\n```python\ntmp = stat.params[1]*df['age'] + stat.params[0]\n\nsns.set_palette(\"crest\")\nplt.scatter(df['age'], df['trestbps'])\nplt.plot(df['age'], tmp, c='r')\nplt.xlabel('Age')\nplt.ylabel('Resting Blood Pressure')\nplt.title('Figure 10. Resting Blood Pressure vs. Age');\n```\n\n\n![png](plot_imgs/output_41_0.png)\n\n\nAbove a linear regression has been attempted on predicting one's resting blood pressure based on their age. However it is shown below that one of the assumptions for this model has been violated, 'Constant variance'. From Wikipedia's article on Linear Regression, https://en.wikipedia.org/wiki/Linear_regression, one of the assumptions is the residuals will be normal distributed. We see in the histogram, Figure 11, that there is a negative skew to this distribution and is not normally distributed, and the Kurtosis is approximately one when it should be closer to three. \n\nAlso the R\u003csup\u003e2\u003c/sup\u003e value for this linear regression is only 0.084 which is a poor fit indead. One of the reasons machine learning has become so important is to come up with more sophiscated models to predict ones medical condition. If we are having a hard time predicting a person's blood pressure, than predicting if they have heart disease will be orders of magnitude more difficult!\n\n\n```python\ntmp_df = tmp - df['trestbps']\ntmp_df.hist(bins=20)\nplt.title('Figure 11. Residuals of Linear Regression in Figure 10')\nplt.xlabel('Residuals')\nplt.ylabel('Frequency')\ntmp_df.describe()\n```\n\n\n\n\n    count    2.970000e+02\n    mean     4.406800e-14\n    std      1.699691e+01\n    min     -6.747517e+01\n    25%     -1.032591e+01\n    50%      8.143871e-01\n    75%      1.194570e+01\n    max      3.879644e+01\n    dtype: float64\n\n\n\n\n![png](plot_imgs/output_43_1.png)\n\n\n\n```python\ntmp_df.skew()\n```\n\n\n\n\n    -0.5722868823804681\n\n\n\n\n```python\ntmp_df.kurtosis()\n```\n\n\n\n\n    0.937352812271532\n\n\n\nScatter Plot below takes a while to run and can be skipped.\n\n\n```python\npd.plotting.scatter_matrix(df, figsize=(20,16));\n```\n\n\n![png](plot_imgs/output_47_0.png)\n\n\n## Addressing Skew and Scaling Input Variables\n\nMany machine learning models benefit from having their input variables being having a normal distrubtion. We see along the diagonal of the scatter matrix plot above that some columns do not have a normal distribution, these are trestbps, thalch, chol, and oldpeak. We see their skew below\n\n\n```python\ndf['trestbps'].skew()\n#df['trestbps'].kurtosis()\n```\n\n\n\n\n    0.7000697177568133\n\n\n\n\n```python\ndf['thalach'].skew()\n```\n\n\n\n\n    -0.5365400799355459\n\n\n\n\n```python\ndf['chol'].skew()\n```\n\n\n\n\n    1.1180955225671279\n\n\n\n\n```python\ndf['oldpeak'].skew()\n```\n\n\n\n\n    1.2471313241482946\n\n\n\nWe can use scipy stats module which will compute boxcox transformations (power transformation) on our skewed data (thalach, trestbps, and chol) and a yeojohnson transformation on our 'oldpeak' data. I would have used boxcox for all the data but 'oldpeak' has values that equal zero and boxcox can not handle values equal or less than zero.\n\n\n```python\nsci_tmp = stats.boxcox_normmax(df['trestbps'].values)\n```\n\n    /Users/DavidH/anaconda2/envs/py382/lib/python3.8/site-packages/scipy/stats/stats.py:3845: PearsonRConstantInputWarning: An input array is constant; the correlation coefficent is not defined.\n      warnings.warn(PearsonRConstantInputWarning())\n\n\n\n```python\nsci_tt = stats.boxcox(df['trestbps'].values, sci_tmp)\n```\n\n\n```python\nstats.skew(sci_tt)\n```\n\n\n\n\n    0.0037114292536530053\n\n\n\n\n```python\nplt.hist(sci_tt);\nplt.title('Figure 12. Distribution of Scaled Resting Blood Pressure')\nplt.xlabel('Scaled Resting Blood Pressure')\nplt.ylabel('Frequency');\n```\n\n\n![png](plot_imgs/output_58_0.png)\n\n\nCompared to original distribution shown below the transformation above is now closer to a Gaussian distribution.\n\n\n```python\ndf['trestbps'].hist()\nplt.title('Figure 13. Distribution of Original Resting Blood Pressure')\nplt.xlabel('Resting Blood Pressure')\nplt.ylabel('Frequency');\n```\n\n\n![png](plot_imgs/output_60_0.png)\n\n\n\n```python\ndf['oldpeak'].describe()\n```\n\n\n\n\n    count    297.000000\n    mean       1.055556\n    std        1.166123\n    min        0.000000\n    25%        0.000000\n    50%        0.800000\n    75%        1.600000\n    max        6.200000\n    Name: oldpeak, dtype: float64\n\n\n\n\n```python\nsns.set_palette(\"viridis\")\nbox_peak = stats.yeojohnson_normmax(df['oldpeak'].values)\nbox_peak = stats.yeojohnson(df['oldpeak'].values, box_peak)\ndf_T['oldpeak'] = box_peak\nplt.hist(box_peak);\n```\n\n\n![png](plot_imgs/output_62_0.png)\n\n\n\n```python\nstats.skew(box_peak)\n```\n\n\n\n\n    0.11474944210994208\n\n\n\n\n```python\ndf['oldpeak'].hist()\n```\n\n\n\n\n    \u003cmatplotlib.axes._subplots.AxesSubplot at 0x12fb19490\u003e\n\n\n\n\n![png](plot_imgs/output_64_1.png)\n\n\n\n```python\nsns.set_palette(\"crest\")\nbox_thal = stats.boxcox_normmax(df['thalach'].values)\nbox_thal = stats.boxcox(df['thalach'].values, box_thal)\ndf_T['thalach'] = box_thal\nplt.hist(box_thal);\n```\n\n\n![png](plot_imgs/output_65_0.png)\n\n\n\n```python\ndf['thalach'].hist();\n```\n\n\n![png](plot_imgs/output_66_0.png)\n\n\n\n```python\nstats.skew(box_thal)\n```\n\n\n\n\n    -0.02153700558620656\n\n\n\n\n```python\nsns.set_palette(\"viridis\")\nbox_chol = stats.boxcox_normmax(df['chol'].values)\nbox_chol = stats.boxcox(df['chol'].values, box_chol)\ndf_T['chol'] = box_chol\nplt.hist(box_chol);\n```\n\n    /Users/DavidH/anaconda2/envs/py382/lib/python3.8/site-packages/scipy/stats/stats.py:3845: PearsonRConstantInputWarning: An input array is constant; the correlation coefficent is not defined.\n      warnings.warn(PearsonRConstantInputWarning())\n\n\n\n![png](plot_imgs/output_68_1.png)\n\n\n\n```python\nplt.hist(df['chol']);\n```\n\n\n![png](plot_imgs/output_69_0.png)\n\n\n\n```python\nstats.skew(box_chol)\n```\n\n\n\n\n    -0.04094429306060899\n\n\n\nLastly we use RobustScaler function from sklean preprocessing module which is meant to good at dealing with outliers when scaling the data. It does this by removing the median and scales the data according to the quantile range\n\n\n```python\nscaler = RobustScaler()\ncollist = df_T.columns.tolist()\ncollist.remove('num')\n#df_T[['thalach','oldpeak','chol','trestbps']] = scaler.fit_transform(df_T[['thalach','oldpeak','chol','trestbps']])\ndf_T[collist] = scaler.fit_transform(df_T[collist])\n```\n\n## Dummy Variables\n\nAlso need dummy variables in our dataframe for all the categorical variables in the dataset. This will help the machine learning models handle categorical data.\n\n\n```python\ncat_var = ['cp','restecg','slope','thal','ca']\ndata = pd.get_dummies(df, prefix=cat_var, columns=cat_var)\ndata_T = pd.get_dummies(df_T, prefix=cat_var, columns=cat_var)\n```\n\n# Models without Transformations\n\n\n```python\nX_train, X_test, y_train, y_test = train_test_split(data.drop('num',axis=1), data['num'], test_size=0.2, random_state=42)\n```\n\n## K Neighbors Classifier\n\n\n```python\nneigh = KNeighborsClassifier()\nparameters = {\n    'n_neighbors':[2,3,4,5,6,7,8,9],\n    'weights':('uniform','distance'),\n    'algorithm':('auto', 'ball_tree', 'kd_tree', 'brute'),\n    'leaf_size':[15,20,25,30,35,40,45],\n    'p':[1,2,3],\n}\nclf = GridSearchCV(neigh, parameters, cv=5, n_jobs=-1, verbose=5)\nclf.fit(X_train, y_train)\n```\n\n    Fitting 5 folds for each of 1344 candidates, totalling 6720 fits\n\n\n    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n    [Parallel(n_jobs=-1)]: Done   2 tasks      | elapsed:    0.0s\n    [Parallel(n_jobs=-1)]: Done 176 tasks      | elapsed:    0.4s\n    [Parallel(n_jobs=-1)]: Done 1552 tasks      | elapsed:    2.6s\n    [Parallel(n_jobs=-1)]: Done 3568 tasks      | elapsed:    5.7s\n    [Parallel(n_jobs=-1)]: Done 6160 tasks      | elapsed:   11.3s\n    [Parallel(n_jobs=-1)]: Done 6720 out of 6720 | elapsed:   12.4s finished\n\n\n\n\n\n    GridSearchCV(cv=5, estimator=KNeighborsClassifier(), n_jobs=-1,\n                 param_grid={'algorithm': ('auto', 'ball_tree', 'kd_tree', 'brute'),\n                             'leaf_size': [15, 20, 25, 30, 35, 40, 45],\n                             'n_neighbors': [2, 3, 4, 5, 6, 7, 8, 9],\n                             'p': [1, 2, 3], 'weights': ('uniform', 'distance')},\n                 verbose=5)\n\n\n\n\n```python\nclf.best_params_\n```\n\n\n\n\n    {'algorithm': 'auto',\n     'leaf_size': 15,\n     'n_neighbors': 9,\n     'p': 1,\n     'weights': 'uniform'}\n\n\n\n\n```python\nclf.best_score_\n```\n\n\n\n\n    0.6918439716312056\n\n\n\n\n```python\nclf.best_estimator_.score(X_test, y_test)\n```\n\n\n\n\n    0.65\n\n\n\n## Logistic Regression Classifier\n\n\n```python\nlr = LogisticRegression()\nestimators = {\n    'penalty':('l1', 'l2', 'elasticnet'),\n    'tol':[1e-6,1e-5,1e-4,1e-3,1e-2],\n    'C':[0.01,0.05,0.1,0.5,1.0,2.0],\n    'solver':('newton-cg','lbfgs','liblinear','sag','saga'),\n    'max_iter':[10000]\n}\n\nclf_lr = GridSearchCV(lr, estimators, cv=5, n_jobs=-1, verbose=5)\nclf_lr.fit(X_train, y_train)\n```\n\n    Fitting 5 folds for each of 450 candidates, totalling 2250 fits\n\n\n    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n    [Parallel(n_jobs=-1)]: Done   2 tasks      | elapsed:    0.0s\n    [Parallel(n_jobs=-1)]: Done 180 tasks      | elapsed:    0.7s\n    [Parallel(n_jobs=-1)]: Done 1120 tasks      | elapsed:    9.3s\n    [Parallel(n_jobs=-1)]: Done 1528 tasks      | elapsed:   19.1s\n    [Parallel(n_jobs=-1)]: Done 2081 tasks      | elapsed:   32.6s\n    [Parallel(n_jobs=-1)]: Done 2250 out of 2250 | elapsed:   38.7s finished\n\n\n\n\n\n    GridSearchCV(cv=5, estimator=LogisticRegression(), n_jobs=-1,\n                 param_grid={'C': [0.01, 0.05, 0.1, 0.5, 1.0, 2.0],\n                             'max_iter': [10000],\n                             'penalty': ('l1', 'l2', 'elasticnet'),\n                             'solver': ('newton-cg', 'lbfgs', 'liblinear', 'sag',\n                                        'saga'),\n                             'tol': [1e-06, 1e-05, 0.0001, 0.001, 0.01]},\n                 verbose=5)\n\n\n\n\n```python\nclf_lr.best_params_\n```\n\n\n\n\n    {'C': 1.0,\n     'max_iter': 10000,\n     'penalty': 'l1',\n     'solver': 'liblinear',\n     'tol': 0.01}\n\n\n\n\n```python\nclf_lr.best_score_\n```\n\n\n\n\n    0.8311170212765957\n\n\n\n\n```python\nclf_lr.best_estimator_.score(X_test, y_test)\n```\n\n\n\n\n    0.8833333333333333\n\n\n\n## SVC Classifier\n\n\n```python\nsvc = SVC()\nest_svc = {\n    'kernel':('linear', 'sigmoid'),\n    'degree':[3,4,5,6],\n    'C':[0.01,0.05,0.1,0.5,1.0,2.0],\n    'tol':[1e-4,1e-3,1e-2],\n    'gamma':('scale','auto')\n}\n\nclf_svc = GridSearchCV(svc, est_svc, cv=5, n_jobs=-1, verbose=5)\nclf_svc.fit(X_train, y_train)\n```\n\n    Fitting 5 folds for each of 288 candidates, totalling 1440 fits\n\n\n    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n    [Parallel(n_jobs=-1)]: Done   2 tasks      | elapsed:    1.2s\n    [Parallel(n_jobs=-1)]: Done  72 tasks      | elapsed:    1.4s\n    [Parallel(n_jobs=-1)]: Done 1094 tasks      | elapsed:   11.6s\n    [Parallel(n_jobs=-1)]: Done 1343 tasks      | elapsed:   22.3s\n    [Parallel(n_jobs=-1)]: Done 1425 out of 1440 | elapsed:   28.5s remaining:    0.3s\n    [Parallel(n_jobs=-1)]: Done 1440 out of 1440 | elapsed:   30.5s finished\n\n\n\n\n\n    GridSearchCV(cv=5, estimator=SVC(), n_jobs=-1,\n                 param_grid={'C': [0.01, 0.05, 0.1, 0.5, 1.0, 2.0],\n                             'degree': [3, 4, 5, 6], 'gamma': ('scale', 'auto'),\n                             'kernel': ('linear', 'sigmoid'),\n                             'tol': [0.0001, 0.001, 0.01]},\n                 verbose=5)\n\n\n\n\n```python\nclf_svc.best_params_\n```\n\n\n\n\n    {'C': 2.0, 'degree': 3, 'gamma': 'scale', 'kernel': 'linear', 'tol': 0.0001}\n\n\n\n\n```python\nclf_svc.best_score_\n```\n\n\n\n\n    0.8355496453900709\n\n\n\n\n```python\nclf_svc.best_estimator_.score(X_test,y_test)\n```\n\n\n\n\n    0.85\n\n\n\n## Random Forest Classifier\n\n\n```python\nrf = RandomForestClassifier()\nest_rf = {\n    'criterion':('gini', 'entropy'),\n    'n_estimators':[50,80,100,120,150,180,200],\n    'max_depth':[None,1,2,3,4,5],\n    'min_samples_split':[2,3,4],\n    'min_samples_leaf':[1,2]\n}\n\nclf_rf = GridSearchCV(rf, est_rf, cv=5, n_jobs=-1, verbose=5)\nclf_rf.fit(X_train, y_train)\n```\n\n    Fitting 5 folds for each of 504 candidates, totalling 2520 fits\n\n\n    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n    [Parallel(n_jobs=-1)]: Done   2 tasks      | elapsed:    1.5s\n    [Parallel(n_jobs=-1)]: Done  56 tasks      | elapsed:    3.8s\n    [Parallel(n_jobs=-1)]: Done 146 tasks      | elapsed:    7.6s\n    [Parallel(n_jobs=-1)]: Done 272 tasks      | elapsed:   13.1s\n    [Parallel(n_jobs=-1)]: Done 434 tasks      | elapsed:   20.4s\n    [Parallel(n_jobs=-1)]: Done 632 tasks      | elapsed:   30.0s\n    [Parallel(n_jobs=-1)]: Done 866 tasks      | elapsed:   42.1s\n    [Parallel(n_jobs=-1)]: Done 1136 tasks      | elapsed:   55.1s\n    [Parallel(n_jobs=-1)]: Done 1442 tasks      | elapsed:  1.2min\n    [Parallel(n_jobs=-1)]: Done 1784 tasks      | elapsed:  1.5min\n    [Parallel(n_jobs=-1)]: Done 2162 tasks      | elapsed:  1.8min\n    [Parallel(n_jobs=-1)]: Done 2520 out of 2520 | elapsed:  2.1min finished\n\n\n\n\n\n    GridSearchCV(cv=5, estimator=RandomForestClassifier(), n_jobs=-1,\n                 param_grid={'criterion': ('gini', 'entropy'),\n                             'max_depth': [None, 1, 2, 3, 4, 5],\n                             'min_samples_leaf': [1, 2],\n                             'min_samples_split': [2, 3, 4],\n                             'n_estimators': [50, 80, 100, 120, 150, 180, 200]},\n                 verbose=5)\n\n\n\n\n```python\nclf_rf.best_params_\n```\n\n\n\n\n    {'criterion': 'entropy',\n     'max_depth': 1,\n     'min_samples_leaf': 2,\n     'min_samples_split': 4,\n     'n_estimators': 100}\n\n\n\n\n```python\nclf_rf.best_score_\n```\n\n\n\n\n    0.835372340425532\n\n\n\n\n```python\nclf_rf.best_estimator_.score(X_test,y_test)\n```\n\n\n\n\n    0.8833333333333333\n\n\n\n# Model with Transformations\n\n\n```python\nX_tr, X_t, y_tr, y_t = train_test_split(data_T.drop('num',axis=1), data_T['num'], test_size=0.2, random_state=42)\n```\n\n## K Neighbors Classifier\n\n\n```python\nneigh_T = KNeighborsClassifier()\nparameters_T = {\n    'n_neighbors':[2,3,4,5,6,7,8,9],\n    'weights':('uniform','distance'),\n    'algorithm':('auto', 'ball_tree', 'kd_tree', 'brute'),\n    'leaf_size':[15,20,25,30,35,40,45],\n    'p':[1,2,3],\n}\nclf_T = GridSearchCV(neigh_T, parameters_T, cv=5, n_jobs=-1, verbose=5)\nclf_T.fit(X_tr, y_tr)\n```\n\n    Fitting 5 folds for each of 1344 candidates, totalling 6720 fits\n\n\n    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n    [Parallel(n_jobs=-1)]: Done   2 tasks      | elapsed:    1.2s\n    [Parallel(n_jobs=-1)]: Done  64 tasks      | elapsed:    1.4s\n    [Parallel(n_jobs=-1)]: Done 1072 tasks      | elapsed:    3.1s\n    [Parallel(n_jobs=-1)]: Done 3088 tasks      | elapsed:    6.3s\n    [Parallel(n_jobs=-1)]: Done 5680 tasks      | elapsed:   11.1s\n    [Parallel(n_jobs=-1)]: Done 6720 out of 6720 | elapsed:   13.2s finished\n\n\n\n\n\n    GridSearchCV(cv=5, estimator=KNeighborsClassifier(), n_jobs=-1,\n                 param_grid={'algorithm': ('auto', 'ball_tree', 'kd_tree', 'brute'),\n                             'leaf_size': [15, 20, 25, 30, 35, 40, 45],\n                             'n_neighbors': [2, 3, 4, 5, 6, 7, 8, 9],\n                             'p': [1, 2, 3], 'weights': ('uniform', 'distance')},\n                 verbose=5)\n\n\n\n\n```python\nclf_T.best_params_\n```\n\n\n\n\n    {'algorithm': 'auto',\n     'leaf_size': 15,\n     'n_neighbors': 7,\n     'p': 1,\n     'weights': 'distance'}\n\n\n\n\n```python\nclf_T.best_score_\n```\n\n\n\n\n    0.8312943262411349\n\n\n\n\n```python\nclf_T.best_estimator_.score(X_t, y_t)\n```\n\n\n\n\n    0.9\n\n\n\n## Logisitic Regression Classifier\n\n\n```python\nlr_T = LogisticRegression()\nestimators_T = {\n    'penalty':('l1', 'l2', 'elasticnet'),\n    'tol':[1e-6,1e-5,1e-4,1e-3,1e-2],\n    'C':[0.01,0.05,0.1,0.5,1.0,2.0],\n    'solver':('newton-cg','lbfgs','liblinear','sag','saga'),\n    'max_iter':[10000]\n}\n\nclf_lr_T = GridSearchCV(lr_T, estimators_T, cv=5, n_jobs=-1, verbose=5)\nclf_lr_T.fit(X_tr, y_tr)\n```\n\n    Fitting 5 folds for each of 450 candidates, totalling 2250 fits\n\n\n    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n    [Parallel(n_jobs=-1)]: Done   2 tasks      | elapsed:    0.0s\n    [Parallel(n_jobs=-1)]: Done 176 tasks      | elapsed:    0.3s\n    [Parallel(n_jobs=-1)]: Done 1914 tasks      | elapsed:    2.2s\n    [Parallel(n_jobs=-1)]: Done 2250 out of 2250 | elapsed:    2.6s finished\n\n\n\n\n\n    GridSearchCV(cv=5, estimator=LogisticRegression(), n_jobs=-1,\n                 param_grid={'C': [0.01, 0.05, 0.1, 0.5, 1.0, 2.0],\n                             'max_iter': [10000],\n                             'penalty': ('l1', 'l2', 'elasticnet'),\n                             'solver': ('newton-cg', 'lbfgs', 'liblinear', 'sag',\n                                        'saga'),\n                             'tol': [1e-06, 1e-05, 0.0001, 0.001, 0.01]},\n                 verbose=5)\n\n\n\n\n```python\nclf_lr_T.best_params_\n```\n\n\n\n\n    {'C': 0.01,\n     'max_iter': 10000,\n     'penalty': 'l2',\n     'solver': 'newton-cg',\n     'tol': 1e-06}\n\n\n\n\n```python\nclf_lr_T.best_score_\n```\n\n\n\n\n    0.8310283687943263\n\n\n\n\n```python\nclf_lr_T.best_estimator_.score(X_t, y_t)\n```\n\n\n\n\n    0.9\n\n\n\n## SVC Classifier\n\n\n```python\nsvc = SVC()\nest_svc = {\n    'kernel':('linear', 'sigmoid'),\n    'degree':[3,4,5,6],\n    'C':[0.01,0.05,0.1,0.5,1.0,2.0],\n    'tol':[1e-4,1e-3,1e-2],\n    'gamma':('scale','auto')\n}\n\nclf_svc = GridSearchCV(svc, est_svc, cv=5, n_jobs=-1, verbose=5)\nclf_svc.fit(X_train, y_train)\n```\n\n    Fitting 5 folds for each of 288 candidates, totalling 1440 fits\n\n\n    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n    [Parallel(n_jobs=-1)]: Done   2 tasks      | elapsed:    1.2s\n    [Parallel(n_jobs=-1)]: Done  72 tasks      | elapsed:    1.4s\n    [Parallel(n_jobs=-1)]: Done 1094 tasks      | elapsed:   11.6s\n    [Parallel(n_jobs=-1)]: Done 1343 tasks      | elapsed:   22.3s\n    [Parallel(n_jobs=-1)]: Done 1425 out of 1440 | elapsed:   28.5s remaining:    0.3s\n    [Parallel(n_jobs=-1)]: Done 1440 out of 1440 | elapsed:   30.5s finished\n\n\n\n\n\n    GridSearchCV(cv=5, estimator=SVC(), n_jobs=-1,\n                 param_grid={'C': [0.01, 0.05, 0.1, 0.5, 1.0, 2.0],\n                             'degree': [3, 4, 5, 6], 'gamma': ('scale', 'auto'),\n                             'kernel': ('linear', 'sigmoid'),\n                             'tol': [0.0001, 0.001, 0.01]},\n                 verbose=5)\n\n\n\n\n```python\nclf_svc.best_params_\n```\n\n\n\n\n    {'C': 2.0, 'degree': 3, 'gamma': 'scale', 'kernel': 'linear', 'tol': 0.0001}\n\n\n\n\n```python\nclf_svc.best_score_\n```\n\n\n\n\n    0.8355496453900709\n\n\n\n\n```python\nclf_svc.best_estimator_.score(X_test,y_test)\n```\n\n\n\n\n    0.85\n\n\n\n## Random Forest Classifier\n\n\n```python\nrf_T = RandomForestClassifier()\nest_rf_T = {\n    'criterion':('gini', 'entropy'),\n    'n_estimators':[50,80,100,120,150,180,200],\n    'max_depth':[None,1,2,3,4,5],\n    'min_samples_split':[2,3,4],\n    'min_samples_leaf':[1,2]\n}\n\nclf_rf_T = GridSearchCV(rf_T, est_rf_T, cv=5, n_jobs=-1, verbose=5)\nclf_rf_T.fit(X_tr, y_tr)\n```\n\n    Fitting 5 folds for each of 504 candidates, totalling 2520 fits\n\n\n    [Parallel(n_jobs=-1)]: Using backend LokyBackend with 8 concurrent workers.\n    [Parallel(n_jobs=-1)]: Done   2 tasks      | elapsed:    0.1s\n    [Parallel(n_jobs=-1)]: Done  96 tasks      | elapsed:    4.6s\n    [Parallel(n_jobs=-1)]: Done 276 tasks      | elapsed:   12.3s\n    [Parallel(n_jobs=-1)]: Done 528 tasks      | elapsed:   24.0s\n    [Parallel(n_jobs=-1)]: Done 852 tasks      | elapsed:   40.4s\n    [Parallel(n_jobs=-1)]: Done 1176 tasks      | elapsed:   57.6s\n    [Parallel(n_jobs=-1)]: Done 1410 tasks      | elapsed:  1.2min\n    [Parallel(n_jobs=-1)]: Done 1680 tasks      | elapsed:  1.4min\n    [Parallel(n_jobs=-1)]: Done 1986 tasks      | elapsed:  1.7min\n    [Parallel(n_jobs=-1)]: Done 2328 tasks      | elapsed:  2.0min\n    [Parallel(n_jobs=-1)]: Done 2520 out of 2520 | elapsed:  2.1min finished\n\n\n\n\n\n    GridSearchCV(cv=5, estimator=RandomForestClassifier(), n_jobs=-1,\n                 param_grid={'criterion': ('gini', 'entropy'),\n                             'max_depth': [None, 1, 2, 3, 4, 5],\n                             'min_samples_leaf': [1, 2],\n                             'min_samples_split': [2, 3, 4],\n                             'n_estimators': [50, 80, 100, 120, 150, 180, 200]},\n                 verbose=5)\n\n\n\n\n```python\nclf_rf_T.best_params_\n```\n\n\n\n\n    {'criterion': 'entropy',\n     'max_depth': 2,\n     'min_samples_leaf': 2,\n     'min_samples_split': 2,\n     'n_estimators': 50}\n\n\n\n\n```python\nclf_rf_T.best_score_\n```\n\n\n\n\n    0.8312943262411346\n\n\n\n\n```python\nclf_rf_T.best_estimator_.score(X_t,y_t)\n```\n\n\n\n\n    0.8833333333333333\n\n\n\n# Conclusion\n\nThe datasets on heart disease has been used by many research groups around the world, in particular the Cleveland dataset. This project has investigated this datasetand have found some very interesting trends.\nThe data suggests old age is a significant factor to heart disease. Heart disease typically affects men more than women and this trend was also observed in men at a younger age than women, on average. Individuals with heart disease have a lower maximum heart rate, whereas healthy patients are able to achieve a higher maximum heart rate. It is also important to note that high cholesterol is dangerous and one will have a higher likelihood of heart disease.\n\nSome limitations encountered in this dataset is how small it is. There are only 303 rows of data and we only used 297 after the cleaning stage of this analysis. In the future one could acquire more data since this is typically is a better approach than even the most sophisticated models in machine learning, as stated by Peter Norvig et al. (2009), “But invariably simple models and a lot data trump more elaborate models based on less data.”(2009)\u003csup\u003e6\u003c/sup\u003e.\n\n\n6. https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/35179.pdf\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidhintelmann%2Fheart-disease","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavidhintelmann%2Fheart-disease","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidhintelmann%2Fheart-disease/lists"}