{"id":20285442,"url":"https://github.com/mramshaw/ml_with_scipy","last_synced_at":"2026-05-08T02:18:37.882Z","repository":{"id":44056296,"uuid":"162912824","full_name":"mramshaw/ML_with_SciPy","owner":"mramshaw","description":"Machine Learning with SciPy","archived":false,"fork":false,"pushed_at":"2024-06-08T09:06:12.000Z","size":789,"stargazers_count":0,"open_issues_count":22,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-14T08:12:09.113Z","etag":null,"topics":["machine-learning","ml","pip","pip3","python","python3","scipy","seaborn","sklearn"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mramshaw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-12-23T17:57:33.000Z","updated_at":"2018-12-31T16:04:18.000Z","dependencies_parsed_at":"2023-02-09T01:03:32.668Z","dependency_job_id":"17d5b415-1a20-4103-bbe8-0db082115b5d","html_url":"https://github.com/mramshaw/ML_with_SciPy","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mramshaw%2FML_with_SciPy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mramshaw%2FML_with_SciPy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mramshaw%2FML_with_SciPy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mramshaw%2FML_with_SciPy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mramshaw","download_url":"https://codeload.github.com/mramshaw/ML_with_SciPy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241780465,"owners_count":20019058,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","ml","pip","pip3","python","python3","scipy","seaborn","sklearn"],"created_at":"2024-11-14T14:26:43.463Z","updated_at":"2026-05-08T02:18:32.856Z","avatar_url":"https://github.com/mramshaw.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Machine Learning with SciPy\n\n[![Known Vulnerabilities](http://snyk.io/test/github/mramshaw/ML_with_SciPy/badge.svg?style=plastic\u0026targetFile=requirements.txt)](http://snyk.io/test/github/mramshaw/ML_with_SciPy?style=plastic\u0026targetFile=requirements.txt)\n\nA quick end-to-end exploration of a simple Machine Learning project.\n\n## Motivation\n\nWhile I had previously investigated\n[Machine Learning](http://github.com/mramshaw/Intro-to-ML)\nand [Data Cleaning](http://github.com/mramshaw/Data-Cleaning),\nthe opportunity to follow along and experience an ML project\nfrom end to end seemed like a great way to gain perspective\non what actually happens in a simple project.\n\nThe sequence of events is as follows:\n\n1. Install Python and the SciPy platform\n2. Load the Iris dataset\n3. Summarize the dataset\n4. Visualize the dataset\n5. Evaluate some algorithms\n6. Make some predictions\n\nI had previously done most of these things - with the exception of\n__5__ (evaluating different algorithms), which was thus of particular interest.\n\n## Table of Contents\n\nThe contents are as follows:\n\n* [Prerequisites](#prerequisites)\n* [Data](#data)\n* [Data Summarized](#data-summarized)\n* [Data Visualized](#data-visualized)\n    * [Univariate plots](#univariate-plots)\n    * [Multivariate plots](#multivariate-plots)\n* [Evaluate Algorithms](#evaluate-algorithms)\n* [Make Predictions](#make-predictions)\n* [Versions](#versions)\n* [Concepts](#concepts)\n    * [Box-and-whisker plots](#box-and-whisker-plots)\n* [Seaborn](#seaborn)\n* [Reference](#reference)\n* [To Do](#to-do)\n* [Credits](#credits)\n\n## Prerequisites\n\nEither __Python 2__ or __Python 3__ is required, as well as\na copy of `pip` (either `pip` for Python 2 or `pip3` for\nPython 3).\n\nInstall the required libraries as follows:\n\n    $ pip install --user -r requirements.txt\n\n[I never recommend Global installation. Replace with `pip3` for Python 3.]\n\nFind the installed versions by running \u003ckbd\u003epython versions.py\u003c/kbd\u003e as shown:\n\n```bash\n$ python versions.py\nPython: 2.7.12 (default, Nov 12 2018, 14:36:49) \n[GCC 5.4.0 20160609]\nscipy: 0.17.0\nnumpy: 1.14.0\nmatplotlib: 2.0.2\npandas: 0.20.3\nsklearn: 0.20.0\nseaborn: 0.9.0\n$\n```\n\n## Data\n\nWe will use the well-known\n[Iris data set](http://en.wikipedia.org/wiki/Iris_flower_data_set),\nwhich I previously used in my\n[Iris](http://github.com/mramshaw/Intro-to-ML/tree/master/Iris) exercise.\n\n[There we accessed the data set via sklearn's `load_iris` convenience method.]\n\nThis data set should be available at:\n\n    http://archive.ics.uci.edu/ml/datasets/Iris\n\nMy experience has been that data sets, as well as software libraries, tend\nto experience ___drift___ over time. Accordingly, in order to try to replicate\nthe published results as closely as possible, I downloaded the data set from\nthe [author's published version](http://raw.githubusercontent.com/jbrownlee/Datasets/master/iris.csv).\n\nAs this is a well-known data set, we will not need to do any data cleaning\n(which would generally be a considerable time-sink in any ML exercise).\n\nLikewise, we will check for any missing data points (there aren't any, but\nit is probably always a good practice to check for data completeness).\n\n## Data Summarized\n\nThe __summarization__ part of the output should look as follows:\n\n```bash\n$ python iris.py\nRows, columns = (150, 5)\n\nThe first 20 observations\n-------------------------\n    sepal-length  sepal-width  petal-length  petal-width        class\n0            5.1          3.5           1.4          0.2  Iris-setosa\n1            4.9          3.0           1.4          0.2  Iris-setosa\n2            4.7          3.2           1.3          0.2  Iris-setosa\n3            4.6          3.1           1.5          0.2  Iris-setosa\n4            5.0          3.6           1.4          0.2  Iris-setosa\n5            5.4          3.9           1.7          0.4  Iris-setosa\n6            4.6          3.4           1.4          0.3  Iris-setosa\n7            5.0          3.4           1.5          0.2  Iris-setosa\n8            4.4          2.9           1.4          0.2  Iris-setosa\n9            4.9          3.1           1.5          0.1  Iris-setosa\n10           5.4          3.7           1.5          0.2  Iris-setosa\n11           4.8          3.4           1.6          0.2  Iris-setosa\n12           4.8          3.0           1.4          0.1  Iris-setosa\n13           4.3          3.0           1.1          0.1  Iris-setosa\n14           5.8          4.0           1.2          0.2  Iris-setosa\n15           5.7          4.4           1.5          0.4  Iris-setosa\n16           5.4          3.9           1.3          0.4  Iris-setosa\n17           5.1          3.5           1.4          0.3  Iris-setosa\n18           5.7          3.8           1.7          0.3  Iris-setosa\n19           5.1          3.8           1.5          0.3  Iris-setosa\n\nMissing Data\n------------\nsepal-length    0\nsepal-width     0\npetal-length    0\npetal-width     0\nclass           0\ndtype: int64\n\nStatistics\n----------\n       sepal-length  sepal-width  petal-length  petal-width\ncount    150.000000   150.000000    150.000000   150.000000\nmean       5.843333     3.054000      3.758667     1.198667\nstd        0.828066     0.433594      1.764420     0.763161\nmin        4.300000     2.000000      1.000000     0.100000\n25%        5.100000     2.800000      1.600000     0.300000\n50%        5.800000     3.000000      4.350000     1.300000\n75%        6.400000     3.300000      5.100000     1.800000\nmax        7.900000     4.400000      6.900000     2.500000\n\nClass Observations\n------------------\nclass\nIris-setosa        50\nIris-versicolor    50\nIris-virginica     50\ndtype: int64\n\n...\n```\n\n## Data Visualized\n\nWe will start with __univariate__ plots and then proceed to __multivariate__ plots.\n\n#### Univariate plots\n\nThe [box-and-whisker plots](#box-and-whisker-plots) should look as follows:\n\n![Box-and-whisker Plot](images/Figure_1.png)\n\nThe histograms should look as follows:\n\n![Histograms](images/Figure_2.png)\n\n[Possibly the sepal-length and sepal-width follow a __normal__ (or __gaussian__)\n distribution, although the small sample size makes it hard to be too conclusive.]\n\n#### Multivariate plots\n\nThe scatter-plot matrix should look as follows:\n\n![Scatter-plot Matrix](images/Figure_3.png)\n\n[Note that the diagonal from top-left to bottom-right shows our histograms,\n although in a different order than we saw them before.]\n\nWe can obtain even more useful results using [Seaborn](#seaborn):\n\n![Seaborn Scatter-plot](images/Seaborn_Scatterplot.png)\n\nHere we can see that `petal-length` and `petal-width` are distributed\ndifferently for our various Iris categories.\n\nThis corresponds with the results obtained in my\n[Iris](http://github.com/mramshaw/Intro-to-ML/tree/master/Iris) exercise:\n\n![Iris decision tree](images/IrisTree.dot.svg)\n\n## Evaluate Algorithms\n\nAlmost every ML paper these days has a table contrasting the accuracy\nof different algorithms, so let's follow that pattern.\n\n[I listened to an interview with\n[Geoffrey Hinton](https://en.wikipedia.org/wiki/Geoffrey_Hinton)\n where he bemoaned this fact. In his opinion, what is really needed\n is new and original thinking, rather than small tweaks to well-established\n methods. However, it is still necessary to evaluate methods based upon\n some criteria, so why not compare against competing methods?]\n\nAccording to the tutorial, the results should have been as follows:\n\n```\nLR: 0.966667 (0.040825)\nLDA: 0.975000 (0.038188)\nKNN: 0.983333 (0.033333)\nCART: 0.975000 (0.038188)\nNB: 0.975000 (0.053359)\nSVM: 0.981667 (0.025000)\n```\n\nAlong with some deprecation warnings, only CART and SVM differed.\n\nI got:\n\n```\nCART: 0.966667 (0.040825)\nSVM: 0.991667 (0.025000)\n```\n\nThe final comparison looked as follows:\n\n![Algorithm comparison](images/Algorithm_comparison.png)\n\n[This diagram looks somewhat different than in the tutorial, but the\n overall results seem to be about the same. Note that the KNN, NB and\n SVM boxes are squashed up at 100 percent accuracy.]\n\n## Make Predictions\n\nExtrapolate a trend, make a recommendation, the goal is to make some\nsort of prediction.\n\nAccording to the tutorial, results should have been as follows:\n\n```\n0.9\n\n[[ 7  0  0]\n [ 0 11  1]\n [ 0  2  9]]\n\n             precision    recall  f1-score   support\n\nIris-setosa       1.00      1.00      1.00         7\nIris-versicolor   0.85      0.92      0.88        12\nIris-virginica    0.90      0.82      0.86        11\n\navg / total       0.90      0.90      0.90        30\n```\n\nThe first __0.9__ indicates 90 percent accuracy.\n\nInstead of the final \"avg / total\" line I got the following:\n\n```\n      micro avg       0.90      0.90      0.90        30\n      macro avg       0.92      0.91      0.91        30\n   weighted avg       0.90      0.90      0.90        30\n```\n\n[Seems to be the same apart from the \"macro avg\".]\n\n## Versions\n\n* Python __2.7.12__\n\n[The tutorial also covered Python 3, but I used Python 2]\n\n* matplotlib __2.0.2__\n* numpy __1.14.0__\n* pandas __0.20.3__\n* scipy __0.17.0__\n* seaborn __0.9.0__\n* sklearn __0.20.0__\n\n## Concepts\n\nThis tutorial reminded me of some concepts, which are listed below.\n\n#### Box-and-whisker plots\n\nThese are a type of __univariate__ plot, much like a __Probability Density Function__.\n\nIf I have understood things correctly, the green line should indicate the second quartile (the __median__).\nOf course, in the tutorial, the corresponding line was red (rather than green as mine shows).\n\nOutliers (see __sepal-width__) show as circles (although in the tutorial, they show as __+__ symbols).\n\nWikipedia has a great diagram showing Box-and-whisker plots contrasted with Probability Density Functions:\n\n![Box-and-whisker Plots contrasted with Probability Density Functions](images/Boxplot_vs_PDF.svg)\n\n## Seaborn\n\nWhen creating multivariate visualizations, colour becomes important.\n\n[Seaborn](http://seaborn.pydata.org/) offers a number of great palettes and is worth installing for this fact alone.\n\nOf course, it is also possible to do something much like this with `matplotlib` but `seaborn` makes it simpler.\n\nFor help in visualizing any installed color schemes, check out my [No More Blue](http://github.com/mramshaw/No_More_Blue) repo.\n\n## Reference\n\nread_csv\n\n    http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html\n\nbox-and-whisker plots\n\n    http://en.wikipedia.org/wiki/Box_plot\n\nSeaborn scatter plot\n\n    http://seaborn.pydata.org/examples/scatterplot_matrix.html\n\n## To Do\n\n- [x] Add check for missing data points\n- [x] Add __Snyk.io__ vulnerability scanning\n- [x] Fix annoying `sklearn` deprecation warnings\n- [ ] Investigate naming figures with `matplotlib.pyplot`\n- [x] Add notes on using `seaborn` for multivariate visualizations\n- [ ] Verify code conforms to `pylint`, `pycodestyle` and `pydocstyle`\n- [ ] 12-Factor everything (validation segment size, seeds, test splits)\n- [ ] Investigate [populating missing data](http://machinelearningmastery.com/handle-missing-data-python/) and/or [Dora](http://github.com/NathanEpstein/Dora)\n\n## Credits\n\nI followed this excellent tutorial:\n\n    http://machinelearningmastery.com/machine-learning-in-python-step-by-step/\n\nWhile I already had experience with all of the individual steps, it was nice to see them in an end-to-end format.\n\nBox-and-whisker plots contrasted with Probability Density Function:\n\n    http://upload.wikimedia.org/wikipedia/commons/1/1a/Boxplot_vs_PDF.svg\n\nHow to produce a better scatter plot from this StackOverflow question:\n\n    http://stackoverflow.com/questions/22943894/class-labels-in-pandas-scattermatrix\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmramshaw%2Fml_with_scipy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmramshaw%2Fml_with_scipy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmramshaw%2Fml_with_scipy/lists"}