{"id":15384688,"url":"https://github.com/bblodfon/ml-notes","last_synced_at":"2026-01-05T23:02:50.671Z","repository":{"id":90675270,"uuid":"292038515","full_name":"bblodfon/ml-notes","owner":"bblodfon","description":"My personal notes on ML stuff","archived":false,"fork":false,"pushed_at":"2024-10-19T12:54:29.000Z","size":734,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-30T13:22:54.894Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://bblodfon.github.io/ml-notes/","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bblodfon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-01T15:39:04.000Z","updated_at":"2024-10-19T12:54:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"8ac01aee-03d6-4b0d-bbc6-80577cc89e70","html_url":"https://github.com/bblodfon/ml-notes","commit_stats":{"total_commits":25,"total_committers":2,"mean_commits":12.5,"dds":"0.040000000000000036","last_synced_commit":"d2fdf7115585423a75e697a585fbcd7d1141de9e"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bblodfon%2Fml-notes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bblodfon%2Fml-notes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bblodfon%2Fml-notes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bblodfon%2Fml-notes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bblodfon","download_url":"https://codeload.github.com/bblodfon/ml-notes/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245483353,"owners_count":20622849,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T14:43:05.098Z","updated_at":"2026-01-05T23:02:50.614Z","avatar_url":"https://github.com/bblodfon.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# ml-notes\n\n## General\n\n- [CRAN Task View: ML](https://cran.r-project.org/web/views/MachineLearning.html)\n- [`mlr3` framework](https://github.com/mlr-org/mlr3/) and [mlr3book](https://mlr3book.mlr-org.com/)\n- [`caret` package](https://cran.r-project.org/web/packages/caret/index.html)\n- [mlim](https://github.com/haghish/mlim) R package for **ML-driven data imputation**\n\n## Hyperparameter Optimization\n\n- [Nice article](https://www.vantage-ai.com/en/blog/bayesian-optimization-for-quicker-hyperparameter-tuning) on Bayesian Optimization\n\n## Boosting\n\n- `xgboost` R package ([docs](https://xgboost.readthedocs.io/en/latest/), [GitHub](https://github.com/dmlc/xgboost/)) and nice intro [slides](http://www.chengli.io/tutorials/gradient_boosting.pdf)\n\n## Feature selection/importance\n\n- For **linear regression** (i.e. after using the `lm` function in R), a good package that provides *feature selection* based on diverse criteria (e.g. p-value) is [`SignifReg`](https://cran.r-project.org/web/packages/SignifReg/index.html).\n- Correlation can be used in 2 ways:\n  - Remove features with an absolute correlation of **0.75 or higher** when finding the correlation between variables in a training dataset (NO RESPONSE is used here - unsupervised so-to-speak)\n  - Variables that correlate highly with the response output, have **higher importance or significance**\n  - Visualize correlations with: `corrplot` library ([examples](https://cran.r-project.org/web/packages/corrplot/vignettes/corrplot-intro.html))\n- Use **Lasso (L1)** regularization or **Elastic Net (L1 and L2)** to remove unimportant features - **larger coefficient =\u003e more importance** (zeroed coefficients =\u003e good riddance!)\n  - Use [`gmlnet` package](https://cran.r-project.org/web/packages/glmnet/index.html)\n  - [Nice analysis](https://www4.stat.ncsu.edu/~post/josh/LASSO_Ridge_Elastic_Net_-_Examples.html)\n  for cross-validating the `alpha` parameter\n  - [Slides for Elastic Nets](https://web.stanford.edu/~hastie/TALKS/enet_talk.pdf) \n- Use [`randomForest` package](https://cran.r-project.org/web/packages/randomForest/index.html) \n  - Visualize importance: `varImpPlot()`\n  - [Article: Tune number of Trees?](https://stats.stackexchange.com/questions/348245/do-we-have-to-tune-the-number-of-trees-in-a-random-forest) - `500` is alright in general, **tune the `mtry` parameter** using the function `RFtune()`!!!\n  - If `random.Forest()` is run with `proximity=TRUE` (keep *N* less than 10000, depending on your RAM as well) it generates a *N* x *N* matrix of **proximity (similarity)** (*N* = number of rows/data points).\n  This can be scaled to 2D using `MDSplot()` (same data, same response vector, same random forest used to train the data), which internally uses the `stat::cmdscale()` to **see the dataset (every point) in 2D** (very slow).\n- For faster and multi-threaded Random Forests, use the [`ranger`](https://github.com/imbs-hl/ranger) R package\n  - No tuning offered for `mtry` (so do that with `randomForest` on random samples of your dataset), but everything else is better and faster!\n- Use the [Boruta R package](https://cran.r-project.org/web/packages/Boruta/index.html) and plot the importance boxplot result!\n  - Uses `ranger` under the hood currently, so multi-thread support for free\n\n## Correlation\n\n- [Article: Correlation Test Between Two Variables in R](http://www.sthda.com/english/wiki/correlation-test-between-two-variables-in-r)\n- Use `ggscatter`: prints `R^2` and p-value.\n- Use `ComplexHeatmap` to **visualize a correlation matrix between two variables of interest**, \ne.g. mRNA and protein expressions (if the data dimensions are large)\n- [On the correlation measure to use between two variables **(continuous vs discrete)**](https://medium.com/@outside2SDs/an-overview-of-correlation-measures-between-categorical-and-continuous-variables-4c7f85610365). From this article I quote: \"The idea behind using logistic regression to understand correlation between cont and categorical variables is actually quite straightforward and follows as such: **If there is a relationship between the categorical and continuous variable**, we should be able to construct an accurate predictor of the categorical variable from the continuous variable. If the **resulting classifier has a high degree of fit**, is accurate, sensitive, and specific we can conclude the two variables share a relationship and are indeed correlated.\"\n\n## Logistic Regression in R\n\n- [Application and Interpretation article](https://rpubs.com/rslbliss/r_logistic_ws)\n- If the **response categorical variable is binary**, use `stats::glm(family = binomial)`\n- For **multiple response classes**, use **Ordinal Logistic Regression** only when the *proportional odds assumption* is true (the relationship between each pair of response groups is the same - e.g. in a survey where you answer with 3 choices, the distance between *unlikely* and *somewhat likely* may be shorter than the distance between *somewhat likely* and *very likely*). In R, use `MASS::polr` or `rms::lrm`. Otherwise, go with **Multinomial Logistic Regression**: `nnet:multinom`.\n- Measures of goodness of fit (fit statistics - related to model validation) for **logistic binary regression**: [article IBM](https://www.ibm.com/support/knowledgecenter/SSLVMB_24.0.0/spss/tutorials/plum_germcr_rsquare.html)\n  - Mcfaddens `R^2`: `pscl:pR2`, `rms` =\u003e returns `res$stats` (has Nagelkerke's `R^2`)\n  - [Mcfaddens `R^2` interpretation](https://stats.stackexchange.com/questions/82105/mcfaddens-pseudo-r2-interpretation)\n  - [Nice reference](https://hbiostat.org/bib/r2.html) about `R^2` measures\n\n## Dimensionality Reduction\n\n- **PCA** (Prinicipal Componenent Analysis): [article](http://www.sthda.com/english/articles/31-principal-component-methods-in-r-practical-guide/112-pca-principal-component-analysis-essentials/)\n  - Use R packages `FactoMineR` (run PCA) and `factoexta` (for visualization)\n- **MCA** (Multiple Correspondence Analysis - PCA for categorical variables): [article](http://www.sthda.com/english/articles/31-principal-component-methods-in-r-practical-guide/114-mca-multiple-correspondence-analysis-in-r-essentials/)\n- **UMAP**: a non-linear dimensionality reduction method, check R package [uwot](https://github.com/jlmelville/uwot)\n\n## PR (Precision-Recall curves)\n\n- [Intro Article](https://classeval.wordpress.com/introduction/introduction-to-the-precision-recall-plot/)\n\n## Hierarchical Clustering Analysis\n\n- [Nice explanatory article](https://uc-r.github.io/hc_clustering)\n- Comparing Dendrograms in R\n  - [Datanovia article](https://www.datanovia.com/en/lessons/comparing-cluster-dendrograms-in-r/)\n  - R package [dendextend](https://talgalili.github.io/dendextend/articles/dendextend.html)\n\n## Survival Analysis\n\n- [CRAN Task View: Survival Analysis](https://cran.r-project.org/web/views/Survival.html)\n- [KM-plots and log-rank tests in R](http://www.sthda.com/english/wiki/survival-analysis-basics)\n- [Cox Proportional Hazards Modeling in R](http://www.sthda.com/english/wiki/cox-proportional-hazards-model)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbblodfon%2Fml-notes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbblodfon%2Fml-notes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbblodfon%2Fml-notes/lists"}