{"id":28287787,"url":"https://github.com/nacezavrtanik/aml","last_synced_at":"2025-09-04T15:38:53.324Z","repository":{"id":155409280,"uuid":"605021929","full_name":"nacezavrtanik/aml","owner":"nacezavrtanik","description":"Advanced Machine Learning","archived":false,"fork":false,"pushed_at":"2023-06-15T13:26:05.000Z","size":2006,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-18T05:39:44.694Z","etag":null,"topics":["data-streams","deep-neural-networks","equation-discovery","meta-learning","structured-data"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nacezavrtanik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-02-22T09:24:07.000Z","updated_at":"2023-09-18T08:23:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"d0ca4b3d-4718-497e-9b65-399e1abf8796","html_url":"https://github.com/nacezavrtanik/aml","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/nacezavrtanik/aml","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nacezavrtanik%2Faml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nacezavrtanik%2Faml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nacezavrtanik%2Faml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nacezavrtanik%2Faml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nacezavrtanik","download_url":"https://codeload.github.com/nacezavrtanik/aml/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nacezavrtanik%2Faml/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273632398,"owners_count":25140766,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-04T02:00:08.968Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-streams","deep-neural-networks","equation-discovery","meta-learning","structured-data"],"created_at":"2025-05-21T22:14:26.202Z","updated_at":"2025-09-04T15:38:53.313Z","avatar_url":"https://github.com/nacezavrtanik.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003cbr\u003e\n  \u003ca href=\"https://www.fmf.uni-lj.si/en/\"\u003e\u003cimg src=\"http://phd.fmf.uni-lj.si/img/logo.gif\" width=\"300\"\u003e\u003c/a\u003e\n  \u003cbr\u003e\n  Advanced Machine Learning\n  \u003cbr\u003e\n\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://www.fmf.uni-lj.si/sl/imenik/238/todorovski-ljupco/\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Professor-Ljup%C4%8Do%20Todorovski-red\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.fmf.uni-lj.si/sl/imenik/1402/brence-jure/\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Assistant-Jure%20Brence-red\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.fmf.uni-lj.si/sl/imenik/273/osojnik-aljaz/\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Assistant-Alja%C5%BE%20Osojnik-red\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.fmf.uni-lj.si/sl/imenik/177/petkovic-matej/\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Assistant-Matej%20Petkovi%C4%87-red\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/nacezavrtanik\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/RepositoryOwner-Nace%20Zavrtanik-lightgrey\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\nThis is my personal repository for the course Advanced Machine Learning (Napredno strojno učenje) as taken at the\nFaculty of Mathematics and Physics, University of Ljubljana, in the 2\u003csup\u003end\u003c/sup\u003e semester of 2022/2023. This\nrepository contains, of course, the code I wrote during the course of this course, such as code from exercise classes\nand homework assignments.\n\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/python-3.10-9cf\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/venv-requirements.txt-9cf\"\u003e\n\u003c/p\u003e\n\n\n## Table of Contents\n\n- [Exercise Classes](#exercise-classes)\n  - [Class 1: Fundamentals of Machine Learning in Python](#class-1-fundamentals-of-machine-learning-in-python)\n  - [Class 2: Meta-learning](#class-2-meta-learning)\n  - [Class 3: Meta-learning, and Hyperparameter Optimisation](#class-3-meta-learning-and-hyperparameter-optimisation)\n  - [Class 4: Hyperparameter Optimisation with `hyperopt`](#class-4-hyperparameter-optimisation-with-hyperopt)\n  - [Class 7: Machine Learning on Complex Data Structures, Part 1](#class-7-machine-learning-on-complex-data-structures-part-1)\n  - [Class 10: Equation Discovery](#class-10-equation-discovery)\n  - [Class 11: Equation Discovery with `ProGED`](#class-10-equation-discovery-with-proged)\n\n- [Homework Assignments](#homework-assignments)\n  - [Assignment 1: Method Selection, Hyperparameter Optimisation, Meta-learning](#assignment-1-method-selection-hyperparameter-optimisation-meta-learning)\n\n\n## Exercise Classes\n\nBelow, a broad overview of exercise classes is given. The exact instructions are not part of this repository.\nCorresponding code can be found in appropriate subdirectories of `ex/`.\n\n### Class 1: Fundamentals of Machine Learning in Python\n\n- Exercise A: ***Data Processing***\n  1. Load data\n  2. Extract basic statistics\n  3. Handle `NaN` values\n  4. Visualise data\n  5. Encode categorical features\n     - `sklearn.preprocessing.OneHotEncoder`\n     - `sklearn.compose.make_column_transformer`\n- Exercise B: ***Binary Classification***\n  1. Train model on entire dataset\n     - `sklearn.neighbors.KNeighborsClassifier`\n  2. Evaluate accuracy of model\n  3. Split dataset into train data and test data\n     - `sklearn.model_selection.train_test_split`\n  4. Scale features, analyze hyperparameters\n     - `sklearn.preprocessing.StandardScaler`\n     - `sklearn.model_selection.validation_curve`\n  5. Calculate alternative metrics\n     - `sklearn.metrics.confusion_matrix`\n     - `sklearn.metrics.precision_recall_curve`\n     - `sklearn.metrics.roc_curve`\n     - `sklearn.metrics.roc_auc_score`\n- Exercise C: ***Linear Regression***\n  - `sklearn.linear_model.LinearRegression`\n  1. Calculate regression metrics\n     - `sklearn.metrics.mean_squared_error`\n     - `sklearn.metrics.r2_score`\n  2. Cross-validate and compare models\n     - `sklearn.model_selection.cross_validate`\n     - `sklearn.svm.SVR`\n     - `sklearn.ensemble.RandomForestRegressor`\n     - `sklearn.neighbors.KNeighborsRegressor`\n\n[(Back to top)](#table-of-contents)\n\n### Class 2: Meta-learning\n\n- Exercise A: ***Obtaining Data from [OpenML](https://www.openml.org/)***\n  1. List dataset info\n     - `openml.datasets.list_datasets`\n  2. Load datasets\n     - `openml.datasets.get_datasets`\n  3. Filter datasets\n     - `pandas.api.types.is_numeric`\n- Exercise B: ***Preparing Target Variables***\n  1. Compare model accuracies for datasets\n     - `sklearn.tree.DecisionTreeClassifier`\n     - `sklearn.naive_bayes.GaussianNB`\n- Exercise C: ***Preparing Meta-features***\n  1. Extract meta-features from dataset\n     - `pymfe.mfe.MFE.fit`\n     - `pymfe.mfe.MFE.extract`\n  2. Extract meta-features from fitted model\n     - `pymfe.mfe.MFE.extract_from_model`\n  3. Extract meta-features for all datasets\n\n[(Back to top)](#table-of-contents)\n\n### Class 3: Meta-learning, and Hyperparameter Optimisation\n\n- Exercise A: ***Meta-classification, Meta-regression***\n  1. Preprocess data\n  2. Cross-validate and compare meta-models\n     - `sklearn.ensemble.RandomForestClassifier`\n     - `sklearn.dummy.DummyClassifier`\n  3. Compare features by importance\n     - `sklearn.ensemble.RandomForestClassifier.feature_importances_`\n     - `numpy.argsort`\n  4. Use a regression meta-model to predict accuracy\n- Exercise B: ***Hyperparameter Optimisation***\n  - `sklearn.*.model.get_params`\n  1. Vary hyperparameters in a decision tree model\n  2. Perform a grid search\n     - `sklearn.model_selection.GridSearchCV`\n     - `sklearn.model_selection.GridSearchCV.best_params_`\n     - `sklearn.model_selection.GridSearchCV.best_estimator_`\n  3. Visualise grid search results\n     - `grid_search.param_grid`\n\n[(Back to top)](#table-of-contents)\n\n### Class 4: Hyperparameter Optimisation with [hyperopt](https://github.com/hyperopt)\n\n- Exercise A: ***Hyperparameter Optimisation with hyperopt***\n  1. Minimise function of one variable, uniform distribution\n     - `hyperopt.fmin`\n     - `hyperopt.Trials`\n     - `hyperopt.tpe.suggest`\n     - `hyperopt.hp.uniform`\n  2. Minimise function of two variables, normal distribution\n     - `hyperopt.hp.normal`\n  3. Find best algorithm\n     - `hyperopt.hp.choice`\n  4. Define hyperparameter space\n  5. Compare algorithm performance with default and optimised hyperparameters\n\n[(Back to top)](#table-of-contents)\n\n### Class 7: Machine Learning on Complex Data Structures, Part 1\n\n- Exercise A: ***Analyzing the Given Code Template***\n- Exercise B: ***Filling in Missing Code***\n  1. Complete `razsiri_z_hours`\n  2. Complete `razsiri_z_attributes`\n  3. Complete the final for-loop\n- Exercise C: ***Contemplating the Inclusion of Tables `REVIEWS` and `USERS`***\n\n[(Back to top)](#table-of-contents)\n\n### Class 10: Equation Discovery\n\n- Exercise A: ***Equation Discovery with Linear Regression***\n  1. Implement linear regression\n     - `sklearn.preprocessing.PolynomialFeatures`\n  2. Test linear regression on given data\n  3. Handle noise with ridge regression\n  4. Handle noise with lasso regression\n- Exercise B: ***Equation Discovery with the BACON algorithm***\n  1. Implement the BACON algorithm\n  2. Test the BACON algorithm on given data\n\n[(Back to top)](#table-of-contents)\n\n### Class 10: Equation Discovery with [ProGED](https://github.com/brencej/ProGED)\n\n- Exercise A: ***Probabilistic Grammatics and `ProGED`***\n  1. Discover Newton's Second Law\n     - `ProGED.EqDisco.generate_models`\n     - `ProGED.EqDisco.fit_models`\n     - `ProGED.EqDisco.get_results`\n  2. Discover a linear function\n     - `ProGED.generators.GeneratorGrammar`\n  3. Discover the energy conservation law\n\n[(Back to top)](#table-of-contents)\n\n\n## Homework Assignments\n\n  Below, a broad overview of homework assignments is given. The exact instructions are not part of this repository.\n  Corresponding code can be found in appropriate subdirectories of `hw/`.\n\n### Assignment 1: Method Selection, Hyperparameter Optimisation, Meta-learning\n\n- Problem 1: ***Method Selection and Hyperparameter Optimisation***\n    1. Manual approach\n    2. Automated approach\n- Problem 2: ***Meta-learning***\n    1. Method selection with meta-learning\n\n[(Back to top)](#table-of-contents)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnacezavrtanik%2Faml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnacezavrtanik%2Faml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnacezavrtanik%2Faml/lists"}