{"id":20735163,"url":"https://github.com/djsutherland/ds3-kernels-21","last_synced_at":"2026-04-16T15:03:30.284Z","repository":{"id":138669389,"uuid":"326962598","full_name":"djsutherland/ds3-kernels-21","owner":"djsutherland","description":null,"archived":false,"fork":false,"pushed_at":"2021-01-09T18:42:20.000Z","size":46516,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"built","last_synced_at":"2025-01-18T00:52:05.535Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/djsutherland.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-05T10:23:10.000Z","updated_at":"2021-01-09T18:42:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"4e20fd01-1bd2-4b05-92f9-2c29e1ebee2e","html_url":"https://github.com/djsutherland/ds3-kernels-21","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/djsutherland%2Fds3-kernels-21","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/djsutherland%2Fds3-kernels-21/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/djsutherland%2Fds3-kernels-21/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/djsutherland%2Fds3-kernels-21/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/djsutherland","download_url":"https://codeload.github.com/djsutherland/ds3-kernels-21/tar.gz/refs/heads/built","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243020930,"owners_count":20223061,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T05:34:39.383Z","updated_at":"2026-04-16T15:03:30.243Z","avatar_url":"https://github.com/djsutherland.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"This is the practical component of the [Data Science Summer School](https://www.ds3-datascience-polytechnique.fr) 2021 session on \"Kernel Methods: From Basics to Modern Applications\" by [Danica Sutherland](https://djsutherland.ml).\nSlides are available [here](https://djsutherland.ml/slides/ds3-21/) (or in [pdf](https://djsutherland.ml/slides/ds3-21.pdf)).\n\nThese materials are (slightly) updated from [the 2019 version](https://github.com/djsutherland/ds3-kernels/) in discussion with [Bharath Sriperumbudur](http://personal.psu.edu/bks18/), which were in turn partially based on [a 2018 version](https://github.com/karlnapf/ds3_kernel_testing) by [Heiko Strathmann](http://herrstrathmann.de/) in discussion with [Arthur Gretton](http://www.gatsby.ucl.ac.uk/~gretton/).\n\nWe'll cover, in varying levels of detail, the following topics:\n\n- Solving regression problems with kernel ridge regression ([`ridge.ipynb`](ridge.ipynb)):\n  - The \"standard\" approach.\n  - Kernel choice, and how it affects the resulting fit.\n  - Optionally: learning an appropriate kernel function in a meta-learning setting.\n- Two-sample testing with the kernel Maximum Mean Discrepancy (MMD) ([`testing.ipynb`](testing.ipynb)):\n  - Estimators for the MMD.\n  - Learning an appropriate kernel function.\n\n## Dependencies\n\n### Colab\n\nThese notebooks are available on Google Colab: [ridge](https://colab.research.google.com/github/djsutherland/ds3-kernels-21/blob/built/ridge.ipynb) or [testing](https://colab.research.google.com/github/djsutherland/ds3-kernels-21/blob/built/testing.ipynb). You don't have to set anything up yourself and it runs on cloud resources, so this is probably the easiest option. If you want to use the GPU, click Runtime -\u003e Change runtime type -\u003e Hardware accelerator -\u003e GPU.\n\n### Local setup\n\nRun `check_imports_and_download.py` to see if everything you need is installed (and download some more small datasets if necessary). If that works, you're set; otherwise, read on.\n\n\n### Files\nThere are a few Python files and some data files in the repository. By far the easiest thing to do is just put them all in the same directory:\n\n```\ngit clone --single-branch https://github.com/djsutherland/ds3-kernels-21\n```\n\n#### Python version\nThis notebook requires Python 3.6+.\n\nIf you've somehow still only used Python 2, it's time to [stop living in the past](https://python3statement.org/), but don't worry! It's almost the same; for the purposes of this notebook, you probably only need to know that you should write `print(\"hi\")` since it's a function call now, and you can write `A @ B` instead of `np.dot(A, B)`.\n\n#### Python packages\n\nThe main thing we use is PyTorch and Jupyter. If you already have those set up, you should be fine; just additionally make sure you also have (with `conda install` or `pip install`) `seaborn`, `tqdm`, and `sckit-learn`. We import everything right at the start, so if that runs you shouldn't hit any surprises later on.\n\nIf you don't already have a setup you're happy with, we recommend the `conda` package manager - start by installing [miniconda](https://docs.conda.io/en/latest/miniconda.html). Then you can create an environment with everything you need as:\n\n```bash\n# replace cpuonly with an appropriate cudatoolkit version if you want GPU support\nconda create --name ds3-kernels --override-channels -c pytorch -c defaults --strict-channel-priority python=3 notebook ipywidgets numpy scipy scikit-learn matplotlib seaborn tqdm pytorch torchvision cpuonly\n\nconda activate ds3-kernels\n\ngit clone https://github.com/djsutherland/ds3-kernels-21\ncd ds3-kernels-21\npython check_imports_and_download.py\njupyter notebook\n```\n\n(If you have an old conda setup, you can use `source activate` instead of `conda activate`, but it's better to [switch to the new style of activation](https://conda.io/projects/conda/en/latest/release-notes.html#recommended-change-to-enable-conda-in-your-shell). This won't matter for this tutorial, but it's general good practice.)\n\n(You can make your life easier when using jupyter notebooks with multiple kernels by installing `nb_conda_kernels`, but as long as you install and run `jupyter` from inside the env it will also be fine.)\n\n\n## PyTorch\n\nWe're going to use PyTorch in this tutorial, even though we're not doing a ton of \"deep learning.\" (The CPU version will be fine, though a GPU might let you get slightly better performance in some of the \"advanced\" sections.)\n\nIf you haven't used PyTorch before, don't worry! The API is unfortunately a little different from NumPy (and TensorFlow), but it's pretty easy to get used to; you can refer to [a cheat sheet vs NumPy](https://github.com/wkentaro/pytorch-for-numpy-users/blob/master/README.md) as well as the docs: [tensor methods](https://pytorch.org/docs/stable/tensors.html) and [the `torch` namespace](https://pytorch.org/docs/stable/torch.html#torch.eq). Feel free to ask if you have trouble figuring something out.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdjsutherland%2Fds3-kernels-21","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdjsutherland%2Fds3-kernels-21","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdjsutherland%2Fds3-kernels-21/lists"}