{"id":15359138,"url":"https://github.com/santisoler/scipy2024-pooch","last_synced_at":"2026-01-03T06:07:09.955Z","repository":{"id":248068745,"uuid":"761461206","full_name":"santisoler/scipy2024-pooch","owner":"santisoler","description":"Abstract for SciPy 2024","archived":false,"fork":false,"pushed_at":"2024-07-16T15:40:35.000Z","size":16301,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-27T04:46:40.299Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc-by-4.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/santisoler.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-21T21:53:32.000Z","updated_at":"2024-07-16T15:40:39.000Z","dependencies_parsed_at":"2024-07-16T02:03:56.961Z","dependency_job_id":"1c9ab30e-cfb7-4a32-91ed-d34674de0b87","html_url":"https://github.com/santisoler/scipy2024-pooch","commit_stats":null,"previous_names":["santisoler/scipy2024-pooch"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/santisoler%2Fscipy2024-pooch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/santisoler%2Fscipy2024-pooch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/santisoler%2Fscipy2024-pooch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/santisoler%2Fscipy2024-pooch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/santisoler","download_url":"https://codeload.github.com/santisoler/scipy2024-pooch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243829150,"owners_count":20354609,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T12:43:54.454Z","updated_at":"2026-01-03T06:07:09.949Z","avatar_url":"https://github.com/santisoler.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pooch: a friend to fetch your data files\n\n[Santiago Soler][santisoler]\n\n| Information |                                                                                    |\n| ----------- | ---------------------------------------------------------------------------------- |\n| **Track**   | General                                                                            |\n| **When**    | July 12, 14:35 - 15:05                                                             |\n| **Where**   | Room 316, Tacoma Convention Center, Tacoma, WA, USA                                |\n| **Slides**  | [Download PDF][slides-pdf], [Download ODP][slides-odp]                             |\n| **DOI**     | [10.6084/m9.figshare.26312236.v1](https://doi.org/10.6084/m9.figshare.26312236.v1) |\n\n\nRead the [abstract][abstract] in [SciPy2024 schedule][schedule].\n\n## Abstract\n\nPooch is a Python library that can download and locally cache files from\nthe web without hassle. Novices can use it to simply download files\nin one line of code and focus on the data.\nPackage maintainers can use it to provide sample datasets\nto their users, in examples and tutorials, as libraries like [SciPy][scipy],\n[scikit-image][scikit-image], [napari][napari] and [MetPy][metpy] do.\nDuring this talk, we'll show you how you can use the different features that\nPooch offers and also how you can extend its capabilities by writing your own\ndownloaders or post-processors.\n\n![Abstract image. Showing Pooch logo along with the title of the talk and\na link to Pooch's website: https://www.fatiando.org/pooch](_assets/image.png)\n\n\n## Description\n\nTutorials and examples are among the most critical assets that scientific\nPython libraries have to drive the adoption of their tools by the community.\nThey usually require some sample datasets to demonstrate the capabilities\nof their packages and to show how they can be used in comprehensive workflows.\nThese datasets should be readily available to the users so that they can focus\non learning and not micromanaging file downloads.\nShipping these data files along with the software is cumbersome, and makes the\npackages larger than needed.\nAlternatively, they could be hosted in a different location, which\nintroduces the need for an easy way to access them.\n\n[Pooch][pooch] is a slim, pure Python library that allows you to download and\ncache files from the web.\nIt is specifically designed to provide package developers with an easy way to\nmake their sample datasets available to their users, while also providing extra\nfeatures that make it useful in many other scenarios.\n\n\nWith the [`pooch.retrieve()`][pooch.retrieve] function we can download a file\nfrom the web, check its integrity through its checksum, cache it locally in the\ndesired location, and finally get the path to that file.\nIf the file has already been downloaded, the function will avoid re-downloading\nand simply return its path.\n\nTo manage the download of multiple data files, we make use of the\n[`pooch.Pooch`][pooch.Pooch] class, which can keep a record of the available\nfiles for download through a _registry_: a dictionary with all remote files and\ntheir hashes.\nThen these files can easily be downloaded and cached through the\n[`Pooch.fetch()`][Pooch.fetch] method.\nThe [`Pooch`][pooch.Pooch] class is particularly useful for package\nmaintainers who want to provide an easy way to download sample datasets for\ntheir libraries, as it supports ways to separate the cached data files for\ndifferent versions of their packages.\n\n[Pooch][pooch] supports downloading files using different protocols, like HTTP\nand FTP.\nIt can also download files from repositories like\n[Zenodo][zenodo], [figshare][figshare], and [dataverse][dataverse] directly\nthrough their [DOI (Digital object identifier)][doi].\nIt also offers a simple way to perform post-download tasks through\n_post-processors_. Pooch already includes built-in post-processor classes for\nunpacking zip and tar archives and decompressing files.\nIts modular design allows us to plug in custom downloaders, so we can make it\nwork with other protocols or APIs, or custom post-processors, to carry out any\nrequired task after the files have been downloaded.\n\n\nPooch was created as part of the [Fatiando a Terra][fatiando] project,\na community that develops open-source Python tools for geosciences.\nIt started as a collaboration between [Fatiando a Terra][fatiando] and\n[MetPy][metpy] to standardize how sample datasets are downloaded for gallery\nexamples and tutorials.\nIt became popular among the scientific Python community, and nowadays it is\nbeing used by projects like [SciPy][scipy], [scikit-image][scikit-image],\n[napari][napari] and [MNE Tools][mne-tools], among others.\nPooch has proved to be useful when running live-coding tutorials, as it allows\ninstructors and attendees to get their hands on the data without any frills,\nand also ensures that they have obtained the right data file.\n\nA peer-reviewed paper about Pooch was published in the [Journal of Open Source\nSoftware][joss] (doi: [10.21105/joss.01943][pooch-doi]), and presented in\na [lightning-talk][pycascades-pooch] at [PyCascades 2023][pycascades2023].\n\n\nDuring this talk, we'll show you how you can use the different features that\nPooch offers and also discuss how you can extend its capabilities by writing\nyour custom downloaders or post-processors.\n\n\n[santisoler]: https://www.santisoler.com\n[fatiando]: https://www.fatiando.org\n[pooch]: https://www.fatiando.org/pooch\n[pooch.retrieve]: https://www.fatiando.org/pooch/v1.8.1/api/generated/pooch.retrieve.html\n[pooch.pooch]: https://www.fatiando.org/pooch/v1.8.1/api/generated/pooch.Pooch.html\n[pooch.fetch]: https://www.fatiando.org/pooch/v1.8.1/api/generated/pooch.Pooch.html#pooch.Pooch.fetch\n[metpy]: https://unidata.github.io/MetPy\n[napari]: https://napari.org\n[scipy]: https://scipy.org\n[scikit-image]: https://scikit-image.org\n[doi]: https://en.wikipedia.org/wiki/Digital_object_identifier\n[zenodo]: https://zenodo.org/\n[figshare]: https://figshare.com/\n[dataverse]: https://dataverse.org/\n[mne-tools]: https://mne.tools\n[joss]: https://joss.theoj.org/\n[pooch-doi]: https://doi.org/10.21105/joss.01943\n[pycascades2023]: https://2023.pycascades.com/\n[pycascades-pooch]: https://www.youtube.com/watch?v=KvxBc4xUMyg\n[abstract]: https://cfp.scipy.org/2024/talk/AWFAQF\n[schedule]: https://cfp.scipy.org/2024/schedule\n[slides-pdf]: https://github.com/santisoler/scipy2024-pooch/raw/main/slides/slides.pdf\n[slides-odp]: https://github.com/santisoler/scipy2024-pooch/raw/main/slides/slides.odp\n\n## License\n\nThe abstract and the slides are distributed under a\n[Creative Commons Attribution 4.0 International License][cc-by].\n\n[![CC BY 4.0][cc-by-image]][cc-by]\n\n[cc-by]: http://creativecommons.org/licenses/by/4.0/\n[cc-by-image]: https://i.creativecommons.org/l/by/4.0/88x31.png\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsantisoler%2Fscipy2024-pooch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsantisoler%2Fscipy2024-pooch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsantisoler%2Fscipy2024-pooch/lists"}