{"id":19874195,"url":"https://github.com/pythoninthegrass/download","last_synced_at":"2025-08-15T14:14:35.152Z","repository":{"id":196397323,"uuid":"696000242","full_name":"pythoninthegrass/download","owner":"pythoninthegrass","description":"Feed requests a URL, extension, and local directory to download files scraped from a website.","archived":false,"fork":false,"pushed_at":"2023-12-23T16:49:30.000Z","size":6175,"stargazers_count":1,"open_issues_count":5,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-11T16:50:31.144Z","etag":null,"topics":["python"],"latest_commit_sha":null,"homepage":"","language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pythoninthegrass.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["pythoninthegrass"]}},"created_at":"2023-09-24T21:01:47.000Z","updated_at":"2023-09-25T14:15:05.000Z","dependencies_parsed_at":"2024-11-12T16:34:51.629Z","dependency_job_id":null,"html_url":"https://github.com/pythoninthegrass/download","commit_stats":null,"previous_names":["pythoninthegrass/download"],"tags_count":0,"template":false,"template_full_name":"pythoninthegrass/mvp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pythoninthegrass%2Fdownload","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pythoninthegrass%2Fdownload/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pythoninthegrass%2Fdownload/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pythoninthegrass%2Fdownload/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pythoninthegrass","download_url":"https://codeload.github.com/pythoninthegrass/download/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241301907,"owners_count":19940730,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python"],"created_at":"2024-11-12T16:22:17.098Z","updated_at":"2025-03-01T01:23:19.714Z","avatar_url":"https://github.com/pythoninthegrass.png","language":"Makefile","readme":"# download\n\n[**Download**](https://www.youtube.com/watch?v=T5jHO4jMPxA)  \n\u003cimg src=\"static/sfa_radiator.png\" width=\"300\"\u003e\n\n## Summary\nFeed `requests` a URL, extension, and local directory to download files scraped from a website.\n\nCache initial response to avoid too many requests to server.\n\n**Table of Contents**\n* [download](#download)\n  * [Summary](#summary)\n  * [Setup](#setup)\n  * [Quickstart](#quickstart)\n    * [Demo](#demo)\n    * [Usage](#usage)\n  * [Development](#development)\n  * [TODO](#todo)\n  * [Further Reading](#further-reading)\n\n## Setup\n* Dependencies\n  * make\n    * [Linux](https://www.gnu.org/software/make/)\n    * [macOS](https://www.freecodecamp.org/news/install-xcode-command-line-tools/)\n  * [editorconfig](https://editorconfig.org/)\n* Install python and tooling\n    ```bash\n    # install python and dependencies (e.g., git, ansible, etc.)\n    ./bootstrap install\n    ```\n\n## Quickstart\n### Demo\n\u003cimg src=\"static/demo.gif\" width=\"850\"\u003e\n\n### Usage\n```bash\n# run script with defaults\n./main.py\n\n# add environment variables for url, ext, and path\ncp .env.example .env\n\n# fill out (e.g., 'EXT=\".csv\"')\n\n# run with overrides\n./main.py\n```\n\n## Development\n```bash\n# install tools and runtimes (cf. xcode, brew, asdf, poetry, etc.)\n./bootstrap \u003crun|run-dev\u003e   # dev only runs plays w/tags and is verbose\n\n# install git hooks\n./bootstrap install-precommit\n\n# update git hooks\n./bootstrap update-precommit\n```\n\n## TODO\n* [Open Issues](https://github.com/pythoninthegrass/download/issues)\n\n## Further Reading\n* [python](https://www.python.org/)\n* [asdf](https://asdf-vm.com/guide/getting-started.html#_2-download-asdf)\n* [poetry](https://python-poetry.org/docs/)\n* [docker-compose](https://docs.docker.com/compose/install/)\n* [pre-commit hooks](https://pre-commit.com/)\n","funding_links":["https://github.com/sponsors/pythoninthegrass"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpythoninthegrass%2Fdownload","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpythoninthegrass%2Fdownload","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpythoninthegrass%2Fdownload/lists"}