https://github.com/av1m/datagovuk-scraper
Scrap public data from data.gov.uk without an API KEY
https://github.com/av1m/datagovuk-scraper
public-data python-scraper python3 scraper
Last synced: 3 months ago
JSON representation
Scrap public data from data.gov.uk without an API KEY
- Host: GitHub
- URL: https://github.com/av1m/datagovuk-scraper
- Owner: av1m
- License: mit
- Created: 2022-04-10T01:14:27.000Z (about 3 years ago)
- Default Branch: master
- Last Pushed: 2023-07-20T15:35:20.000Z (almost 2 years ago)
- Last Synced: 2025-01-18T09:34:04.131Z (5 months ago)
- Topics: public-data, python-scraper, python3, scraper
- Language: Python
- Homepage:
- Size: 39.1 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# datagovuk-scraper
[](https://docs.python.org/3/whatsnew/3.10.html)
[](https://github.com/av1m/datagovuk-scraper/blob/master/LICENSE)
Scrap public data from data.gov.uk without an API KEY
## Get started 🎉
1. Clone the project
```bash
git clone https://github.com/av1m/datagovuk-scraper
cd datagovuk-scraper
```
2. Install dependencies
```bash
pip install -r requirements.txt
```
3. Run the project 🚀
You can run the project in different ways.
* To get started quickly, you can use the [notebook](get-started-sample.ipynb).
* Plus, you can use the [command line](#cli)
* Or, directly in your code by importing the `datagovuk` module.
## CLI Usage 📖
> 💡 You can directly install dependencies instead of clone the repository :
>
> ```bash
> pip install git+https://github.com/av1m/datagovuk-scraper.git
> ```
>
To run the project from the command line, use the command `datagovuk`.
Here is the list of available commands:
```bash
datagovuk --help
usage: __main__.py [-h] --query QUERY --number-record NUMBER_RECORD [--output {csv,ods,html,pdf,xls,zip}] [-d] [-v]
Scrap public data from data.gov.uk
options:
-h, --help show this help message and exit
--query, -q Search query (required)
--number-record, -n Number of records to fetch, must be a valid number (required)
--output, -o Output file (must be csv,ods,html,pdf,xls,zip). Default is csv
-d, --debug Put the logger in debug mode (default: Warning)
-v, --verbose Put the logger in info mode (default: Warning)
```
And, here an example of some examples:
* Get the first 10 records for the query `"house"` and with only CSV file as output (debug mode activated):
```bash
datagovuk -q house -n 10 -o csv -d
```