Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/germainlefebvre4/bspauto-bash-scraper-v1

Scraper to retrieve the list of car prices on a car rental search from BSP Auto page.
https://github.com/germainlefebvre4/bspauto-bash-scraper-v1

bash hacktoberfest python scraper

Last synced: 15 days ago
JSON representation

Scraper to retrieve the list of car prices on a car rental search from BSP Auto page.

Host: GitHub
URL: https://github.com/germainlefebvre4/bspauto-bash-scraper-v1
Owner: germainlefebvre4
License: mit
Created: 2020-01-07T21:21:22.000Z (almost 5 years ago)
Default Branch: master
Last Pushed: 2023-02-28T23:23:36.000Z (over 1 year ago)
Last Synced: 2024-10-10T20:52:22.154Z (about 1 month ago)
Topics: bash, hacktoberfest, python, scraper
Language: Shell
Homepage:
Size: 22.5 KB
Stars: 0
Watchers: 3
Forks: 0
Open Issues: 3
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

        # BSP Auto Bash Scraper

## Getting started

```bash

./bsp-auto.sh --aws-profile my_aws_profile --aws-bucket my-aws-bucket --quiet --airport '3715' --departure-date '21/05/2023' --departure-time '08:00' --return-date '30/05/2023' --return-time '20:00' --cars 'PEUGEOT 208'

./bsp-auto.sh --aws-profile my_aws_profile --aws-bucket my-aws-bucket --quiet --airport '3715' --departure-date '21/05/2023' --departure-time '08:00' --return-date '30/05/2023' --return-time '20:00' --cars 'RENAULT CLIO,PEUGEOT 208,FIAT 500 X,VW GOLF,RENAULT MEGANE,JEEP RENEGADE,PEUGEOT 308,VW PASSAT,PEUGEOT 3008' --current-price 500 --ntfy-token 'my_ntfy_token'

```

Output:

```raw

From 21/05/2023 08:00 to 30/05/2023 20:00

URL: https://www.bsp-auto.com/fr/list.asp?ag_depart=2715&date_a=21%2F05%2F2023&heure_a=08%3A00&ag_retour=2715&date_d=30%2F05%2F2023&heure_d=20%3A00&chkage=1&age=25

Result file: results/20230521-08h00_20230530-20h00/2715/results.csv

date,RENAULT CLIO,PEUGEOT 208,FIAT 500 X,VW GOLF,RENAULT MEGANE,JEEP RENEGADE,PEUGEOT 308,VW PASSAT,PEUGEOT 3008,Current Price

2023-02-21 12:27:57,433,437,454,475,518,547,553,566,576,

2023-02-21 12:17:07,433,437,454,475,518,547,553,566,576,

2023-02-21 12:16:33,433,437,454,475,518,547,553,566,576,

upload: results/20230521-08h00_20230530-20h00/2715/results.csv to s3://my-aws-bucket/results/20230521-08h00_20230530-20h00/2715/results.csv

New lowest price: 334 for PEUGEOT 437

```

## Configuration

### Script parameters

| Name           | Description                                                              | Variable               | Parameter              | Short param | Long param         | Default value    | Example                             |

|----------------|--------------------------------------------------------------------------|------------------------|------------------------|-------------|--------------------|------------------|-------------------------------------|

| AWS Profile    | AWS profile name set into file `~/.aws/credentials`.                     | 
[x] 
 | [x]  | `-profile`  | `--aws-profile`    | `my_aws_profile` | `--aws-profile my_aws_profile`      |

| AWS Bucket     | AWS Bucket name where store the results.                                 | [x] 
 | [x]  | `-bucket`   | `--aws-bucket`     | `my-aws-bucket`  | `--aws-bucket my-aws-bucket`        |

| Quiet          | Reduce verbosity of the script.                                          | [x] 
 | [x]  | `-q`        | `--quiet`          | *None*           | `--quiet`                           |

| Airport ID     | ID of the Airport for departure AND return.                              |                        | [x]  | `-a`        | `--airport`        | *None*           | `--airport '3720'`                   |

| Departure date | Date of the departure in format `DD/MM/YYYY`.                            |                        | [x]  | `-dd`       | `--departure-date` | *None*           | `--departure-date '13/05/2023'`     |

| Departure time | Time of the departure in format `HH:MM`.                                 |                        | [x]  | `-dt`       | `--departure-time` | *None*           | `--departure-tome '13:00'`          |

| Return date    | Date of the return in format `DD/MM/YYYY`.                               |                        | [x]  | `-rd`       | `--return-date`    | *None*           | `--return-date '25/05/2023'`        |

| Return time    | Time of the return in format `HH:MM`.                                    |                        | [x]  | `-rt`       | `--return-time`    | *None*           | `--return-time '10:00'`             |

| Cars           | List of cars to watch for.                                               |                        | [x]  | `-ca`       | `--cars`           | *None*           | `--cars 'RENAULT CLIO,PEUGEOT 208'` |

| Current price  | Price to show as static value for comparison (usually the booked price). | [x] 
 | [x]  | `-cp`       | `--current-price`           | *None*           | `--current price 500` |

| Ntfy token | Token to trigger phone push notification from [Ntfy](https://ntfy.sh/) platform (free).                                               |                        | [x] 
 | `-ntfy`       | `--ntfy-token`           | *None*           | `--ntfy-token 'my_tnfy_token'` |

## Cronjob

### Add the parser script as cronjob

```bash

crontab -e

*/30 * * * * /opt/bspauto/bspauto-bash-scraper-v1/bsp-auto.sh --aws-profile my_aws_profile --aws-bucket bspauto-results --quiet --airport '3720' --departure-date '14/05/2023' --departure-time '13:30' --return-date '25/05/2023' --return-time '10:30' --current-price '575'

```

## Dockerhub

Link to Dockerhub : [https://hub.docker.com/r/germainlefebvre4/bspauto-bash-scraper](https://hub.docker.com/r/germainlefebvre4/bspauto-bash-scraper)