https://github.com/vmesel/request-miner

https://github.com/vmesel/request-miner

Last synced: 1 day ago
JSON representation

• Host: GitHub
• URL: https://github.com/vmesel/request-miner
• Owner: vmesel
• License: mit
• Created: 2023-09-04T20:25:39.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-09-04T20:44:15.000Z (over 1 year ago)
• Last Synced: 2024-11-18T03:38:16.011Z (about 2 months ago)
• Language: Python
• Size: 4.88 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Request Miner

A tool that enables you to automatically request and retry failing requests for a certain amount of times.

This tool is built on top of: `requests` and `backoff`, which this package relies on.

## Installation

```bash

pip install request-miner

```

## Usage

To create a request without passing a session:

```python

from request_miner import mine

from requests.exceptions import RequestException

get_req = mine(method="GET", url="https://www.google.com")

post_req = mine(

    method="POST",

    url="https://www.google.com",

    json={"key": "value"},

    backoff_type = "expo", # default is constant, but you can override with existing backoff functions or your own

    max_tries = 3, # max tries of a single request

    max_time = 10, # max time of a request

    exception = RequestException, # exception types that can be handled

    raise_on_giveup = True # if you want to raise the exception on giveup

) # This request will fail 

```

To create a prepared request with a session:

```python

import requests

from requests import Request, Session

from request_miner import process_request

from requests.exceptions import RequestException

session = requests.Session()

get_req = Request(method="GET", url="https://www.google.com").prepare()

process_request(

    get_req,

    session,

    backoff_type = "expo", # default is constant, but you can override with existing backoff functions or your own

    max_tries = 3, # max tries of a single request

    max_time = 10, # max time of a request

    exception = RequestException, # exception types that can be handled

    raise_on_giveup = True # if you want to raise the exception on giveup

)

```