{"id":1990589,"url":"https://github.com/Skyscanner/pyfailsafe","last_synced_at":"2025-03-13T17:33:58.815Z","repository":{"id":54223349,"uuid":"65978108","full_name":"Skyscanner/pyfailsafe","owner":"Skyscanner","description":"Simple failure handling. Failsafe implementation in Python","archived":false,"fork":false,"pushed_at":"2023-07-20T15:09:14.000Z","size":137,"stargazers_count":90,"open_issues_count":6,"forks_count":8,"subscribers_count":16,"default_branch":"master","last_synced_at":"2024-10-12T09:22:13.755Z","etag":null,"topics":["aiohttp","asyncio","circuit-breakers","failsafe-call","fallbacks","python-3-5","retries"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Skyscanner.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-08-18T08:08:45.000Z","updated_at":"2024-08-29T22:02:44.000Z","dependencies_parsed_at":"2024-01-20T15:54:18.547Z","dependency_job_id":"40fa08fe-702b-4e75-a276-bbe80faad492","html_url":"https://github.com/Skyscanner/pyfailsafe","commit_stats":{"total_commits":120,"total_committers":14,"mean_commits":8.571428571428571,"dds":0.475,"last_synced_commit":"84c88e8ededefb1c0bc3e3ec9b5e8207aafe0812"},"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Skyscanner%2Fpyfailsafe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Skyscanner%2Fpyfailsafe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Skyscanner%2Fpyfailsafe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Skyscanner%2Fpyfailsafe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Skyscanner","download_url":"https://codeload.github.com/Skyscanner/pyfailsafe/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221391510,"owners_count":16811437,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aiohttp","asyncio","circuit-breakers","failsafe-call","fallbacks","python-3-5","retries"],"created_at":"2024-01-20T15:37:50.725Z","updated_at":"2024-10-25T05:30:59.417Z","avatar_url":"https://github.com/Skyscanner.png","language":"Python","readme":"# Pyfailsafe\n[![Build Status](https://travis-ci.org/Skyscanner/pyfailsafe.svg)](https://travis-ci.org/Skyscanner/pyfailsafe)\n[![PyPI](https://img.shields.io/pypi/v/pyfailsafe.svg)](https://pypi.python.org/pypi/pyfailsafe)\n\nA Python library for handling failures, heavily inspired by the Java project [Failsafe](https://github.com/jhalterman/failsafe).\n\nPyfailsafe provides mechanisms for dealing with operations that inherently can fail, such as calls to external \nservices. It takes advantage of the Python's coroutines, and supports both \"classic\" and async operations, starting \nfrom Python 3.5.\n\n  * [Installation](#installation)\n  * [Usage](#usage)\n    * [Bare Failsafe call](#bare-failsafe-call)\n    * [Failsafe call with retries](#failsafe-call-with-retries)\n    * [Failsafe call with abortable exceptions](#failsafe-call-with-abortable-exceptions)\n    * [Circuit breakers](#circuit-breakers)\n      * [CircuitBreaker interface](#circuitbreaker-interface)\n      * [Circuit breaker with retries](#circuit-breaker-with-retries)\n    * [RetryPolicy and CircuitBreaker events](#retrypolicy-and-circuitbreaker-events)\n    * [Using Pyfailsafe to make HTTP calls](#using-pyfailsafe-to-make-http-calls)\n      * [Making HTTP calls with fallbacks](#making-http-calls-with-fallbacks)\n  * [Examples](#examples)\n  * [Developing](#developing)\n  * [Publishing](#publishing)\n  * [Contributing](#contributing)\n\n## Installation\n\nTo get started using Pyfailsafe, install with\n\n```sh\npip install pyfailsafe\n```\n\nthen read the rest of this document to learn how to use it.\n\n## Usage\n\n### Bare Failsafe call\n\n```python\nfrom failsafe import Failsafe\n\nasync def my_async_function():\n    return 'done'\n\n# this is the same as just calling:\n# result = await my_async_function()\nresult = await Failsafe().run(my_async_function)\nassert result == 'done'\n```\n\n### Failsafe call with retries\n\nUse `RetryPolicy` class to define the number of retries which should be made before operation fails.\n\n```python\nfrom failsafe import Failsafe, RetryPolicy\n\nasync def my_async_function():\n    raise Exception()  # by default, every exception will cause a retry\n\nretry_policy = RetryPolicy(allowed_retries=3)\n\nawait Failsafe(retry_policy=retry_policy).run(my_async_function)\n# raises failsafe.RetriesExhausted\n# my_async_function was called 4 times (1 regular call + 3 retries)\n```\n\nBy default, retries are executed immediately - there is no backoff.\nIf you want to wait before executing a retry, you can use the `backoff` parameter:\n\n```python\nfrom datetime import timedelta\nfrom failsafe import Failsafe, RetryPolicy, Delay\n\nasync def my_async_function():\n    raise Exception()  # by default, every exception will cause a retry\n\ndelay = Delay(timedelta(seconds=5))\nretry_policy = RetryPolicy(allowed_retries=3, backoff=delay)\n\nawait Failsafe(retry_policy=retry_policy).run(my_async_function)\n# raises failsafe.RetriesExhausted\n# my_async_function was called 4 times, waiting for 5 seconds between each call.\n```\n\nWe also ship a `Backoff` class that supports jitter and incremental delay\n\n```python\nfrom datetime import timedelta\nfrom failsafe import Failsafe, RetryPolicy, Backoff\n\nasync def my_async_function():\n    raise Exception()  # by default, every exception will cause a retry\n\nbackoff = Backoff(\n    delay=timedelta(seconds=2),  # the initial delay\n    max_delay=timedelta(seconds=15),\n    jitter=False  # if True, the wait time will be random between 0 and the actual time for this attempt\n)\nretry_policy = RetryPolicy(allowed_retries=3, backoff=backoff)\n\nawait Failsafe(retry_policy=retry_policy).run(my_async_function)\n# raises failsafe.RetriesExhausted\n# my_async_function was called 4 times, waiting for 2, 4 and 8 seconds respectively.\n```\n\nIt is possible to provide your own backoff logic by subclassing the\n`failsafe.retry_logic.Backoff` class and overriding the `.for_attempt(attempt)` method.\n\nIt is possible to specify a particular set of exceptions that should cause a retry - any exception not contained in that set will cause immediate failure instead.\n\n```python\nfrom failsafe import Failsafe, RetryPolicy\n\nasync def my_async_function():\n    return 3/0\n\nretry_policy = RetryPolicy(allowed_retries=3, retriable_exceptions=[ZeroDivisionError])\n\nawait Failsafe(retry_policy=retry_policy).run(my_async_function)\n# raises failsafe.RetriesExhausted\n# my_async_function was called 4 times (1 regular call + 3 retries)\n```\n\n```python\nfrom failsafe import Failsafe, RetryPolicy\n\nasync def my_async_function():\n    raise TypeError()\n\nretry_policy = RetryPolicy(allowed_retries=3, retriable_exceptions=[ZeroDivisionError])\n\nawait Failsafe(retry_policy=retry_policy).run(my_async_function)\n# TypeError is not ZeroDivisionError, so my_async_function was called just once in this example\n```\n\nRetryPolicy instances are immutable and thread-safe. They can be safely shared between Failsafe instances.\n\n### Failsafe call with abortable exceptions\n\nIf you need your code to be able to raise certain exceptions that should not be handled by the failsafe,\nyou can add them as abortable_exceptions in RetryPolicy. This is useful when you know that the nature of failure was\nsuch that consecutive calls would never succeed.\n\n```python\nfrom failsafe import Failsafe, RetryPolicy\n\nasync def my_async_function():\n    raise ValueError()  # ValueError is an abortable exception, so it will not cause retry\n\nretry_policy = RetryPolicy(allowed_retries=4, abortable_exceptions=[ValueError])\n\nawait Failsafe(retry_policy=retry_policy).run(my_async_function)\n# raises ValueError\n# my_async_function was called 1 time (1 regular call)\n```\n\n### Circuit breakers\n\n[Circuit breakers](http://martinfowler.com/bliki/CircuitBreaker.html) are a way of creating systems that fail-fast by temporarily disabling execution as a way of preventing system overload.\n\n```python\nfrom failsafe import Failsafe, CircuitBreaker\n\nasync def my_async_function():\n    raise Exception()\n\ncircuit_breaker = CircuitBreaker(maximum_failures=3, reset_timeout_seconds=60)\nfailsafe = Failsafe(circuit_breaker=circuit_breaker)\n\nawait failsafe.run(my_async_function)\nawait failsafe.run(my_async_function)\nawait failsafe.run(my_async_function)\n# now circuit breaker will get open and other calls to Failsafe.run will\n# immediately raise the failsafe.CircuitOpen exception and the passed\n# function will not even be called.\n# Circuit will be closed again in 60 seconds.\n```\n\nA circuit breaker instance can and should be shared across code that accesses inter-dependent system components that fail together. This ensures that if the circuit is opened, executions against one component that rely on another component will not be allowed until the circuit is closed again.\n\nA circuit breaker instance is stateful - it remembers how many failures occur and whether the circuit is open or closed.\n\nA circuit breaker will not take into account abortable exceptions.\n\n#### CircuitBreaker interface\n\nA CircuitBreaker can also be manually operated in a standalone way:\n\n```python\nfrom failsafe import CircuitBreaker\n\ncircuit_breaker = CircuitBreaker()\n\ncircuit_breaker.open()  # executions won't be allowed when circuit breaker is open\ncircuit_breaker.close()\ncircuit_breaker.current_state  # 'open' or 'closed'\n\nif circuit_breaker.allows_execution():\n    try:\n        do_something()\n        circuit_breaker.report_success()\n    except:\n        circuit_breaker.report_failure()\n```\n\n#### Circuit breaker with retries\n\nIt is recommended to use circuit breakers together with retry policies. Every failed retry will count as a failure to the circuit breaker.\n\n```python\nfrom failsafe import Failsafe, CircuitBreaker, RetryPolicy\n\nasync def my_async_function():\n    raise Exception()\n\ncircuit_breaker = CircuitBreaker()\nretry_policy = RetryPolicy()\nfailsafe = Failsafe(circuit_breaker=circuit_breaker, retry_policy=retry_policy)\nawait failsafe.run(my_async_function)\n```\n\n### RetryPolicy and CircuitBreaker events\n\n`RetryPolicy` and `CircuitBreaker` accept event handlers at construction time, such as `on_retry`, `on_retries_exhausted`, \n`on_abort`, `on_failed_attempt` in the case of `RetryPolicy`, and `on_open`, `on_half_open`, `on_close` for \n`CircuitBreaker`. These event handlers can be useful for things like logging or metric recording.  \n\n```python\nimport logging\nfrom failsafe import Failsafe, CircuitBreaker, RetryPolicy\n\nlogger = logging.getLogger(\"MyLogger\")\ncircuit_breaker = CircuitBreaker(on_open=lambda: logger.error(\"Circuit open!\"))\nretry_policy = RetryPolicy(on_retry=lambda: logger.warning(\"Retrying...\"))\nfailsafe = Failsafe(retry_policy=retry_policy, circuit_breaker=circuit_breaker)\n```\n\n### Using Pyfailsafe to make HTTP calls\n\nFailsafe is not dependent on any HTTP client library, so a function making a call has to be provided by the developer. Said function must return a coroutine.\n\nThe example below uses aiohttp client to make a call.\n\n```python\nfrom failsafe import Failsafe, RetryPolicy, CircuitBreaker, FailsafeError\nimport aiohttp\n\ncircuit_breaker = CircuitBreaker()\nretry_policy = RetryPolicy()\nfailsafe = Failsafe(circuit_breaker=circuit_breaker, retry_policy=retry_policy)\n\nasync def make_get_request(url):\n    async def _make_get_request(_url):\n        with aiohttp.ClientSession() as session:\n            async with session.get(_url) as resp:\n                if resp.status != 200:\n                    raise Exception()  # exception tells Failsafe to retry\n                return await resp.json()\n\n    try:\n        return await failsafe.run(_make_get_request, url)\n    except FailsafeError:\n        raise RuntimeError(\"Error while getting data\")\n\n\nif __name__ == \"__main__\":\n    async def print_response():\n        from pprint import pprint\n        result = await make_get_request('https://api.github.com/users/skyscanner/repos')\n        pprint(result)\n\n\n    import asyncio\n\n    loop = asyncio.get_event_loop()\n    loop.run_until_complete(print_response())\n    loop.close()\n```\n\n#### Making HTTP calls with fallbacks\n\nUse FallbackFailsafe class to simplify handling fallbacks:\n\n```python\nimport aiohttp\nfrom urllib.parse import urljoin\n\nfrom failsafe import FallbackFailsafe\n\nclass SortingClient:\n    def __init__(self):\n        endpoint_main = \"http://eu-west-1.sorting-service.local\"\n        endpoint_secondary = \"http://eu-central-1.sorting-service.local\"\n\n        self.fallback_failsafe = FallbackFailsafe([endpoint_main, endpoint_secondary])\n\n    async def get_sorting(self, name, age):\n        query_path = \"/v1/sort/name/{0}/age/{1}\".format(name, age)\n        return await self.fallback_failsafe.run(self._request, query_path)\n\n    async def _request(self, endpoint, query_path):\n        url = urljoin(endpoint, query_path)\n\n        async with aiohttp.ClientSession() as session:\n            async with session.get(url) as resp:\n                if resp.status != 200:\n                    raise Exception()\n                return await resp.json()\n```\n\n## Examples\n\nIt is recommended to wrap calls in the class which will abstract away the outside service.\n\nCheck [examples](examples) folder for comprehensive examples of how Pyfailsafe should be used. See [examples/README.md](examples/README.md) to run examples.\n\n## Developing\n\nWhen making changes to the module it is always a good idea to run everything within a python virtual environment to ensure isolation of dependencies.\n\n```sh\n# Python 3.5 or greater needed\npython3 -m venv venv\nsource venv/bin/activate\npip install -r requirements_test.txt\n```\n\nUnit tests are written using pytest and can be run from the root of the project with\n\n```sh\npy.test tests/ -v\n```\n\nCoding standards are maintained using the flake8 tool which will run as part of the build process. To run locally simply use:\n\n```sh\nflake8 failsafe/ tests/ examples/\n```\n\n## Publishing\n\n1. Set new version number in `failsafe/init.py` and commit it\n2. `git tag X.Y.Z`\n3. `git push --tags`\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) file to add a contribution.\n\nMaintainers:\n\n- https://github.com/jakubka\n- https://github.com/carl0FF\n- https://github.com/cajturner\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSkyscanner%2Fpyfailsafe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSkyscanner%2Fpyfailsafe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSkyscanner%2Fpyfailsafe/lists"}