https://github.com/fivetran/prefect-fivetran
https://github.com/fivetran/prefect-fivetran
Last synced: 8 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/fivetran/prefect-fivetran
- Owner: fivetran
- License: apache-2.0
- Created: 2022-08-26T18:22:09.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2024-10-17T18:13:21.000Z (over 1 year ago)
- Last Synced: 2025-03-29T05:33:49.816Z (about 1 year ago)
- Language: Python
- Homepage: https://fivetran.github.io/prefect-fivetran/
- Size: 577 KB
- Stars: 14
- Watchers: 1
- Forks: 5
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# prefect-fivetran
## Welcome!
Prefect integrations with Fivetran
### Disclaimer:
Please note that Fivetran no longer officially supports or maintains this repository. If a customer needs to add or modify something here, they will need to make a fork and work with it on their own. This repository is being left public and accessible for the benefit of customers who have built services around this product. Any edits or modifications are the sole responsibility of the modifying party.
## Getting Started
### Python setup
Requires an installation of Python 3.7+.
We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.
These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to the [Prefect documentation](https://orion-docs.prefect.io/).
### Installation
Install `prefect-fivetran` with `pip`:
```bash
pip install prefect-fivetran
```
Then, register the blocks in this collection to [view them in Prefect Cloud](https://orion-docs.prefect.io/ui/blocks/):
```bash
prefect block register -m prefect_fivetran
```
Note, to use the `load` method on Blocks, you must already have a block document [saved through code](https://orion-docs.prefect.io/concepts/blocks/#saving-blocks) or [saved through the UI](https://orion-docs.prefect.io/ui/blocks/).
### Write and run a flow
```python
from prefect import flow
from prefect_fivetran import FivetranCredentials
from prefect_fivetran.connectors import trigger_fivetran_connector_sync_and_wait_for_completion
@flow
def my_flow():
...
fivetran_credentials = FivetranCredentials(
api_key="my_api_key",
api_secret="my_api_secret",
)
fivetran_result = await trigger_fivetran_connector_sync_and_wait_for_completion(
fivetran_credentials=fivetran_credentials,
connector_id="my_connector_id",
poll_status_every_n_seconds=30,
)
...
my_flow()
```
## Resources
If you encounter any bugs while using `prefect-fivetran`, feel free to open an issue in the [prefect-fivetran](https://github.com/pubchimps/prefect-fivetran) repository.
If you have any questions or issues while using `prefect-fivetran`, you can find help in either the [Prefect Discourse forum](https://discourse.prefect.io/) or the [Prefect Slack community](https://prefect.io/slack).
## Development
If you'd like to install a version of `prefect-fivetran` for development, clone the repository and perform an editable install with `pip`:
```bash
git clone https://github.com/pubchimps/prefect-fivetran.git
cd prefect-fivetran/
pip install -e ".[dev]"
# Install linting pre-commit hooks
pre-commit install
```