https://github.com/singer-io/tap-closeio
Tap for Close.io
https://github.com/singer-io/tap-closeio
singer tap
Last synced: about 1 year ago
JSON representation
Tap for Close.io
- Host: GitHub
- URL: https://github.com/singer-io/tap-closeio
- Owner: singer-io
- License: agpl-3.0
- Created: 2017-01-10T15:13:44.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2025-01-21T11:43:48.000Z (over 1 year ago)
- Last Synced: 2025-04-05T02:06:01.153Z (about 1 year ago)
- Topics: singer, tap
- Language: Python
- Homepage:
- Size: 131 KB
- Stars: 7
- Watchers: 12
- Forks: 10
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# tap-closeio
This is a [Singer](https://singer.io) tap that produces JSON-formatted data following the [Singer spec](https://github.com/singer-io/getting-started/blob/master/SPEC.md).
This tap:
- Pulls raw data from Close.io's [REST API](https://developer.close.io/)
- Extracts the following resources from Close.io:
- [Activities](https://developer.close.io/#activities)
- [Leads](https://developer.close.io/#leads)
- Outputs the schema for each resource
- Incrementally pulls data based on the input state
## Quick start
1. Install
```bash
> pip install tap-closeio
```
2. Get your Close.io API Key
Login to your Close.io account, navigate to your account settings and "Your API Keys". Generate a New API Key, you'll need it for the next step.
3. Create the config file
Create a JSON file called `config.json` containing the api key you just
generated
```json
{
"start_date": "2010-01-01",
"api_key": "your-api-token",
"date_window": 10
}
```
The `start_date` is the date at which the tap will begin syncing data. Ie.
if there is data in your Close.io account older than `start_date`, it will
not be synced.
The `date_window` is the window in the which API calls will take place. The default window size is 15 days.
4. Run the tap in discovery mode
```bash
tap-closeio --config config.json --discover
```
See the Singer docs on discovery mode
[here](https://github.com/singer-io/getting-started/blob/master/BEST_PRACTICES.md#discover-mode-and-connection-checks).
5. Run the tap in sync mode
```bash
tap-closeio --config config.json --properties catalog.json
```
## Event Logs
The event log endpoint returns two fields that are troublesome in terms of
describing them with a JSON schema and fitting them into tabular structures,
like PostgreSQL or Redshift. They are the `data` and `previous_data` fields.
These fields vary depending on the type of the event, meaning an event for a
lead will have a vastly different structure than an event for a task. Due to
the varying nature of these fields, the tap JSON-encodes the fields during
sync.
## Activities
The activities endpoint does not provide a way to filter data based on when an
activity was updated. Because of this, there is no way to have the tap sync
changes to previously-synced activities without syncing the entire data set
during every run. As an alternative, your configuration file can contain the
key `activities_window_seconds`. When provided, any activity which was created
`activities_window_seconds` seconds before the bookmark in the `state.json`
file will be synced. For example, if your configuration file includes
```json
{
...,
"activities_window_seconds": 3600
}
```
and the previous run of tap synced activities up until 10am today, the next
sync will start syncing activities that were created at 9am today.
---
Copyright © 2017 Stitch