https://github.com/artefactory/artefactory-connectors-kit

ACK is an E(T)L tool specialized in API data ingestion. It is accessible through a Command-Line Interface. The application allows you to easily extract, stream and load data (with minimum transformations), from the API source to the destination of your choice.
https://github.com/artefactory/artefactory-connectors-kit

adobe-analytics amazon-s3 api confluence dcm dv360 facebook google-ads google-analytics google-cloud-storage google-sheets mysql mytarget radarly salesforce search-console the-trade-desk twitter yandex

Last synced: 4 days ago
JSON representation

ACK is an E(T)L tool specialized in API data ingestion. It is accessible through a Command-Line Interface. The application allows you to easily extract, stream and load data (with minimum transformations), from the API source to the destination of your choice.

• Host: GitHub
• URL: https://github.com/artefactory/artefactory-connectors-kit
• Owner: artefactory
• License: lgpl-3.0
• Created: 2020-04-02T10:39:01.000Z (over 4 years ago)
• Default Branch: dev
• Last Pushed: 2023-03-27T21:55:19.000Z (over 1 year ago)
• Last Synced: 2023-03-27T22:28:03.184Z (over 1 year ago)
• Topics: adobe-analytics, amazon-s3, api, confluence, dcm, dv360, facebook, google-ads, google-analytics, google-cloud-storage, google-sheets, mysql, mytarget, radarly, salesforce, search-console, the-trade-desk, twitter, yandex
• Language: Python
• Homepage:
• Size: 65.3 MB
• Stars: 35
• Watchers: 7
• Forks: 4
• Open Issues: 31
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Artefactory Connectors Kit

**ACK is an E(T)L tool specialized in API data ingestion. It is accessible through a Command-Line Interface. The application allows you to easily extract, stream and load data (with minimum transformations), from the API source to the destination of your choice.**

As of now, the most common output format of data loaded by the application is .njson (i.e. a file of n lines, where each line is a json-like dictionary).

Official documentation is available [here](https://artefactory.github.io/artefactory-connectors-kit/).

---

## Philosophy

The application is composed of **3 main components** (*implemented as Python classes*). When combined, these components act as an E(T)L pipeline, allowing you to stream data from a source to the destination of your choice:

- [Readers](ack/readers) are reading data from an API source, and transform it into a stream object.

- [Streams](ack/streams) (*transparent to the end-user*) are local objects used by writers to process individual records collected from the source.

- [Writers](ack/writers) are writing the output stream object to the destination of your choice.

## Why not Airbyte ?

[Airbyte](https://github.com/airbytehq/airbyte) is an open source project that has a mission to make data integration pipelines a commodity.

We love this project and would probably not encourage using ACK to connect data that could be already connected with Airbyte.

You can still use ACK if some of your data has no connector available in Airbyte yet.

The list of connectors unavailable in Airbyte will most certainly decrease over time following the introduction of the [Python Connector Development Kit](https://github.com/airbytehq/airbyte/tree/master/airbyte-cdk/python).

## Available connectors

As of now, the application is offering the following Readers & Writers:

 

### Readers

- **Analytics**

    - Adobe Analytics 1.4

    - Adobe Analytics 2.0

    - Google Analytics (available in Airbyte)

- **Advertising - Adserver**

    - Google Campaign Manager

- **Advertising - DSP**

    - Google Display & Video 360

    - The Trade Desk

- **Advertising - Search**

    - Google Ads (available in Airbyte)

    - Google Search Ads 360

    - Google Search Console

    - Yandex Campaign

    - Yandex Statistics

- **Advertising - Social**

    - Facebook Marketing (available in Airbyte)

    - MyTarget

    - Radarly

    - Twitter Ads

- **Affiliates**

    - AWIN Advertiser

- **CRM**

    - SalesForce (available in Airbyte)

- **Databases**

    - MySQL (available in Airbyte)

- **DevTools**

    - Confluence

- **Files (.csv, .njson)**

    - Amazon S3

    - Google Cloud Storage

    - Google Sheets (available in Airbyte)

### Writers

- **Data Warehouses**

    - Google BigQuery (available in Airbyte)

- **Debugging**

    - Console

- **Files (.njson)**

    - Amazon S3

    - Google Cloud Storage

    - Azure Blob Storage

    - Local file (available in Airbyte)