{"id":19493003,"url":"https://github.com/nuxeo/mkto-pd-sync-app","last_synced_at":"2025-02-25T20:19:24.930Z","repository":{"id":137033415,"uuid":"82191755","full_name":"nuxeo/mkto-pd-sync-app","owner":"nuxeo","description":"Flask application that synchronizes data between Marketo and Pipedrive.","archived":false,"fork":false,"pushed_at":"2019-07-17T08:54:24.000Z","size":969,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-01-08T09:13:05.837Z","etag":null,"topics":["flask","google-app-engine","marketo","pipedrive"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nuxeo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-02-16T14:52:46.000Z","updated_at":"2019-07-17T08:54:26.000Z","dependencies_parsed_at":"2023-04-28T07:24:32.652Z","dependency_job_id":null,"html_url":"https://github.com/nuxeo/mkto-pd-sync-app","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuxeo%2Fmkto-pd-sync-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuxeo%2Fmkto-pd-sync-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuxeo%2Fmkto-pd-sync-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuxeo%2Fmkto-pd-sync-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nuxeo","download_url":"https://codeload.github.com/nuxeo/mkto-pd-sync-app/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240738420,"owners_count":19849595,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["flask","google-app-engine","marketo","pipedrive"],"created_at":"2024-11-10T21:23:50.645Z","updated_at":"2025-02-25T20:19:24.911Z","avatar_url":"https://github.com/nuxeo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MKTO-PD Sync App\n\n[![Travis Build Status](https://travis-ci.org/hjonin/mkto-pd-sync-app.svg?branch=master)](https://travis-ci.org/hjonin/mkto-pd-sync-app)\n\nMKTO-PD Sync App is a Flask application running on a Google App Engine that synchronizes data between marketing software Marketo and CRM Pipedrive.\n\n\n## Features\n\nThe application handles user requests **asynchronously** using **task (push) queues**.\n\n```bazaar\n                     app.yaml      Application       queue.yaml                       worker.yaml\n                        +          enqueues task         +                                 +\n                        |                +               |                                 |\n             +----------+----------+     |     +---------+----------+           +----------+----------+\n             |                     |     |     |                    |           |                     |\n             |     Application     |     |     | Task Queue Service |           |       Worker        |\n  HTTP POST  |                     |     |     |                    | HTTP POST |                     |\n +----+-----\u003e+ +-----------------+ +-----+----\u003e+ +----------------+ +----------\u003e+ +-----------------+ |\n      |      | |Flask Application| |           | |   Push Queue   | |           | |Flask Application| |\n      |      | +-----------------+ +\u003c----+-----+ +----------------+ +\u003c----+-----+ +-----------------+ |\n      |      +---------------------+     |     +--------------------+     |     +---------------------+\n      |                                  |                                |                |\n      +                                  +                                +                +\nUser calls API                     Returns task                        Returns   Worker executes task\n                                   name and ETA                        sync\n                                                                       status\n\n```\n\n### Marketo module\n\nSimple Python client for the Marketo REST API.\n\n#### Features\n\n**Lead**, **opportunities**, **roles** and **companies** are **classes** that can be instantiated given a Marketo **client** instance.\n\nAll **fields** are **loaded** and available for **reading** and a defined **subset** for each entity is available for **updating**.\n\nAn entity also has an \"**id field**\" that will be used to match an existing entity.\n\nAn **existing** entity can be **loaded** given an identifier to the constructor and an eventual id field to look for different from its actual one\nwhich can be any exiting field for leads but should be a \"**searchable field**\" for opportunities, roles and companies.\n\nAn entity can be **saved**: it is then **created** or **updated** depending on if it matches an existing one.\n\nAn entity can be **deleted**.\n\n#### Usage\n\n```python\nimport sync.marketo as marketo\n\nmkto = marketo.MarketoClient(IDENTITY_ENDPOINT, CLIENT_ID, CLIENT_SECRET, API_ENDPOINT)\n\n# Get a lead by id\nlead = marketo.Lead(mkto, \"12345\")\n\n# Get a lead by email\nlead = marketo.Lead(mkto, \"lead@mail.com\", \"email\")\n\n# Update lead data\nlead.firstName = \"New Name\"\nlead.save()\n\n# Delete lead\nlead.delete()\n```\n\n### Pipedrive module\n\nSimple Python client for the Pipedrive REST API.\n\n#### Features\n\n**People**, **organizations**, **deals** and **activities** are classes that can be instantiated given a Pipedrive **client** instance.\n\nAll **fields** are **loaded** and available for reading and updating.\n\nAn **existing** entity can be **loaded** given an **identifier** or a **name** if the \"id field\" parameter is given as \"name\". A lead can also be loaded given a **custom filter** a the \"id field\". Currently supported filters are \"email domain\" and \"marketo id\".\n\n**Custom fields** can be read using the snake case display name rather than their hash key.\n\nSome entities are **related** to other and can be loaded as well if so.\n\nAn entity can be **saved**: it is then **created** or **updated** depending on if it matches an existing one.\n\nFor related entities, only the association can be updated (**no cascade update**).\n\nAn entity can be **deleted**.\n\n#### Usage\n\n```python\nimport sync.pipedrive as pipedrive\n\npd = pipedrive.PipedriveClient(PD_API_TOKEN)\n\n# Get a person by id\nperson = pipedrive.Person(pd, \"12345\")\n\n# Get an organization by name\norganization = pipedrive.Organization(pd, \"MyCompany\", \"name\")\n\n# Get a person custom field with its nicer name\nperson.lead_score  # or getattr(person, {HASH KEY})\n\n# Get a person related organization\norganization = person.organization\n\n# Update person data\nperson.name = \"New Name\"\nperson.save()\n\n# Delete person\nperson.delete()\n```\n\n### API reference\n\nAll requests to the API must be made over SSL (https not http) and validated against the `prod` key (see configuration).\n\n#### List of endpoints\n\n- **POST**: `/marketo/lead/\u003cint:lead_id\u003e`: to send lead data from Marketo to Pipedrive\n\nActually synchronizes data if and only if it is new or it has changed.\n\n- **POST**: `/pipedrive/person/\u003cint:person_id\u003e` (or `/pipedrive/person` for Pipedrive notification usage): to send person data from Pipedrive to Marketo\n\nActually synchronizes data if and only if it is new or it has changed.\n\n- **POST**: `/pipedrive/organization/\u003cint:organization_id\u003e` (or `/pipedrive/organization` for Pipedrive notification usage): to send organization data from Pipedrive to Marketo\n\nActually synchronizes data if and only if it is new or it has changed.\n\n- **POST**: `/pipedrive/deal/\u003cint:deal_id\u003e` (or `/pipedrive/deal` for Pipedrive notification usage): to send deal data from Pipedrive to Marketo\n\nActually synchronizes data if and only if it is new or it has changed.\n\n- **POST**: `/marketo/lead/\u003cint:lead_pipedrive_id\u003e/delete`: to delete a person in Pipedrive (using lead pipedrive id)\n\n- **POST**: `/pipedrive/person/\u003cint:pipedrive_marketo_id\u003e/delete` (or `/pipedrive/person/delete` for Pipedrive notification usage): to delete a lead in Marketo (using person marketo id)\n\n- **POST**: `/marketo/lead/\u003cint:lead_id\u003e/activity` : to send an activity to Pipedrive from Marketo lead data. It allows a parameter `type` which if set to `email_sent` send activities to Pipedrive from the latest Emails sent the same day in Marketo.\n\n- **POST**: `/pipedrive/organization/\u003cint:organization_id\u003e/compute` (or `/pipedrive/organization/compute` for Pipedrive notification usage): to compute organization data in Pipedrive\n\n- **POST**: `/pipedrive/deal/notify` (for Pipedrive notification usage only): to notify about a deal (status change and note added) in Slack\n\n### Mapping\n\nA mapping file describes how Marketo and Pipedrive entity fields should be matched.\n\n#### Structure\n```\nMAPPING_NAME = {\n    to_field_key_1: {\n        \"fields\": [from_field_key_1, [from_field_key_2...]],\n        [\"mode\": \"join\"/\"choose\",]\n        [\"pre_adapter\": function,]\n        [\"post_adapter\": function]\n    },\n    to_field_key_2: {\n        \"transformer\": function\n    }\n}\n```\n\n## Development\n\n### Requirements\n\n- Install and initialize the [Google Cloud SDK](https://cloud.google.com/sdk/docs/).\n- Create a Google Cloud Platform Console project.\n- Install [Eclipse](http://www.eclipse.org/downloads/packages/eclipse-ide-javascript-and-web-developers/neonr) and import the project.\n- Create a a [path variable](http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.platform.doc.user%2Fconcepts%2Fcpathvars.htm) `HOME` that targets the folder where you extracted the SDK (see [also](docs/assets/path_variable.png)).\n\n### Installation\n\nAssuming you already have [Python 2.7](https://www.python.org/download/releases/2.7/) installed:\n- Install the required packages into a `lib/` folder:\n```\npip install -r requirements.txt -t lib/\n```\n\n### Configuration\n\n* Create an `instance` folder in the root folder of the project.\n* Copy the `config.py.sample` file to this folder (remove the `.sample` extension) and set:\n  * `IDENTITY_ENDPOINT`, `CLIENT_ID`, `CLIENT_SECRET`, `API_ENDPOINT` from your Marketo credentials,\n  * `PD_API_TOKEN` from your Pipedrive API token,\n  * `FLASK_AUTHORIZED_KEYS` with any keys (one for unit testing and the other for production).\n  * `SLACK_WEBHOOK_URL` and `DATADOG_API_KEY`\n\n`DEBUG` and `TESTING` are logging control variables.\n\n### Running\n\n```\ndev_appserver.py -A sync-app app.yaml worker.yaml  # Start the local development server\n```\n\n### Testing\n\nMarketo and Pipedrive clients are tested in the `tests` module but should not be continuously run as they actually call Marketo and Pipedrive APIs. \n\nThe application is also tested in this module where calls to the APIs have been mocked (see files in the `resources` folder under `tests`).\n\nLaunch the test:\n\n```\nexport GOOGLE_APP_ENGINE={PATH_TO_YOUR_GOOGLE_SDK}/platform/google_appengine\npython -m tests.test_sync\n```\n\nTo run the test in Eclipse, you must change the `Working directory` to `Default` (see [this screenshot](docs/assets/run_config1.png)) and set an environment variable for `GOOGLE_APP_ENGINE` pointing to the directory where you extracted the SDK (see [this other screenshot](docs/assets/run_config2.png)) in your configuration.\n\nTo run a single unit test in Eclipse you can use the keybinding **CTRL + F9** while the file open.\n\n## Deployment\n\nYou can [upload](https://cloud.google.com/appengine/docs/python/tools/uploadinganapp) the application running the following command from within the root directory of the project (don't forget the `config.py` file):\n```\ngcloud app deploy app.yaml worker.yaml queue.yaml [--project [YOUR_PROJECT_ID]]\n```\nor\n```\ngcloud config set project marketo-1041\ngcloud components update\ngcloud app deploy app.yaml worker.yaml queue.yaml\n```\n\nOtherwise, this project is continuously delivered (tested and deployed) with Travis and triggered by a push to the master branch of this repository\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnuxeo%2Fmkto-pd-sync-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnuxeo%2Fmkto-pd-sync-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnuxeo%2Fmkto-pd-sync-app/lists"}