Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/mars/event-driven-functions

🚇 Invoke javascript functions in a Heroku app via Salesforce Platform Events
https://github.com/mars/event-driven-functions

Last synced: over 1 year ago
JSON representation

🚇 Invoke javascript functions in a Heroku app via Salesforce Platform Events

Awesome Lists containing this project

README 

          
# Event-driven Functions



Invoke javascript functions in a [Heroku app](https://www.heroku.com/platform) via [Salesforce Platform Events](https://developer.salesforce.com/docs/atlas.en-us.platform_events.meta/platform_events/platform_events_define_ui.htm).



💻👩‍🔬 *This project is a exploration into a new pattern for extending the compute capabilities of Salesforce. **Shared freely via [MIT license](LICENSE). Use at your own risk.***



* 📐 [Design](#user-content-design)

* 🏙 [Architecture](#user-content-architecture)

* 🌱 [Example](#user-content-example)

  * [Heroku function](#user-content-heroku-function)

  * [Account object](#user-content-account-object)

  * [Platform Events](#user-content-platform-events)

  * [Process Builder flows](#user-content-process-builder-flows)

* ✅ [Requirements](#user-content-requirements)

* ⚡️ [Usage](#user-content-usage)

  * [First time setup](#user-content-first-time-setup)

  * [Salesforce setup](#user-content-salesforce-setup)

  * [Run locally](#user-content-run-locally)

  * [Developing more functions](#user-content-developing-more-functions)

* 🎛 [Configuration](#user-content-configuration) via environment variables

  * [Authentication](#user-content-configure-authentication)

  * [Runtime behavior](#user-content-configure-runtime-behavior)

* 🔬 [Testing](#user-content-testing)

* 🚀 [Deployment](#user-content-deployment)

  * [Salesforce components](#user-content-deploy-salesforce-components)

  * [Heroku app](#user-content-deploy-heroku-app)



Design

------



The high-level flow is:



> Platform Event → **this app** → Platform Event



This flow maps specific **Invoke events** (topics) to function calls that return values by producing **Return events**.



> `Heroku_Function_*_Invoke__e` → Node.js function call → `Heroku_Function_*_Return__e`



These functions are composed in a Heroku app. Each function's arguments, return values, and their types must be encoded in the Invoke and Return events' fields.



Architecture

------------



This event-driven functions app is a **Node.js** app, along with an **sfdx** project providing the Salesforce customizations.



Based on [improvements to the **jsforce** Streaming module](https://github.com/jsforce/jsforce/pull/740) to support durable consumption of the Salesforce Streaming API. Those changes were merged to become **jsforce 1.9.0**.



Example

-------



This repo contains an example implementation of a UUID generator for Salesforce Accounts. This could generate UUIDs for any Salesforce object by implementing a Process Builder flow for each desired object type.



### Heroku function



Implemented in Node.js, an event-driven function is defined in the [JavaScript module `Generate_UUID`](lib/functions/Generate_UUID.js) and [registered as a function export](lib/functions/index.js).



### Account object



In **Salesforce Setup** → **Object Manager**, a custom field `UUID__c` is defined for Account.



This [UUID field](force-app/main/default/objects/Account/fields/UUID__c.field-meta.xml) is blank by default, to be filled with a universally unique external indentifier generated by the Heroku app/function.



### Platform Events



In **Salesforce Setup** → **Platform Events**, two events are defined.



#### Invoke event



Salesforce Platform Event [`Heroku_Function_Generate_UUID_Invoke__e`](force-app/main/default/objects/Heroku_Function_Generate_UUID_Invoke__e)



```json

{

  "Context_Id__c": "xxxxx"

}

```



* `Context_Id` should be passed-through unchanged from Invoke to Return. It provides an identifier to associate the return value with the original invocation

* This example  is a minimal Invoke event payload with no additional fields for function arguments. When defining the Platform Event in Salesforce Setup, it may contain as many fields as necessary

* This object is included in the Salesforce [Permission Set](force-app/main/default/permissionsets/Heroku_Function_Generate_UUID.permissionset-meta.xml) for this app.



#### Return event



Salesforce Platform Event  [`Heroku_Function_Generate_UUID_Return__e`](https://github.com/mars/event-driven-functions/tree/master/force-app/main/default/objects/Heroku_Function_Generate_UUID_Return__e)



```json

{

  "Context_Id__c": "xxxxx",

  "Value__c": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx"

}

```



* `Context_Id__c` should be passed-through unchanged from Invoke to Return

* `Value__c` is a minimal Return event payload. In this example it contains the string UUID

* This object is included in the Salesforce [Permission Set](force-app/main/default/permissionsets/Heroku_Function_Generate_UUID.permissionset-meta.xml) for this app.



👓 *About the suffixes in Salesforce identifiers: `__e` is appended to Platform Event names, and `__c` is appended to custom object & field names.*



### Process Builder flows



In **Salesforce Setup** → **Process Builder**, two flows are defined.



#### Generate UUID for Account



[The first flow](force-app/main/default/flows/Generate_UUID_for_Account-1.flow-meta.xml) triggers when creating or updating an Account that does not yet have a UUID. This publishes an [Invoke event](#user-content-invoke-event).



#### Set UUID for Account



[The second flow](force-app/main/default/flows/Set_UUID_for_Account-1.flow-meta.xml) triggers when a [Return event](#user-content-return-event) is received. This updates the Account with the returned UUID.



Requirements

------------



* [Node.js](https://nodejs.org/) 8.11 with npm 5

* [redis](https://redis.io)



Usage

-----



### First time setup



```bash

git clone https://github.com/mars/event-driven-functions.git

cd event-driven-functions/

npm install

cp .env.sample .env

```



### Salesforce setup



Next, we'll use [`sfdx`](https://developer.salesforce.com/docs/atlas.en-us.212.0.sfdx_dev.meta/sfdx_dev/sfdx_dev_intro.htm) to deploy the Salesforce customizations. If you don't yet have access to a Dev Hub org, or this is your first time using `sfdx`, then see [**Setup Salesforce DX** in Trailhead](https://trailhead.salesforce.com/trails/sfdx_get_started/modules/sfdx_app_dev/units/sfdx_app_dev_setup_dx).



Deploy the included `force-app` code to a scratch org:



```bash

sfdx force:org:create -s -f config/project-scratch-def.json -a EventDrivenFunctions

sfdx force:source:push

sfdx force:user:permset:assign -n Heroku_Function_Generate_UUID

```



View the scratch org description:



```bash

sfdx force:user:display

```



Then, update `.env` file with the **Instance Url** & **Access Token** values from the scratch org description:



```

SALESFORCE_INSTANCE_URL=xxxxx

SALESFORCE_ACCESS_TOKEN=yyyyy

```



⚠️ *Scratch orgs and their authorizations expire, so this setup may need to be repeated whenever beginning local development work. View the current status of the orgs with `sfdx force:org:list`.*



### Run locally



Open the scratch org's Accounts:



```bash

sfdx force:org:open --path one/one.app#/sObject/Account/list

```



Run this `node` command in a shell terminal:



```bash

READ_MODE=changes \

PLUGIN_NAMES=invoke-functions \

OBSERVE_SALESFORCE_TOPIC_NAMES=/event/Heroku_Function_Generate_UUID_Invoke__e \

node lib/exec

```



🔁 *This command runs continuously, listening for the Platform Event.*



▶️ Watch this command's output as your create Accounts in the scratch org.



### Developing more functions



For a given function, three identifiers are used.



* **Function Name**

  * example `Generate_UUID`

  * used for the JavaScript module file & its `functions` export

* **Invoke Event Name**

  * example `Heroku_Function_Generate_UUID_Invoke__e`

  * used for the Platform Event that runs the function

* **Return Event Name**

  * example `Heroku_Function_Generate_UUID_Return__e`

  * used for the Platform Event that receives the function's result



Note: the **Function Name** must be embedded exactly in both **Event Names**.



To implement a new function:



1. create the new Platform Events using `sfdx` workflow

   * develop in a scratch org and pull changes into this repo

   * define each **Invoke & Return Event** and its schema (fields & their types)

   * define a new **Permission Set** for access or add to an existing Set

1. create the function as a default export in `lib/functions/`

   * use **Function Name** for the module file & its `functions` export

   * the function receives the Invoke event's payload and must honor the Return event's schema

   * see: [example `Generate_UUID.js`](lib/functions/Generate_UUID.js) & [the export](lib/functions/index.js)

1. include the new Invoke Event Name in `OBSERVE_SALESFORCE_TOPIC_NAMES` env var

   * example: `OBSERVE_SALESFORCE_TOPIC_NAMES=/event/Heroku_Function_Generate_UUID_Invoke__e,/event/Heroku_Function_Generate_Haiku_Invoke__e`



Configuration

-------------



### Configure Authentication



Performed based on environment variables. Either of the following authentication methods may be used:



* Username + password

  * `SALESFORCE_USERNAME`

  * `SALESFORCE_PASSWORD` (password+securitytoken)

  * `SALESFORCE_LOGIN_URL` (optional; defaults to **login.salesforce.com**)

* Existing OAuth token

  * `SALESFORCE_INSTANCE_URL`

  * `SALESFORCE_ACCESS_TOKEN`

  * Retrieve from an [sfdx](https://developer.salesforce.com/docs/atlas.en-us.212.0.sfdx_dev.meta/sfdx_dev/sfdx_dev_intro.htm) scratch org with:



    ```bash

    sfdx force:org:create -s -f config/project-scratch-def.json -a EventDrivenFunctions

    sfdx force:org:display

    ```

* OAuth client

  * `SALESFORCE_URL`

    * *Must include oAuth client ID, secret, & refresh token*

    * Example: `force://{client-id}:{secret}:{refresh-token}@{instance-name}.salesforce.com`



### Configure Runtime Behavior



* `VERBOSE`

  * enable detailed runtime logging to stderr

  * example: `VERBOSE=true`

  * default value: unset, no log output

* `PLUGIN_NAMES`

  * configure the consumers/observers of the Salesforce data streams

  * example: `PLUGIN_NAMES=invoke-functions`

  * default value: `console-output`

* `OBSERVE_SALESFORCE_TOPIC_NAMES`

  * the path part of a Streaming API URL

  * a comma-delimited list

  * example: `OBSERVE_SALESFORCE_TOPIC_NAMES=/event/Heroku_Function_Generate_UUID_Invoke__e`

  * default value: no Salesforce observer

* `REDIS_URL`

  * connection config to Redis datastore

  * example: `REDIS_URL=redis://localhost:6379`

  * default: unset, no Redis

* `REPLAY_ID`

  * force a specific replayId for Salesforce Streaming API

  * ensure to unset this after usage to prevent the stream from sticking

  * example: `REPLAY_ID=5678` (or `-2` for all possible events)

  * default: unset, receive all new events



Testing

-------



Implemented with [AVA](https://github.com/avajs/ava), concurrent test runner.



`npm test` runs only unit tests. It skips integration tests, because Salesforce and AWS config is not automated.



### Unit Tests



* `npm run test:unit`

* Defined in `lib/` alongside source files

* Salesforce API calls are mocked by way of [Episode 7](https://github.com/mars/episode-7)



Deployment

----------



### Deploy Salesforce components



#### Example, ready-to-install package from this repo



```bash

sfdx force:auth:web:login -a AnotherOrg

sfdx force:package:install --id 04tf4000001ft4hAAA -u AnotherOrg

```



#### Custom, package it yourself



##### 2nd-generation unnamespaced, unlocked package (2GP)



Based on [Salesforce DX 2GP docs](https://developer.salesforce.com/docs/atlas.en-us.sfdx_dev.meta/sfdx_dev/sfdx_dev_dev2gp_workflow.htm):



```bash

sfdx force:package2:create --name Event_Driven_Functions_Generate_UUID_2GP_sans_NS --description "Integration with event-driven-functions Heroku app" --containeroptions Unlocked --nonamespace

```



In `sfdx-project.json`, update `packageDirectories`.`0`.`id` with the output `Package2 Id`:



```json

{

  "packageDirectories": [

    {

      "path": "force-app",

      "default": true,

      "id": "0Hof4000000blNuCAI",

      "versionName": "Initial 2GP",

      "versionNumber": "1.0.0"

    }

  ],

  "namespace": "",

  "sfdcLoginUrl": "https://login.salesforce.com",

  "sourceApiVersion": "42.0"

}

```



```bash

sfdx force:package2:version:create --directory force-app --wait 10

sfdx force:package:install --id  -u OrgAliasOrUserId --wait 10 --publishwait 10

sfdx force:org:open -u EventDrivenFunctions

```



##### 1st-generation unmanaged (unnamespaced) package



Follow [Build and Release Your App with Managed Packages](https://developer.salesforce.com/docs/atlas.en-us.sfdx_dev.meta/sfdx_dev/sfdx_dev_build_man_pack.htm) to prepare a packaging org.



Diverging from those directions, we'll prepare an unmanaged package without a namespace. We have to skip namespacing for now, because of problems with Process Builder + Platform Events embedding namespaces in the metadata (evidence [1](NOTES.md#user-content-push-to-scratch-org-error), [2](NOTES.md#user-content-manually-pruned-package-fails-to-install)). [Link its namespace with your Hub org](https://developer.salesforce.com/docs/atlas.en-us.sfdx_dev.meta/sfdx_dev/sfdx_dev_reg_namespace.htm), and set the established `"namespace"` in [sfdx-project.json](sfdx-project.json), and then [provision & push to a fresh scratch org](#user-content-salesforce-setup).



Now, pull the Salesforce customizations back out of the scratch org in the Metadata API format:



```bash

sfdx force:source:convert --outputdir mdapi_output_dir --packagename Event_Driven_Functions_Generate_UUID

```



Login to the packaging org and create the Beta package:



```bash

sfdx force:org:list

sfdx force:auth:web:login -a PkgFunctions



sfdx force:mdapi:deploy --deploydir mdapi_output_dir --targetusername PkgFunctions



# Find the package ID in the URL of the packaging org:

#   Setup → Package Manager → View/Edit the Package

sfdx force:package1:version:create --packageid XXXXX --name r00000 -u PkgFunctions



sfdx force:package1:version:list -u PkgFunctions

```



Install the beta package into another org by its id `METADATAPACKAGEVERSIONID`:



```bash

sfdx force:auth:web:login -a AnotherOrg

sfdx force:package:install --id YYYYY -u AnotherOrg

```



### Deploy Heroku app



```bash

heroku create



heroku config:set \

  SALESFORCE_USERNAME=mmm@mmm.mmm \

  SALESFORCE_PASSWORD=nnnnnttttt \

  VERBOSE=true \

  PLUGIN_NAMES=invoke-functions \

  OBSERVE_SALESFORCE_TOPIC_NAMES=/event/Heroku_Function_Generate_UUID_Invoke__e \

  READ_MODE=changes



heroku addons:create heroku-redis:premium-0



git push heroku master



heroku ps:scale web=0:Standard-1x worker=1:Standard-1x

```