https://github.com/raisely/airblast

Robust cloud function job queuing and processing
https://github.com/raisely/airblast

Last synced: 1 day ago
JSON representation

Robust cloud function job queuing and processing

• Host: GitHub
• URL: https://github.com/raisely/airblast
• Owner: raisely
• Created: 2018-11-03T04:54:59.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2022-12-08T04:30:31.000Z (almost 2 years ago)
• Last Synced: 2024-04-15T20:24:15.052Z (7 months ago)
• Language: JavaScript
• Size: 501 KB
• Stars: 2
• Watchers: 4
• Forks: 0
• Open Issues: 7
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
Airblast provides a simple framework to get cloud function jobs with retries up and

running quickly so you can focus on building your jobs with minimal effort on

interacting with cloud infrastructure.

Airblast handles dealing with pubub, express and datastore and you just handle

processing the data.

The framework allows for building controllers that send jobs to one another for processing

so you can logically breakup your background processing by logical concerns.

See the [samples](/samples) for examples of setting up simple controllers

Example of job chains

```javascript

class JobController extends AirblastController {

  // Will return 400 if someone tries to post an invalid payload

  validate({ data }) {

    if (!data.about) {

      throw new this.AppError(400, 'validation', 'Data must contain about attribute');

    }

  }

  // Send the job to the email and airtable controller for processing

  async process({ key }) {

    return Promise.all([

      // Pass the key of this record to the other controllers

      this.controllers.email.enqueue({ key }),

      this.controllers.airtable.enqueue({ key }),      

    ])

  }

}

class EmailController extends AirblastController {

  async process({ data }) {

    // Load the record that was passed in

    const payload = await this.load(data.key);

    sendEmail({

      to: '[email protected]',

      subject: 'Job received',

      body: `The job is about ${data.about}`

    });

  }

}

class AirtableController extends AirblastController {

  async process({ data }) {

    const payload = await this.load(data.key);

    airtables.insert(payload);

  }

}

```

That's it, once you've written your process function,

you've written cloud functions that are ready to deploy!

See the [sample](/sample) directory for scripts

to deploy them.

## Setting up for deployment

1. Install gcloud command line

```sh

gcloud auth login

gcloud config set project 

# List available zones to deploy functions

gcloud functions regions list

gcloud config set functions/region 

# If you are also deploying the retry cron server, you will

# probably want to set the default app engine zone to the same

gcloud compute project-info add-metadata --metadata google-compute-default-region=

# Re-run gcloud init to make use of your new defaults

gcloud init

```

## Testing

Most of the time you can test your controller simply by running

`controller.process(body)`

but if you need to test the webhook endpoint with authorization and 

validation, there is a helper to help you:

```javascript

const runController = require('airblast/test/runController');

await runController(new MyController(), {

  body: { my: 'value' },

});

```

Options and default values that you can pass to 

```

{

  // The controller function to execute (defaults to the function that receives and routes

  // the http requests, usuall you don't need to change this)

  function: 'http',

  // Mock the enqueue (avoids the need for datastore & pubsub emulators)

  mockEnqueue: true,

  // Throw an error if the response status is not 200

  throwOnError: true,

}

```