https://github.com/rsoesemann/sobject-work-queue

Custom Apex queue to process arbitrary work on existing SObject in a guaranteed order.
https://github.com/rsoesemann/sobject-work-queue

asynchronous-programming framework proof-of-concept queues salesforce-apex

Last synced: 24 days ago
JSON representation

Custom Apex queue to process arbitrary work on existing SObject in a guaranteed order.

• Host: GitHub
• URL: https://github.com/rsoesemann/sobject-work-queue
• Owner: rsoesemann
• License: mit
• Created: 2013-08-20T09:07:54.000Z (about 11 years ago)
• Default Branch: master
• Last Pushed: 2023-10-18T14:54:36.000Z (about 1 year ago)
• Last Synced: 2023-10-18T16:03:04.887Z (about 1 year ago)
• Topics: asynchronous-programming, framework, proof-of-concept, queues, salesforce-apex
• Language: Apex
• Size: 99.6 KB
• Stars: 53
• Watchers: 12
• Forks: 16
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# SObject Work Queue [![Codacy Badge](https://api.codacy.com/project/badge/Grade/05c1be483aa944de9e5c9313c0fa6f12)](https://www.codacy.com/app/rsoesemann/sobject-work-queue?utm_source=github.com&utm_medium=referral&utm_content=rsoesemann/sobject-work-queue&utm_campaign=Badge_Grade)



  



Apex Batch is something like the last resort for Apex developers to circumvent limitations of the Salesforce Platform when working with "Large" data volumes.  When using Batch as the asynch backbone of a bigger system you soon find obvious drawbacks:

- Jobs are put in a queue, but when that queue is full (Max. 5 concurrent batches), the job fails instead of being scheduled for later processing.

- Jobs of different Batches might work on the same data and produce conflicts. There is no locking mechanism or guaranteed order.

- Poor support to handle party failed batch runs. Its really hard to find out where and why a single job failed and to restore data consitency.

## Design Goals: ##

We tried to build a custom queue that overcomes those drawbacks.

- Must prevent Max 5 batch in parallel limit - We should never run into this limit with work that is processed over the queue.	 	 	 

- The queue needs to be so generic that "work" on any type of database object needs to be enqueued.	 	 	 

- Any type of modification of database objects need to be possible. This should be transparently handled by the queue.	 	 	 

- Provides better error diagnostics like Batch. Knows last successful Id, full stacktrace and sends email to developers.	 

- Secures data integrity like Batch or better. Failures should not leave data in inconsistent state or user of the infrastructure should be able to handle them.	 	 	 

- Optimistic locking : Instead of locking all many records we do not to process work on Ids that have other work already scheduled.	 	 	 

- Work that can be run synchronously, should not be queued and processed asynch.

![SObject Work Queue : How work is definied, enqueued and processed](https://raw.github.com/rsoesemann/sobject-work-queue/master/resources/SObjectWorkQueueInfrastructure.png)

## Architectural Components ##

- `interface SObjectProcessor` : Command interface to define a type of work that can be performed on a Set of Ids

        public interface SObjectProcessor {

          String getFullClassName();

	        void setRecordIds(List recordIds);

	        void setParameterObject(Object parameterObject);

	        Boolean canRunSynchronously();

	        void process(SObjectWork.LastSuccessfulId lastSuccesfulId);

        }

- `class SObjectWork` : Defines concrete work to do on which concrete Ids

        List ids = ...;

        Map paramsMap = new Map();

        paramsMap.put('name', value);

        SObjectProcessor processor = new SObjectWorkTestHelper.ExampleSObjectProcessor();

        

        SObjectWork work = new SObjectWork.Builder(oppsToProcess, processor).withParams(paramsMap).build();

- `class SObjectWorkQueue` : singleton class to add work to the queue 

        SObjectWorkQueue.addWork(work);

- `SObject SObjectWork__c` : a Custom SObject that persits SObjectWork instances in the database for later processing

- `class SObjectWorkSerializer` : converts an instance of SObjectWork into 1-n records of SObjectWork__c

- `class SObjectWorkDeserializer`: converts SObjectWork__c records back to their memory representation SObjectWork

## Usage ##

For usage examples please see the test classes especially `SObjectWorkQueue_Test` and `SObjectWorkTestHelper`.