How to Use Bulk Within a Procedure

Doing bulk within a procedure can be a little difficult. Bulk by nature is an asynchronous job, while procedures are a synchronous procedure. One way to get them to work together is via webhooks.

There are two ways to know when a bulk job is completed. You can poll the GET /bulk/{id}/status endpoint repeatedly until status is COMPLETED or ABORTED or you can set the header Elements-Async-Callback-Url and receive a webhook when the job is done. 

When using webhooks, the key is to have two procedures. One to start the bulk job and set the Elements-Async-Callback-Url to manually trigger a second procedure that downloads the results and sends them somewhere else. 

Here are two example procedures when using the webhook method. They will bulk download from one instance, and bulk upload to another. 

IMPORTANT NOTE: for these procedures to work, you must use a common resource! You must be able to call GET /MyResource, copy a payload that gets returned and call POST /myResource to the other endpoint. I will include my example transformations. 

To Configure: 

  1. Import both procedures Bulk Step 1.json and Bulk Step 2.json.
  2. Create the common resource by calling POST /organizations/objects/MyContact/definitions with this payload: ObjectDefinition.json
  3. Create transformations for Salesforce by calling POST /organizations/elements/sfdc/transformations/MyContactwith: MyContactSFDC.json
  4. Create transformations for DynamicsCRM by calling POST /organizations/elements/dynamicscrm/transformations/MyContact with: dynamicscrmMyContact.json
  5. Create a procedure instance for Bulk Step 2 and capture the ID of the created procedure instance.

    You will need to give the Bulk Step 1 procedure the instance ID for the second procedure. This will let the webhook trigger the correct procedure instance

    Image_2018-01-24_at_10.31.26_PM.png

  6. Create a procedure instance for Bulk Step 1 using the procedure instance id from step 5.