There always comes a point where items created in Sugar Integrate - the Common Resources, transformations, adapters, and/or procedures - need to be changed. Technically, you could just make the changes through our UI, but there are a number of risks you take with doing that. There are better ways, and we've collected a number of best practices for deploying changes in Sugar Integrate and we present them here.
There are a number of considerations when thinking about how to deploy changes to Sugar Integrate.
- How will you minimize the potential impact on current users?
- How will you make sure that any changes won’t have a negative impact?
- What will you do if the changes do turn out to cause problems?
Following are a list of our best practices for deployment.
Rather than relying solely on Sugar Integrate to save the items you’ve worked so hard to build, we recommend backing them up elsewhere. We recommend using our command line tool, ce-utils (also affectionately known as The Doctor). You can find the tool and documentation on how to use it to save your Sugar Integrate assets at https://github.com/cloud-elements/the-doctor or you can install it using npm (npm install -g ce-utils). You will need to have already installed Node.js (https://nodejs.org) on your computer.
Merely backing up assets that make up your integration (adapters, Common Resources/transformations, and procedures) to your local computer isn’t particularly useful. We recommend also using a Source Control Management system, such as GitHub (https://github.com/) to store the assets. SCMs provide reliable storage, the ability to revert back to previous changes in case of a bug, tools to determine if the changes made are what you expect them to be, and more.
Whenever you make changes, there’s a chance that your changes have unanticipated side effects that could break your integration. We recommend automated testing whenever deploying changes. The two major types of testing we recommend are
- API testing - test that your APIs produce the results you expect. A good tool for building these is chakram (http://dareid.github.io/chakram) which we use for our suite of tests for the Sugar Integrate platform and adapters, called churros (https://github.com/cloud-elements/churros)
- UI testing - test that your user interface works as expected. There are a number of tools that can be used for web-based UIs, such as Nightwatch.js (http://nightwatchjs.org), CasperJS (http://casperjs.org), and Geb (http://www.gebish.org).
We also recommend both thorough testing in your QA environment (which should utilize our staging environment) as well as simple smoke testing in production.
If a deployment to your production environment fails the smoke test, it would be best to restore your Sugar Integrate assets to their previous state. Using your SCM, it is a simple matter to retrieve the prior version of your assets and deploy them using our platform APIs or The Doctor.
There are two general approaches that we recommend when deploying Sugar Integrate assets with varying levels of downtime and applicability to the various types of Sugar Integrate assets (adapters, Common Resources/transformations, and procedures). It is important to understand the differences between the two approaches so you can decide which one better fits your needs.
Minimal Downtime Deployment (AKA disable & retry)
This approach is applicable to all of the deployable Sugar Integrate assets. It is simple to implement and is generally the best approach unless you have need for the lowest downtime possible.
The steps to take are:
- Disable all adapters instances (DELETE /instances/enabled)
- Wait for any potential in-process transactions to complete
- Deploy any assets
- Enable all adapters instances (PUT /instances/enabled)
The amount of time to wait should be made long enough to cover the kinds of transactions your application makes. This is highly dependent on your application, though for most users, 60 seconds should be sufficient for all in process transactions to complete.
This approach requires that your application retries any requests that fail due to encountering a disabled adapter. We recommend that you use an exponential backoff for retry delay. Delays of powers of 2 in seconds (i.e., 1 second, 2 seconds, 4 seconds, 8 seconds) will result in a total delay of over 60 seconds after 6 retries.
If your solution includes procedure, you can leverage the existing support that our adapter request step has for retries to provide this.
Zero Downtime Deployment (AKA versioning)
If you need even less downtime, then the following approach may work for you. It is only applicable to Common resources /objects and Transformations. The changes you’re deploying must be limited to these assets (and not adapters or procedures) in order for this to be an option.
The steps are:
- Deploy the assets using Common Resource names that include a version
- Deploy your code that references the new Common Resource names
- When you’re satisfied that the previous version is no longer needed, delete the transformations and Common Resources
The Doctor provides support for versioning. It has an option -v or --version that will add a suffix to any Common Resources to be deployed. We recommend using the industry standard semantic versioning (semver) standard (https://semver.org) when formatting your versions.