Part 2: CI/CD for Application Integration

Google Cloud Integration Services
1

This two-part article guides developers on how to develop, test, publish, and promote code in Application Integration.

Deployments

In the following section, we will learn how to deploy/promote integrations with tools like Cloud Deploy and GitHub Actions

Integration with Cloud Deploy

In this section we will use Cloud Deploy to deploy the integration. Once you have your integration ready in your dev project. You can run the scaffold command with the “–cloud-deploy” argument which will create the Cloud Deploy configuration files as well.

git checkout feature/cicd

token=$(gcloud auth print-access-token)
project=
region=

integrationcli prefs set -p $project -r $region -t $token

integrationcli integrations scaffold -n sample --latest -f . -e dev <strong>--cloud-deploy
</strong>

Add/update the following fragments (items marked bold) to the clouddeploy.yaml file

apiVersion: deploy.cloud.google.com/v1
kind: DeliveryPipeline
metadata:
  name: appint-sample-pipeline
serialPipeline:
  stages:
  - targetId: dev
  **- targetId: qa**
---
apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
  name: dev
customTarget:
  customTargetType: appint-sample-target
deployParameters:
  APP_INTEGRATION_PROJECT_ID: "**<DEV-project-id>**"
  APP_INTEGRATION_REGION: "**<DEV-region>**"
---
<strong>apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
  name: qa
customTarget:
  customTargetType: appint-sample-target
deployParameters:
  APP_INTEGRATION_PROJECT_ID: "<QA-project-id>"
  APP_INTEGRATION_REGION: "<QA-region>"</strong>
---
apiVersion: deploy.cloud.google.com/v1
kind: CustomTargetType
metadata:
  name: appint-sample-target
customActions:
  renderAction: render-sample-integration
  deployAction: deploy-sample-integration

Update the Dev & QA Project ID and Region information in the yaml

Deployments to Dev using Cloud Deploy

The artifacts in the dev folder can now be deployed to the DEV environment (GCP project). Make sure you have enabled Cloud Deploy in your project and have provided the necessary roles to the Default compute service account.

Create/Update the Deployment Pipeline

LOCATION=<set DEV region here>
PROJECT_ID=<set DEV project here>
gcloud deploy apply --file=clouddeploy.yaml --region=${LOCATION} --project=${PROJECT_ID}


# release name can be any string. We recommend the integration name
RELEASE_NAME=sample
gcloud deploy releases create ${RELEASE_NAME}-$(date +'%Y%m%d%H%M%S') --region=${LOCATION} --delivery-pipeline=appint-sample-pipeline --to-target=dev --project=${PROJECT_ID}

Promote Release to QA

To promote using the same pipeline, you will need to add the service account used by the pipeline in the DEV project to your QA project and assign necessary Application Integration and Connector Admin roles

You can click the “Promote” link in the Cloud Deploy via console

unnamed (3).png
unnamed (3).png1600Ă—457 212 KB

or via command line interface:

gcloud deploy releases promote --release=${RELEASE_NAME} --delivery-pipeline=appint-sample-pipeline --project=${PROJECT_ID} --to-target=qa --region=${LOCATION}

where PROJECT_ID and REGION are pointing to the project and region where Cloud Deploy is configured

Execution Service Accounts

By default, Cloud Deploy runs using the default Compute Engine service account. The name of this service account follows this pattern: [project-number]-compute@developer.gserviceaccount.com. This service account must be given Application Integration roles to create and publish Integration Versions, Connectors, Auth Configs etc.

Custom Service Accounts may be used by adding the following (marked in bold) to a Target

apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
  name: dev
customTarget:
  customTargetType: appint-sample-target
deployParameters:
  APP_INTEGRATION_PROJECT_ID: "**<DEV-project-id>**"
  APP_INTEGRATION_REGION: "**<DEV-region>**"
<strong>executionConfigs:
- usages:
  - DEPLOY
  privatePool:
    workerPool: "projects/p123/locations/us-central1/workerPools/wp123"
    serviceAccount: "[name]@[project_name].iam.gserviceaccount.com"
- usages:
  - RENDER
  - PREDEPLOY
  - VERIFY
  - POSTDEPLOY
</strong>

Read more about changing default service accounts here.

Integration with Github Actions

In this section we will use GitHub Actions to deploy the integration. Once you have your integration ready in your dev project. You can run the scaffold command with the “–github-action” argument which will create the Cloud Deploy configuration files as well.

git checkout feature/cicd

token=$(gcloud auth print-access-token)
project=<set DEV project here>
region=<set DEV region here>

integrationcli prefs set -p $project -r $region -t $token

integrationcli integrations scaffold -n sample --latest -f . -e dev <strong>--github-action
</strong>

The file “sample.yaml” contains a Github Action like this:

name: apply-sample-action
permissions: read-all

# Controls when the workflow will run
on: push

env:
  ENVIRONMENT: ${{ vars.ENVIRONMENT }}
  PROJECT_ID: ${{ vars.PROJECT_ID }}
  REGION: ${{ vars.REGION }}
  WORKLOAD_IDENTITY_PROVIDER_NAME: ${{ vars.PROVIDER_NAME }}
  SERVICE_ACCOUNT: ${{ vars.SERVICE_ACCOUNT }}

jobs:

  integrationcli-action:

    permissions:
      contents: 'read'
      id-token: 'write'

    name: Apply integration version
    runs-on: ubuntu-latest
    timeout-minutes: 20

    steps:
      - name: Checkout Code
        uses: actions/checkout@1e31de5234b9f8995739874a8ce0492dc87873e2 #v4

      - name: Authenticate Google Cloud
        id: 'gcp-auth'
        uses: google-github-actions/auth@6fc4af4b145ae7821d527454aa9bd537d1f2dc5f #v2.1.7
        with:
          workload_identity_provider: '${{ env.WORKLOAD_IDENTITY_PROVIDER_NAME }}'
          service_account: '${{ env.SERVICE_ACCOUNT }}'
          token_format: 'access_token'

      - name: Calculate variables
        id: 'calc-vars'
        run: |
          echo "SHORT_SHA=$(git rev-parse --short $GITHUB_SHA)" >> $GITHUB_OUTPUT

      - name: Create and Publish Integration
        id: 'publish-integration'
        uses: docker://us-docker.pkg.dev/appintegration-toolkit/images/integrationcli:0.81.3 #pin to version of choice
        with:
          args: integrations apply --env=${{ env.ENVIRONMENT}} --folder=. --userlabel=${{ steps.calc-vars.outputs.SHORT_SHA }} --wait=true --run-tests=true --proj=${{ env.PROJECT_ID }} --reg=${{ env.REGION }} --token ${{ steps.gcp-auth.outputs.access_token }}

Move this file to the .github/workflows folder. This sample is triggered automatically based on a push to the repo. You can update the trigger based on your requirements.

The GitHub Action template generated uses Workload Identity Federation. You can find more information and configuring in GitHub by following the documentation here. You can use different branches to trigger the code promotion from one env to another by setting the appropriate variables.

Integration with Cloud Build

If you are interested to know more about using Cloud Build to automate the Application Integration deployments, you can refer to this post. There is a section for Cloud Build discussed in detail..

Conclusion

This concludes our two-part series on developing and deploying Application Integration code using best practices and available tools.

Special Thanks to Nandan Sridhar for his collaboration on this article and also for all his contribution to integrationcli

1 Like