Looker GitOps: Building a Secure, Automated Deployment & Rollback Pipeline

Looker Tips & Resources Technical Tips & Tricks
, , ,
1

Looker has changed how organizations model and interact with data. However, managing the release lifecycle of Looker projects across multiple instances has traditionally been a challenge. How do you deploy LookML code, keep shared folders in sync, run automated syntax and data validations, and ensure you can roll back instantly if a release breaks production?

By structuring a complete GitOps pipeline using GitHub Actions, we can bridge this gap and establish a secure, automated release process. You can find the fully functional blueprint in the looker_cicd_sample repository.

Here is how to set up an end-to-end Looker GitOps pipeline that handles PR validation, path-based review routing, automated backups, and gated production releases.

The Looker GitOps Lifecycle

A robust Looker pipeline spans two separate instances (Dev and Production) and relies on automated gates to ensure code quality:

[ Developer Branch ]
         │
         ▼
[ Open Pull Request ] ──► [ 📝 PR Checklist Template ]
         │            ──► [ 🧪 Looker CI ]
         │            ──► [ 📐 LAMS Style Linter ] 
         ▼            ──► [ 👥 Route to CODEOWNERS ]
[ Merge to master ]   
         │
         ▼ (Native Webhook)
[ Dev Prod Mode ]
         │
         ▼ (Release Please PR)
[ Tag pushed (v1.0.0) ]
         │
         ▼
[ Gated PRD Deploy ] 
         │
         ▼
[ Manual Gate Approval ] 
         │
         ▼
[ Deploy LookML & Deploy Content ]
  1. Continuous Integration (PR Validation): Every pull request undergoes automated testing (via Looker CI) and styling audits (via LAMS) before it is allowed to merge.
  2. Native Webhook Promotion: Merging a PR into master instantly deploys the code to the Dev instance’s production mode (serving as a QA sandbox) via secure, native GitHub webhooks.
  3. Gated Production Releases: Releases to the Production (PRD) instance are tag-driven (e.g., v1.0.0) and managed by GitHub Environments with required manual approvals.
  4. Cohesive Content Backups & Rollbacks: Shared folder dashboards and looks are backed up to a versioned GCS bucket upon release. If a release fails, redeploying an older tag automatically restores both the LookML code and its corresponding historical dashboard state.

1. PR Gates: Looker CI & LAMS Linter

Before code ever reaches your master branch, it must pass through automated quality checks.

Pull Request Template

To set expectations early, we use a custom .github/PULL_REQUEST_TEMPLATE.md to prompt developers with a standardized checklist when opening a PR. This forces developers to self-verify that they’ve run LookML validations, confirmed that all new/modified dimensions have descriptions, and ensured database queries perform correctly.

Looker CI

We integrate pull requests with Looker CI to validate code integrity against your live database. On every PR, Looker CI triggers:

  • LookML Validator: Catches syntax and logic errors in the LookML.
  • SQL Validator: Runs queries against the underlying database for every Explored dimension to ensure no database columns are broken.
  • Assert Validator: Executes your LookML custom data assertions.
  • Content Validator: Verifies that no dashboards or Looks are broken by the model changes.

LAMS (Look At Me Sideways)

To maintain code quality and styling consistency, we run LAMS on every commit. LAMS parses the LookML and writes warnings or errors directly as inline comments on the PR lines that violated the styling rules (such as enforcing that all dimensions must have descriptions).

Code Review Routing (CODEOWNERS)

To prevent bottlenecks, we use a CODEOWNERS file at the root of the repository. This automatically routes pull requests to specific team experts depending on which LookML files are touched (e.g., routing changes in finance models to the finance data team, and core configurations to the platform leads).

2. Gated Deployments & Automatic Rollbacks

Once a Release PR is merged, it tags the repository (e.g., v1.0.0) and starts the Deploy to PRD pipeline.

Manual Approval Gates

The deployment targets a GitHub Environment named PRD. This environment is configured with a manual approval rule, requiring a Data Lead or Administrator to explicitly review and click “Approve” in the Actions UI before the runner executes the deployment script.

Automating Content Syncing with looker-deployer

To sync user-facing Looker objects (Dashboards and Looks) along with our LookML code, we use looker-deployer—an open-source python utility built on Looker’s API SDK:

  • Exporting (ldeploy content export): The pipeline queries the Dev Looker instance, reads the content of the Shared folder, and exports the JSON blueprints of all Dashboards and Looks into a local workspace directory.
  • Importing (ldeploy content import): The pipeline uploads those JSON files to Google Cloud Storage. When importing to PRD, looker-deployer reads the JSON folder, reconstructs the folder tree hierarchy, and recreates/updates the corresponding Dashboards and Looks on the PRD instance.

Automated Rollbacks via GCS Snapshots

If a deployment causes issues in production, you can roll back instantly. The deployment pipeline checks GCS for an existing backup tag:

  • New Release: If the backup does not exist, the pipeline exports the shared folder content from Dev, uploads it to GCS, and imports it to PRD.
  • Rollback: If you trigger a manual run targeting an older tag (e.g. v0.9.0), the pipeline detects that the backup already exists on GCS. It skips the export (preserving history) and directly downloads the v0.9.0 snapshot from GCS and imports it into PRD. This ensures your dashboards and Looks revert back to the exact state they were in at that tag.

Key Configuration Steps

To configure this workflow on your repository:

GCP Setup

export PROJECT_ID="your-gcp-project-id"
export BUCKET_NAME="looker-migrations-snapshots-gitops"
export SA_NAME="github-deployer"
export SA_EMAIL="${SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"

# Set your active project
gcloud config set project $PROJECT_ID
  1. Create GCS Bucket: Create a versioned bucket to store content backups
# Create the bucket (defaults to US multi-region; add --location=REGION if needed)
gcloud storage buckets create gs://$BUCKET_NAME

# Enable versioning for backups
gcloud storage buckets update gs://$BUCKET_NAME --versioning
  1. Create Service Account: Create a Service Account
gcloud iam service-accounts create $SA_NAME \
    --display-name="GitHub Deployer for Looker Migrations"
  1. Grant Permissions: Assign the service account Storage Object Admin permissions specifically on your backup GCS bucket.
gcloud storage buckets add-iam-policy-binding gs://$BUCKET_NAME \
    --member="serviceAccount:$SA_EMAIL" \
    --role="roles/storage.objectAdmin"
  1. Generate Keys: Create and download a JSON credential key for the service account, and save it in your GitHub Secrets as GCP_SA_KEY.
gcloud iam service-accounts keys create GCP_SA_KEY.json \
    --iam-account=$SA_EMAIL

Looker Setup

  1. Configure Webhook on Dev: Set a Webhook Deploy Secret in Looker Dev Project Settings, and configure a webhook in GitHub pointing to https://<dev-url>/webhooks/projects/<project_id>/deploy signed with that secret.
  2. Enable Advanced Deploy Mode on PRD: In Looker PRD Project Settings, toggle on Enable Advanced Deploy Mode.
  3. Configure Environment Gates: In GitHub, create a PRD Environment and add required reviewers to gate deployment runs.
  4. Set Up Secrets: Add your API credentials as secrets (DEV_LOOKERSDK_BASE_URL, PRD_LOOKERSDK_BASE_URL, etc.). Important: Ensure your base URLs explicitly include the port suffix (e.g., https://domain.looker.app:443) to prevent dependency parser crashes.
deploy-prd.yml 
name: Deploy to PRD

on:
  push:
    tags:
      - 'v*'
  workflow_dispatch:

jobs:
  deploy:
    name: Deploy to PRD
    runs-on: ubuntu-latest
    environment: PRD
    
    env:
      GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
      GCP_WORKLOAD_IDENTITY_PROVIDER: ${{ vars.GCP_WORKLOAD_IDENTITY_PROVIDER }}
      GCP_SERVICE_ACCOUNT: ${{ vars.GCP_SERVICE_ACCOUNT }}

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Ruby
        uses: actions/setup-ruby@v1
        with:
          ruby-version: '2.7'

      - name: Install Gazer Gem
        run: gem install gazer

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.10'

      - name: Install Looker-Deployer
        run: pip install looker-deployer

      - name: Check Auth Configurations
        run: |
          echo "Checking GCP Auth variables..."
          if [ -n "$GCP_SA_KEY" ]; then
            echo "-> Service Account Key authentication is configured."
          else
            echo "-> Service Account Key is NOT configured."
          fi

      - name: Authenticate to GCP (Service Account Key)
        if: env.GCP_SA_KEY != ''
        uses: google-github-actions/auth@v2
        with:
          credentials_json: ${{ env.GCP_SA_KEY }}

      - name: Authenticate to GCP (Workload Identity Federation)
        if: env.GCP_SA_KEY == '' && env.GCP_WORKLOAD_IDENTITY_PROVIDER != '' && env.GCP_SERVICE_ACCOUNT != ''
        uses: google-github-actions/auth@v2
        with:
          workload_identity_provider: ${{ env.GCP_WORKLOAD_IDENTITY_PROVIDER }}
          service_account: ${{ env.GCP_SERVICE_ACCOUNT }}

      - name: Set up Cloud SDK
        uses: google-github-actions/setup-gcloud@v2

      - name: Configure Credentials and Connection Files
        env:
          DEV_LOOKERSDK_BASE_URL: ${{ secrets.DEV_LOOKERSDK_BASE_URL }}
          PRD_LOOKERSDK_BASE_URL: ${{ secrets.PRD_LOOKERSDK_BASE_URL }}
          DEV_CLIENT_ID: ${{ secrets.DEV_LOOKERSDK_CLIENT_ID }}
          DEV_CLIENT_SECRET: ${{ secrets.DEV_LOOKERSDK_CLIENT_SECRET }}
          PRD_CLIENT_ID: ${{ secrets.PRD_LOOKERSDK_CLIENT_ID }}
          PRD_CLIENT_SECRET: ${{ secrets.PRD_LOOKERSDK_CLIENT_SECRET }}
        run: |
          DEV_BASE_URL="${DEV_LOOKERSDK_BASE_URL%/}"
          PRD_BASE_URL="${PRD_LOOKERSDK_BASE_URL%/}"
          
          # Safeguard workaround: append port 443 if no port exists in secret URL
          if [[ ! "${DEV_BASE_URL}" =~ :[0-9]+$ ]]; then
            DEV_BASE_URL="${DEV_BASE_URL}:443"
          fi
          if [[ ! "${PRD_BASE_URL}" =~ :[0-9]+$ ]]; then
            PRD_BASE_URL="${PRD_BASE_URL}:443"
          fi
          
          cat <<EOF > looker.ini
          [dev]
          base_url=$DEV_BASE_URL
          client_id=$DEV_CLIENT_ID
          client_secret=$DEV_CLIENT_SECRET
          
          [prd]
          base_url=$PRD_BASE_URL
          client_id=$PRD_CLIENT_ID
          client_secret=$PRD_CLIENT_SECRET
          EOF

      - name: Check GCS Backup Existence
        id: check_backup
        env:
          TAG: ${{ github.ref_name }}
        run: |
          GCS_PATH="gs://looker-migrations-snapshots-gitops/looker_backups/${TAG}/"
          if gcloud storage ls "${GCS_PATH}**" > /dev/null 2>&1; then
            echo "backup_exists=true" >> $GITHUB_OUTPUT
            echo "Backup snapshot for tag ${TAG} already exists. Skipping export phase."
          else
            echo "backup_exists=false" >> $GITHUB_OUTPUT
            echo "No backup found for tag ${TAG}. New release backup will be created."
          fi

      - name: Backup Dev Looker Content to GCS
        if: steps.check_backup.outputs.backup_exists == 'false'
        env:
          TAG: ${{ github.ref_name }}
        run: |
          GCS_PATH="gs://looker-migrations-snapshots-gitops/looker_backups/${TAG}/"
          ldeploy content export --ini ./looker.ini --env dev --folders 1 --local-target ./looker_content_export
          gcloud storage cp --recursive ./looker_content_export/* "$GCS_PATH"

      - name: Authenticate and Deploy LookML to PRD
        env:
          TAG: ${{ github.ref_name }}
          PRD_LOOKERSDK_BASE_URL: ${{ secrets.PRD_LOOKERSDK_BASE_URL }}
          PRD_CLIENT_ID: ${{ secrets.PRD_LOOKERSDK_CLIENT_ID }}
          PRD_CLIENT_SECRET: ${{ secrets.PRD_LOOKERSDK_CLIENT_SECRET }}
        run: |
          # Authenticate to PRD instance API and trigger ADM Ref deployment
          auth_resp=$(curl -s -X POST -F "client_id=$PRD_CLIENT_ID" -F "client_secret=$PRD_CLIENT_SECRET" "$PRD_LOOKERSDK_BASE_URL/api/4.0/login")
          token=$(echo "$auth_resp" | python3 -c "import sys, json; print(json.load(sys.stdin)['access_token'])")
          
          curl -s -X PATCH -H "Authorization: Bearer $token" -H "Content-Type: application/json" -d '{"workspace_id":"dev"}' "$PRD_LOOKERSDK_BASE_URL/api/4.0/session"
          curl -s -X POST -H "Authorization: Bearer $token" "$PRD_LOOKERSDK_BASE_URL/api/4.0/projects/looker_cicd_sample/deploy_ref_to_production?ref=$TAG"

      - name: Import Migrated Content to PRD Looker
        env:
          TAG: ${{ github.ref_name }}
        run: |
          GCS_PATH="gs://looker-migrations-snapshots-gitops/looker_backups/${TAG}/"
          mkdir -p ./looker_content_import
          gcloud storage cp --recursive "${GCS_PATH}*" ./looker_content_import/
          ldeploy content import --ini ./looker.ini --env prd --folders ./looker_content_import/Shared --recursive
lams.yml 
name: Looker LAMS Linting

on:
  pull_request:
    branches:
      - master

jobs:
  lams:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      contents: read
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install LAMS
        run: npm install -g @looker/look-at-me-sideways@3

      - name: Run LAMS
        run: |
          lams --reporting=no > lams_output.txt 2>&1
          cat lams_output.txt

      - name: Post LAMS Report as PR Comment
        if: failure()
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
        run: |
          if [ -f lams_output.txt ]; then
            REPORT=$(cat lams_output.txt)
            
            # Construct the Markdown body
            COMMENT_BODY=$(cat <<EOF
          ### ❌ Looker LAMS Linting Errors
          
          LookML style validation failed. Review the style violations below:
          
          \`\`\`text
          $REPORT
          \`\`\`
          
          *Please update your branch with the correct style rules to pass validation checks.*
          EOF
            )
            
            # Post comment to PR
            gh pr comment "$PR_NUMBER" --body "$COMMENT_BODY"
          else
            echo "No LAMS output file found."
          fi
release.yml 
name: release-please

on:
  push:
    branches:
      - master

permissions:
  contents: write
  pull-requests: write

jobs:
  release-please:
    runs-on: ubuntu-latest
    steps:
      - uses: google-github-actions/release-please-action@v4
        with:
          release-type: simple
3 Likes
2

Thanks. I notice you are using LAMS as part of the workflow. When will a style validator be available as part of the standard Looker CI suite? This was mentioned as a future phase when the Spectacles integration was first announced (March 2025), but I’ve not heard anything about it since.

1 Like
3

You can watch the latest Looker Roadmap here.

1 Like
4

Thanks for the link. I am up to date on all the latest roadmap sessions (I have seen that one - the focus was on agent observability, evaluations and 4 new visualisations). I also attend the BICC. I have not heard any mention of the style validator in either of these communication channels. Has it been dropped as a product feature? I have delayed using LAMS since I was aware that a native solution was on the horizon, but I will rethink this approach if you confirm that it is no longer on the roadmap.

2 Likes
5

The product team is fully committed to making Looker CI a complete end-to-end tool for developers, and they are actively working on a native linting solution targeted for Q3 of this year. In the meantime, I would highly encourage setting up a light configuration with LAMS. This will get your teams thinking about and working on what linting rules are most valuable for your organization before the native feature drops.

3 Likes
6

Brilliant! Thanks for the update.