How to test HDE Whistle Match rules locally

Google Cloud Community Articles
, ,
1

Overview

Healthcare Data Engine(HDE) is a GCP offering built on top of Cloud Healthcare API to help healthcare stakeholders transition to FHIR. A typical HDE workflow is a combination of Data Mapping, i.e., data conversion from multiple different formats into FHIR and Data Reconciliation, i.e., merging all records related to a Patient to form a single longitudinal view.

Data Reconciliation is done using Whistle Data Mapping Language. It is an Open Source mapping language and comes with built in plugins that can be used to first Match an incoming resource to an existing resource and if a Match is found, then merge it to form the Longitudinal Patient Record(LPR).

In this blog we will take a quick look at how to use the Open Source Git repo to quickly test out a Whistle Match rule locally.

Before you start…

This article assumes that the user is familiar with Whistle Data Mapping Language concepts and can write Whistle Match Rules based on their requirements.

To get detailed understanding on Whistle Data Mapping Language used for writing Mapping and Reconciliation code, please refer to the official HDE or Whistle docs.

We will be using already existing/converted FHIR resources to try out Match rules.

Leveraging the Whistle Open Source GIT repository

In this tutorial, we can use any common IDE to clone the Whistle Open Source GIT repository and leverage the instructions mentioned in the GIT repo to set up necessary libraries. Once the setup is done, we will write a test Whistle Match rule and see how we can validate it for its correctness. Let’s begin!

1. Setup Whistle Engine on your local machine

Follow the README file in git repo - Healthcare Data Harmonization to set up Whistle Engine on your local machines. This includes downloading and installing below softwares/libraries and cloning the git repo to your local machine.

  • Git
  • JDK 11.x
  • Gradle 7.x

2. Familiarize yourself with the folder setup in the git repo

  • All the pre-written Out of the Box Whistle Mapping, Match and Merge rules are present in the mappings folder. These can be used a reference and users can customize the code based on their requirement
  • quickrun folder allows us to run/test our Match/Merge rules locally. In this blog, we will be focusing on the Match Rules.
  • mappings/reconciliation/unit_tests/matching folder has the sample input Match Rules in .input.json and expected property values extracted from these Match Rules in .output.json. When the Property values match, the 2 resources are considered as a Match

image1.png
image1.png551×737 130 KB

In our case, we will look for the Property values extracted from the input resources using our Match rule. We will be using the [extractPropertyValues()](http://extractPropertyValues) function in the recon plugin for this. If the property values match, our resources will match as well.

3. Test the Match Rule for Patient resources

Next, we’ll test a sample Match Rule for Patient resources. Start by creating a separate folder wstl_match where we will store our input Patient resources in a file named patient.input.json. Output will be written to patient.output.json and the rules to test will be written in match-rule-test.wstl

image4.png
image4.png548×223 55 KB

4. Update the sample Patient record

We’ll use the existing sample Patient record present in the Whistle git repo. Feel free to replace this with your data (against which you wish to test the Match rule). Notice resources is an array here, which means you can add multiple patient resources and check for the property values that are extracted for it.

image3.png
image3.png874×945 282 KB

5. Test the Patient Match rule

We will use the below test Patient Match rule, wherein we are trying to match only on the combination of family and given field under name for a Patient resource

package testWhistle

import "class://com.google.cloud.verticals.foundations.dataharmonization.plugins.reconciliation.ReconciliationPlugin";

var config: getMatchingConfig("Patient");
var input: loadJson(joinPath("/usr/local/google/home/ashwinshetty/wstl_match", "patient.input.json"))
var actualOutput: recon::extractPropertyValues(config, input.resources[])

def getAllMatchingConfigs() {
  Patient: PatientConfig()
}

def getMatchingConfig(resourceType) get(getAllMatchingConfigs(), resourceType)

def PatientConfig() {
    recon::allOf(
      recon::arrayAnyOf("name", recon::anyOf(
          recon::allOf(
            recon::primitive("family"),
            recon::arrayAnyOf("given", recon::primitive(""))
          )
        )
      ) 
    )
}

actualOutput

Code explanation:

  • We are using a modified version of whistle Match rule testing code present in healthcare-data-harmonization/mappings/reconciliation/unit_tests/test_matching.wstl
  • We start with importing the recon plugin, which gives us the extractPropertyValues() function that we use to extract property values.
  • Match rules to test are defined in the PatientConfig() function.

6. Run the Whistle code against the input resources

Use the below gradle command to run the whistle code against the input resources.

gradle run -q --args="-m wstl_match/match-rule-test.wstl" > wstl_match/patient.output.json

Step 7 - Check the patient.output.json file for the property values extracted. In our case, since the resources present in the input file belonged to the same Patient, we see both the extracted property values to be the same. This also means, these 2 resources would be matched and eventually merged in HDE.

image2.png
image2.png537×330 85.5 KB

Feel free to experiment with different Match rules and resources to see how the extracted property values change.

Conclusion

By following the steps outlined above, we explored a way to try out and validate various combinations of Match Rules against our Input FHIR resources. All this was done locally, without the need for any GCP product/services. Once the testing is complete, these finalized Match rules can then be used in HDE deployments.

Reference links: