Can you make built-in assertions a dependency instead of a dependent?

Google Cloud Data Analytics
1

When Dataform was first released as General Availability, built-in assertions (uniqueKey, nonNull,…) were actually run in a workflow as dependencies to whatever table they were attached to and run before the table was built.

At some point in the recent past this was changed and built-in assertions are now run as dependents, after the table is built.

This is all fine but I’m wondering if it’s possible to use the built-in assertions as dependencies that run prior to building the table they are attached to?
I understand that I can make a manual assertion that mimics the built-in assertions functionality. I’m just curious if there’s a simpler way.

Standard config block with assertions as dependents.

config { type: "table",
        name: "my_table",
        schema: "my_schema",
        description: "",
        bigquery: {
          labels: {
            scheduler: "dataform"
          }
         },        
        dependencies: [
                      "total_vol_drop_assertion",
                      "units_drop_assertion",
                      ],
        assertions: {
          uniqueKey: ["id"],
          nonNull:["id"]
                    }
        }

To something like this…

config { type: "table",
        name: "my_table",
        schema: "my_schema",
        description: "",
        bigquery: {
          labels: {
            scheduler: "dataform"
          }
         },        
        dependencies: [uniqueKey: ["id"],
                      nonNull:["id"],
                      "total_vol_drop",
                      "units_drop",
                      ],

                    }
        }

The documentation has the below example but it’s not descriptive enough for me to work with.

The following code sample shows table A assertions added as dependencies to table B:
config {
  type: "table",
  dependencies: [ "A_uniqueKey",  "A_nonNull"]
}
1 Like
2

I am reading this article. What I understand from this is that:

  1. Assertions can be defined to run AFTER a table is created. So, if I have a file called “A.sqlx”, then the assertions included in that definition are run after the SQL execution to assert that the newly created data is valid. That seems to make solid sense to me.

  2. If I have a definition called “B.sqlx” that refers to another table (eg. table A) then we say that table A is a dependency of table B. The documentation says that the SQL for creating table B will execute even if the assertions declared in “A.sqlx” resulted in failure. However, we can declare that we want to explicitly check the assertions declared in “A.sqlx” when we are processing “B.sqlx” by coding dependencies to the assertions in the “B.sqlx”. The apparent way that we code this is through:

dependencies: [_]

For example:

dependencies: [“A_uniqueKey”]

seems to state that we want to find the SQLX for table “A” and within that find the assertion called “uniqueKey” and run that assertion (for table A) … and we will then run the SQL for table B ONLY if the assertion passes.