Working with multiple repositories in GCP Dataform

Google Cloud Data Analytics
1

Hello there,

We originally had one Dataform repository but due to the large number of Dataform actions and compilation errors, we decided to split it to two repositories (repo_1 and repo_2). The split happened based on the third suggestion on this official document:

→ “1 central repository and 1 repository per domain that uses the contents of the central repository as data sources.”

However, the split has increased the development effort significantly, mainly due to the fact that if a change occurs in repo_1, developer has to

  1. Compile repo 1
  2. Trigger execution of the new object so it is created
  3. Declare the new source in repo_2

While I am aware there is an open issue on Google issue tracker for this, I was wondering if anyone else is having this issue, or have tried to address it somehow? For example, is there any way that repo_2 would always be aware about all its sources in repo_1?

Thanks for your help in advance!

Azade

2 Likes
2

Hello,

Thank you for contacting Google Cloud Community !

As mentioned in the Public issue tracker ,we do not have any ETA for this issue. Please monitor the Issue tracker in order to get updates.

Regards,

Jai Ade