Move Dataform Setup and Tables to Another Project

Google Cloud Data Analytics
,
1

Hi All,

Dataform is finally picking up steam at my company which is a good thing, however I’ve come to the realization that I made an error when starting. We had one project at the time and I stumbled through learning Dataform and set up my transformed tables along with our source data. Now seems to be the right time to move all the transformed tables to it’s own project.

I’ve been trying to analyze the best way to do this and my only thoughts are this (unless I don’t know)

  • Create new project
  • Setup Dataform in new project
  • Recreate all the files/folders/schedules/etc in new project. Essentially lift and shift code
  • Move incremental tables to new project to keep the historical data
  • Delete the tables from the old project as they’re no longer needed

It seems pretty easy as far as steps just tedious. The question is, has anyone run into this before and know of a better way, am I missing anything, etc?

Thanks

2

I’m wondering if there might be mileage in using an “external Git repository”. The way I understand Dataform is that when you create a Dataform repo, you can map that to an external Git repo. If you now wish to move your repo to a different GCP project, I am wondering if we couldn’t create a new Dataform repo in the new project and map THAT new repo to the original Git repo. I’d test this with sandbox projects before tinkering with your existing repos. We don’t want to accidently lose anything.