How can I effectively utilize Python code within BigQuery Dataform SQLX (like Javascript)

Google Cloud Data Analytics
1

How can I effectively utilize Python code within BigQuery Dataform SQLX to streamline my data extraction, consolidation, cleansing, and transformation processes? I’m particularly interested in leveraging Python’s capabilities to optimize performance and achieve complex transformations without compromising SQL efficiency.

2

Python cannot be used within Dataform’s SQLX scripts, as Dataform is designed for SQL-based transformations and only supports JavaScript for custom logic in SQLX.

3

Hi @MDSITHIK ,

Welcome to Google Cloud Community!

As of the moment, directly executing Python code within Dataform SQLX files is not yet supported as Dataform primarily uses SQL for transformations.

I suggest submitting this as a feature request. While I can’t provide specific details or timelines at this moment, please keep an eye on the release notes for any updates or new features related to Dataform.

I hope the above information is helpful.