Terraform big query data transfer Error: Missing parameter: connector.endpoint.host. Config name

Google Cloud Data Analytics
,
1

When trying to create Data Transfer from MySQL to BigQuery using Terraform, I receive this error:

Error:

module.bq.google_bigquery_data_transfer_config.mysql_to_bq_transfer: Creating...
╷
│ Error: Error creating Config: googleapi: Error 400: Parameters: fields {
│ key: "data_path"
│ value {
│ string_value: "TABLE_NAME"
│ }
│ }
│ fields {
│ key: "database_name"
│ value {
│ string_value: "DB_NAME"
│ }
│ }
│ fields {
│ key: "destination_table_name_template"
│ value {
│ string_value: "TABLE_NAME"
│ }
│ }
│ fields {
│ key: "password"
│ value {
│ string_value: "DB_PASS"
│ }
│ }
│ fields {
│ key: "query"
│ value {
│ string_value: "SELECT * FROM \'TABLE_NAME\';"
│ }
│ }
│ fields {
│ key: "username"
│ value {
│ string_value: "DB_USER"
│ }
│ }
│ fields {
│ key: "write_disposition"
│ value {
│ string_value: "APPEND"
│ }
│ }
│ Missing parameter: connector.endpoint.host. Config name:
│ 
│ with module.bq.google_bigquery_data_transfer_config.mysql_to_bq_transfer,
│ on ../../modules/bq/main.tf line 26, in resource "google_bigquery_data_transfer_config" "mysql_to_bq_transfer":
│ 26: resource "google_bigquery_data_transfer_config" "mysql_to_bq_transfer" {

Terrafrom:

resource "google_bigquery_data_transfer_config" "mysql_to_bq_transfer" {
  display_name           = "TABLE_NAME"
  location              = "EU"
  data_source_id        = "mysql"
  destination_dataset_id = google_bigquery_dataset.bq_dataset.dataset_id
  schedule = "every 24 hours"
  service_account_name = google_bigquery_connection.bq_mysql_connection.connection_id

  params = {
    destination_table_name_template = "TABLE_NAME"
    write_disposition                = "APPEND"
    query                            = "SELECT * FROM 'TABLE_NAME';"
    database_name                   = "DB_NAME"
    data_path                        = "TABLE_NAME"
    username                         = "USERNAME"
    password                         = data.google_secret_manager_secret_version.mysql_password.secret_data
  }
}
2

Hi @her_I_am ,

Welcome to Google Cloud Community!

This error indicates that you missed a required parameter connector.endpoint.host. A possible solution is to include this parameter in your existing params.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

3

Fixed by putting it in "

resource "google_bigquery_data_transfer_config" "mysql_to_bq_transfer2" {
display_name = "NAME"
location = "EU"
data_source_id = "mysql"
destination_dataset_id = google_bigquery_dataset.bq_dataset.dataset_id
schedule = "every 24 hours"
service_account_name = google_service_account.bq_service_account.account_id

params = {
assets = "
[\"course_overviews_courseoverview\", ]"
"connector.endpoint.host" = var.sql_instance.private_ip_address
"connector.endpoint.port" = 3306
"connector.database" = ""
"connector.authentication.username" = ""
"connector.authentication.password" = data.google_secret_manager_secret_version.mysql_password.secret_data
}
}