hi < kapa ai> hi I am trying to set up a google drive airbyt Airbyte #replication-ask-ai

hi <@U04S8PLJWMT> hi I am trying to set up a googl...

Elena Jianu

03/06/2026, 12:30 PM

hi @kapa.ai hi I am trying to set up a google drive airbyte source using the terraform provider: I want to set up multiple streams from csv with different glob targets. For that I separated the following variable out in my terraform:

Copy code

variable "gdrive_streams_configuration" {
  type        = list(any)
  description = "The list of team configurations for selecting CSV files in the GDrive Airbyte source"
  default = [
        {
            days_to_sync_if_history_is_full = 4
            format = {
              csv_format = {
                "header_definition": "From CSV"
              }
            }
            globs = ["team_1/*.csv"]
            name                                        = "team_1"
            validation_policy                           = "Emit Record"
            use_first_found_file_for_schema_discovery   = true
            primary_key = [["id"]]

        },
        {
            days_to_sync_if_history_is_full = 3
              format = {
                csv_format = {
                  "header_definition": "From CSV"
                }
              }
            globs = ["team_2/*.csv"]
            name                                        = "team_2"
            validation_policy                           = "Emit Record"
            use_first_found_file_for_schema_discovery   = true
            primary_key = [["id"]]

        }

      ]
}

My source definition is the following:

Copy code

resource "airbyte_source_google_drive" "gdrive_source" {
    configuration = {
      credentials = {
        service_account_key_authentication = {
          service_account_info = xxxxxxxx
        }
      }
      streams = var.gdrive_streams_configuration
      
      folder_url = "<https://drive.google.com/drive/folders/${var.gdrive_folder_id}>"
    }
    name          = "GDrive CSV files to BigQuery"
    workspace_id =  var.airbyte_workspace_id
}

But when trying to plan I get this error:

Copy code

Error: Incorrect attribute value type

  on modules/airbyte_connectors/gdrive_to_bq.tf line 3, in resource "airbyte_source_google_drive" "gdrive_source":
   3:     configuration = {
   4:       credentials = {
   5:         service_account_key_authentication = {
   6:           service_account_info = xxxx
   7:         }
   8:       }
   9:       streams = var.gdrive_streams_configuration
  10:       
  11:       folder_url = "<https://drive.google.com/drive/folders/${var.client_billing_gdrive_folder_id}>"
  12:     }
    ├────────────────
    │ var.gdrive_folder_id is "xxxxxxxxx"
    │ var.gdrive_streams_configuration is list of object with 2 elements

Inappropriate value for attribute "configuration": attribute "streams":
incorrect list element type: attribute "format": attribute "csv_format":
attribute "header_definition": object required.

What is the problem and how can I fix it?

kapa.ai

03/06/2026, 12:31 PM

The error is clear: the

header_definition

field expects an object, but you're passing a plain string

"From CSV"

. Looking at the Terraform schema,

header_definition

is an attributes block with three possible nested objects:

from_csv

autogenerated

, or

user_provided

. [header_definition schema] To use "From CSV" (i.e., read headers from the CSV file itself), you need to set the

from_csv

nested block as an empty object

{}

. Here's the corrected stream configuration:

Copy code

variable "gdrive_streams_configuration" {
  type        = list(any)
  description = "The list of team configurations for selecting CSV files in the GDrive Airbyte source"
  default = [
    {
      days_to_sync_if_history_is_full           = 4
      format = {
        csv_format = {
          header_definition = {
            from_csv = {}
          }
        }
      }
      globs                                     = ["team_1/*.csv"]
      name                                      = "team_1"
      validation_policy                         = "Emit Record"
      use_first_found_file_for_schema_discovery = true
      primary_key                               = "id"
    },
    {
      days_to_sync_if_history_is_full           = 3
      format = {
        csv_format = {
          header_definition = {
            from_csv = {}
          }
        }
      }
      globs                                     = ["team_2/*.csv"]
      name                                      = "team_2"
      validation_policy                         = "Emit Record"
      use_first_found_file_for_schema_discovery = true
      primary_key                               = "id"
    }
  ]
}

A couple of additional notes: • header_definition
must be an object with one of

from_csv

autogenerated

, or

user_provided

as a nested block — not a string. [header_definition schema] • primary_key
in the stream schema is typed as a

String

, not a list of lists. You should pass it as

"id"

rather than

[["id"]]

. [streams schema] 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

kapa.ai

03/06/2026, 12:31 PM

The error comes from how

header_definition

is structured in the Terraform schema for the Google Drive source. In the provider,

header_definition

is not a string; it is an object with one of three nested attributes:

autogenerated

from_csv

, or

user_provided

. [CSV header] So this:

Copy code

format = {
  csv_format = {
    "header_definition": "From CSV"
  }
}

does not match the expected type (

object

), which is why Terraform complains:

attribute "header_definition": object required

. To use “From CSV” (i.e., use the header row from the CSV file), you must set

header_definition.from_csv

as an (empty) object, e.g.:

Copy code

variable "gdrive_streams_configuration" {
  type        = list(any)
  default = [
    {
      days_to_sync_if_history_is_full = 4
      format = {
        csv_format = {
          header_definition = {
            from_csv = {}
          }
        }
      }
      globs                                      = ["team_1/*.csv"]
      name                                       = "team_1"
      validation_policy                          = "Emit Record"
      use_first_found_file_for_schema_discovery  = true
      primary_key                                = [["id"]]
    },
    {
      days_to_sync_if_history_is_full = 3
      format = {
        csv_format = {
          header_definition = {
            from_csv = {}
          }
        }
      }
      globs                                      = ["team_2/*.csv"]
      name                                       = "team_2"
      validation_policy                          = "Emit Record"
      use_first_found_file_for_schema_discovery  = true
      primary_key                                = [["id"]]
    }
  ]
}

This matches the nested schema: ~ text configuration.streams.format.csv_format.header_definition { optional: autogenerated { ... } from_csv { ... } user_provided { column_names = [...] } } 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.

3 Views

Open in Slack

Previous Next