@kapa.ai if I have a connection defined which only populates a few columns in the destination, does Airbyte only pull those few columns from the source or does airbyte store all the source columns anyway (in temporary storage or some other mechanism)?

Airbyte, by default, syncs all detected fields from the source. However, you have the option to select specific fields to be synced to the destination. When you deselect a partial set of fields, Airbyte may read all data from the source (depending on the source), but it will only write data to the destination from fields you selected. Deselecting fields will not prevent Airbyte from reading them. It's important to note that it is not possible to deselect a primary key or cursor if your selected sync mode requires it to sync. Field selection is currently only available for top-level fields. You cannot deselect nested fields [^1^]. However, there is a discussion in the Airbyte community forum where a user reported that even after editing the catalog to remove a column, it could still show up in the destination. The user used a BigQuery destination in a connection and removed a column from the catalog. The final (normalized) table looked OK but unfortunately, the raw table still had it [^2^]. [^1^]: (https://docs.airbyte.com/using-airbyte/configuring-schema#select-fields) [^2^]: (https://discuss.airbyte.io/t/2933)

@kapa.ai Specifically in the case of the Hubspot connector, does Airbyte read all columns of the source regardless of if those columns will be used in the destination?

A message from kapa.ai

Yes, for the Hubspot connector, Airbyte reads all columns from the source regardless of whether those columns will be used in the destination. This is because, by default, Airbyte syncs all detected fields from the source. However, it will only write data to the destination from fields you have selected. Even if you deselect certain fields, Airbyte will still read them from the source, but it won't write them to the destination[^1^]. Please note that field selection is currently only available for top-level fields. You cannot deselect nested fields[^1^]. [^1^]: (https://docs.airbyte.com/using-airbyte/configuring-schema#select-fields)

@kapa.ai does that mean that if I create a second destination and send the data there, Airbyte will not need to re-read the source because it will already have it locally?

A message from kapa.ai

No, Airbyte does not store the source data locally after reading it. Each time you create a new destination and start a sync, Airbyte will read the data from the source again. This is because Airbyte operates on a ELT (Extract, Load, Transform) model, where data is extracted from the source and loaded into the destination each time a sync operation is performed[^1^]. [^1^]: (https://airbyte.com/tutorials/full-data-synchronization#step-1)