Hi, I'm evaluating Airbyte. Need to know whether I...
# announcements
d
Hi, I'm evaluating Airbyte. Need to know whether I can map source columns to destination columns in airbyte? If not, Are there any references available on how to do it.
1
s
@Devesh Krishnani do you mean renaming source columns in the destination?
d
Renaming plus basic transformation such as concat, substring etc. For example, Source has A and B, and destination has C. I want to save data into C as concat of A and B
s
The recommended way of doing this would be in the destination itself as it is more in line with ELT (vs ETL)
d
Thanks Shrif. I will go through it.
s
Airbyte does provide ways to namespace your tables, but I suspect the kind of transformation you want is best done in the destination
r
We would like to use this kind of functionality to create add metadata columns to imported data @s, as an example: • to map a source attribute that gives info on when data was extracted from the source to a generic metadata column we call META_SOURCE_DTS • add a META_SOURCE column that adds a static source functional description code to all imported rows cc @Meena Gangadharan
s
@Rogier Werschkull are these metadata columns something that can only be added before the rows have landed in the destination?
For the extraction time Airbyte already provides this on the written data but I understand this may just be an example