Hello all, I am trying to use airbyte for a scenar...
# ask-community-for-troubleshooting
s
Hello all, I am trying to use airbyte for a scenario where data needs to be anonymized before being pushed to S3 destination. I am planning to use faker python package to do the anonymization (first cut) . I looked at the airbyte documentation, but I don't find support for adding custom python code as a transformation step. We are kind of at this point constrained to use a Data Warehouse and dbt to run any custom transformations. Is there a plan to support custom , preferably python based , non-dbt (so that no requirement to have a warehouse setup) transformations in near future. Thanks
s
@Shashi Singh supporting anonymization is on the roadmap potentially for q2 2022 (apr-jun). What sort of anonymization would you like to run?
s
@s thanks for your reply. In the past I have used faker python package to anonymize names and emails. I was hoping to build something similar because the team is more comfortable in python vs SQL/dbt. I don't want to configure a data warehouse for the purpose of anonymization.