<@U04SE163WC8> will future enterprise versions of ...
# ask-aib
Brian Kasen
05/31/2024, 3:00 AM@kapa.ai will future enterprise versions of Airbyte offer dbt transformation feature that will be deprecated in the open source version?
k
kapa.ai
05/31/2024, 3:00 AMBased on the provided knowledge sources, the custom dbt transformation feature is being deprecated across all versions of Airbyte, including the Open Source (OSS) and Self-Managed Enterprise versions. The deprecation is effective as of March 31, 2024. Here are the relevant points: 1. Deprecation Announcement: - Custom dbt transformations have been deprecated in Airbyte OSS as of version 0.57.0 ([source](https://docs.airbyte.com/release_notes/april_2024#deprecated-features)). - The feature will be removed on March 31, 2024 ([source](https://docs.airbyte.com/operator-guides/transformation-and-normalization/transformations-with-dbt)). 2. Future Plans: - There is no indication that the custom dbt transformation feature will continue to be supported in future enterprise versions. The focus is on replacing normalization with Typing and Deduping ([source](https://docs.airbyte.com/operator-guides/transformation-and-normalization/transformations-with-dbt)). - Airbyte plans to support "post-sync" webhooks that trigger on sync completion in the future, but no specific timeline has been committed ([source](https://github.com/airbytehq/airbyte/discussions/34860)). For more detailed information, you can refer to the discussion on the deprecation of custom dbt transformations [here](https://github.com/airbytehq/airbyte/discussions/34860). If you have any further questions or need more details, feel free to ask!
b
Brian Kasen
05/31/2024, 3:01 AM@kapa.ai how are customers handling encryption of data upon ingestion to destination with this feature being deprecated?
k
kapa.ai
05/31/2024, 3:01 AMA message from kapa.ai
kapa.ai
05/31/2024, 3:01 AMWith the deprecation of the custom dbt transformation feature, customers looking to handle encryption of data upon ingestion to the destination will need to explore alternative methods. Here are some actionable steps and recommendations: 1. Use External Orchestration Tools: - Airbyte recommends using external orchestration tools like Dagster or Airflow to manage transformations, including encryption tasks. You can find more information on integrating these tools with Airbyte in the [documentation](https://docs.airbyte.com/operator-guides/using-dagster-integration). 2. Post-Sync Webhooks: - Airbyte plans to support "post-sync" webhooks that trigger on sync completion. This feature can be used to initiate encryption processes after data ingestion. However, the specific timeline for this feature is not yet committed. Keep an eye on the [discussion](https://github.com/airbytehq/airbyte/discussions/34860) for updates. 3. Custom Scripts and Tools: - Customers can create custom scripts or use existing tools to handle encryption. These scripts can be triggered by the external orchestration tools mentioned above or manually executed after data ingestion. 4. Database-Level Encryption: - Utilize encryption features provided by the destination database or data warehouse. Many modern databases and data warehouses offer built-in encryption capabilities that can be configured to encrypt data upon ingestion. 5. Airbyte's Future Enhancements: - Stay updated with Airbyte's roadmap and future enhancements. The platform team is developing core features for Airbyte Enterprise and enhancements to features like schema propagation, which might include better support for encryption and other data security measures. You can reference the roadmap [here](https://github.com/orgs/airbytehq/projects/37/views/1). By leveraging these methods, customers can ensure that their data remains secure and encrypted during and after the ingestion process, even with the deprecation of the custom dbt transformation feature. If you have any further questions or need more details, feel free to ask!