Is the initial ingest going to take a lot of time if we have too many schemas and tables? If yes, is there a way to optimize initial ingest time?
d
delightful-ram-75848
06/08/2023, 12:09 AM
Hi, you could do this by modifying ingestion config.
1. Use patterns / filters for the initial ingestion
2. Turn off lineage / profiling, etc
What is the data source are you integrating with, and what is the scale?
d
dazzling-rainbow-96194
06/23/2023, 3:19 PM
I already added filter and removed both lineage and profiling. Source is Snowflake and we are connecting to only a limited set for testing. Right now I am filtering 5 tables with 1000 or less records in each table.