Has anyone here found S3 as a good alternative to warehouses like Redshift? If yes, what approach was used to sync data from a DB like Postgres into S3? And what BI tool was used to analyse the data?

Has anyone here found S3 as a good alternative

• Isn't that what Amazon-Redshift-Spectrum is for? • Or would you also consider using Snowflake?

@Jins Kadwood everything that I'm reading is that trends are moving away from data lakes such as S3. Cloud data warehouse storage fees are increasingly becoming less cost prohibitive with decreased complexity. That being said GCP and Snowflake are leading the market in data warehousing. Throw in DBT and the Data Visualization tool of your choice and you've just implemented a modern data stack.

Agreed here! I would definitely rather choose BigQuery or Snowflake over Redshift nowadays if i had to make the choice...

I’m really interested in hearing more and more about snowflake but when I look at all the details around the costs and uses I’m very confused

I recommend setting up a POC/trial with them. You get 14 days (i think) and can determine the amount of usage. Pricing sounds complex at first, but is pretty easy once you're used to it. You start with an extra small warehouse which costs 1 credit to run per hour (credits are anywhere from $2-$4 depending on standard vs enterprise plan). Let's assume you are using standard plan and you're running a query that takes 15 minutes, you just spent 50 cents. Now warehouses scale out and up. Each scale up costs twice as much, but also runs twice as fast (up to a point). Moving to a small you can run the same query in 7.5 minutes @ $4/hour or the same 50 cents. At a certain point you hit diminishing returns where a 4xLarge warehouse runs only 1.5x faster (instead of 2x) than the next size down, thus you spend more money. The charges only start adding up significantly when you have the warehouses always on, because you are always running queries/ingesting data.