This message was deleted.
# generals
Slackbot
10/03/2023, 10:22 AMThis message was deleted.
a
Abhishek Agarwal
10/03/2023, 12:45 PMIt would certainly be faster. And as you start running lots of queries running on a system like Athena that charges by data scanned, the price-performance economics becomes more favorable with druid.
m
Michael Yee
10/03/2023, 12:54 PMI, likely most people, are looking at 2 use cases.
We have basic monthly reports, so this would be a query that would hopefully be cached and new data would be added to the cached data to speed things up...
Ad hoc reports is where I was wondering if I could save on data warehouse and hopefully be like sub min query response time depending on amount of data read.
I was wondering if you or anyone here had practical experience to see if this is a possible solution.
a
Abhishek Agarwal
10/03/2023, 1:05 PMAd-hoc reports seem like a more suitable fit than monthly reports, as they run very infrequently. For ad-hoc reports, too, if you don't care about performance that much and you don't have many concurrent users, cloud datawarehouses could be a better fit. If you do care about performance or/and require higher query concurrency and/or require real-time data, then druid is a good alternative.
I am here speaking as a druid developer and also as an employee of the company where we have customers using Druid for various use cases. Some of these customers did shift their workload from Athena to Druid for cost-efficiency.
m
Michael Yee
10/03/2023, 1:33 PMactually, I misspoke when I said monthly reports. These are near real time dashboards and some people base their monthly reports on them.
Michael Yee
10/03/2023, 1:36 PMI was looking at performance benchmarking for Druid, I found a few like https://rockset.com/blog/comparing-rockset-apache-druid-clickhouse-real-time-analytics, but mostly older reports from 2020-2021.
Wondering if you know any recent performance benchmarking for Druid and Athena vs other?
a
Abhishek Agarwal
10/03/2023, 1:49 PMyeah. that's an old one 🙂 and even at that time, wasn't written with the latest version of druid (at that time) as a reference. There is https://imply.io/videos/benchmarking-apache-druid/
Abhishek Agarwal
10/03/2023, 1:54 PMThat being said, I feel benchmarks in general are biased and the performance you will get would often depend on your particular use case. Qualitatively, I can confidently say that druid would definitely beat Athena in terms of performance. Just because the druid is designed from ground up with performance in mind. The target use-cases for the two are quite different in nature.
m
Michael Yee
10/03/2023, 5:18 PMWhat is the best resource to learn / optimize Apache Druid?
Michael Yee
10/03/2023, 5:21 PMa
Abhishek Agarwal
10/04/2023, 4:22 AMYeah. That would be a good place to begin.
2 Views