Hi All, I am new to Apache pinot and considering t...
# general
k
Hi All, I am new to Apache pinot and considering to use it, I am wondering if it is the right fit for our use case. We are a data intensive application and would use Apache pinot as a central db wherein we would gather data from multiple countries. Following are my questions 1. We want to use it for user facing analytics wherein we would ingest data real time and query real time. My question regarding the same is that how frequently can we update historical data/rows or can we not ? For eg, a user is added in our app, we will ingest user details in pinot and now we want to update the user data, can that update happen on the same data we stored in pinot ? I have gone through druid documentation and it seems to be similar to pinot, but druid does not allow row level updates or historical updates. Is the case similar in pinot as well ? 2. We will have to query multiple tables, so will joins be efficient ? Thanks
l
Re: (1) Pinot provides the upsert ability Realtime tables. Don't know if there is a rate limit. I have no direct experience with this, but fwiw recent PR provides for compacting upserts
m
Yes, upsert is what you are looking for. It does require partitioning the data upstream on a pk (to identify rows to mutate). What kind of joins (size of left and right) are you looking at and what’s the throughput/latency requirements.