Hi Pinot Community, We are evaluating Pinot for O...
# general
r
Hi Pinot Community, We are evaluating Pinot for Observability hot store (Distributed tracing, Logging and RUM) at scale (M/sec events ingestion). Any public case study/benchmark for the same ? Seems like Clickhouse is more popular option for Distributed Tracing/Structured Logging (Uber, Ebay, Cloudflare etc) and Pinot is more popular option for RUM (Uber etc). Rather two different stores for different use cases, we are exploring if Pinot alone can serve all O11y use cases. Any suggestions ?
k
You can use Pinot for all of them.
h
Have used Pinot for distributed tracing. Ingestion scale was around ~48TB/day. It worked out great.
r
That’s nice to hear. Any blog/links please ? Have seen couple of youtube videos (from startree) wrt o11y using Pinot. Couldn’t find any other references
We used Opentelemetry-Collector -> Kafka -> Hypertrace -> Pinot. The Opentelemetry collector has sufficient proccessors/exporters developed now and the cluster pipeline can be just
Opentelemetry-Collector -> Kafka -> Pinot
.
r
Yeah this looked promising. But their scale is bit small (100k/s) and no h/w footprint info was mentioned. Also hypertrace OSS project no longer seems active.
h
What's the ingestion scale you are targeting in terms of size?
Also hypertrace OSS project no longer seems active.
Yes, Hypertrace is not required now, This blog is bit old. As I mentioned above the Opentelemetry Collector project is quite mature now and every component of Hypertrace can be simply replaced by Otel-Collector. Hence you can just simply Push events from Otel-Collector to Pinot.
r
ingestion size could be few TB/day (or) few M events/sec QPs would be few hundreds/sec (peak)
p
Razorpay was the use case I knew of that related to Traces. As noted, they've moved on from the architecture mentioned in that blog. More observability use cases for Apache Pinot: • Uber: Serving Real-Time App Crash Analytics While Saving $2M+ With Apache Pinot — metrics (Elasticsearch replacement) • Cisco Webex: Real-Time Observability Using Apache Pinot and Grafana — metrics (Elasticsearch replacement) For logging, most of the focus has been around how Uber added Compressed Log Processor (CLP) to Apache Pinot, which would allow log file entries to remain compressed while being able to "peek" into them to run queries: • Uber: Reducing Logging Cost by Two Orders of Magnitude using CLP • Uber: Modernizing Logging at Uber with CLP (Part II) • Apache Pinot docs: Stream ingestion with CLP Not tied to a specific customer use case, but a nice related blog: • Optimizing Workflow Log Management with Apache Pinot: An Upsert Approach — Shruti works at Moveworks, which is a StarTree customer