My requirement: I want to create a single table wh...
# general
a
My requirement: I want to create a single table where I can do aggregations like sum on integer values of a column; but also hold string values in the same column. What I am dealing with: I have a realtime table where I am ingesting data with fields
timestamp
,
id
and `value`; the
id
describes the
value
being ingested; here
value
can hold both
INT
and
STRING
for my use case. I wanted to know which of these options are feasible: 1. Create columns
value_int
and
value_string
and use a filtering function in Pinot that can save records in
value_int
if
value
is
INT
, and vice-versa for
STRING
values of
value
. I tried this, but the filter function as shown in the docs, does not allow this 2. Store all values as
STRING
and use a pinot-specific
CAST
or
CONVERT
function to do aggregations. But I could not find a cast/convert function in Pinot. Thus I am not able to do
sum
operations on the data I would welcome any ideas/workaround for the same.
m
Either option works, it depends on your latency/sla/cost requirements on which one would be better. While 1 is better read optimized than 2, 2 avoids additional storage at the cost of read time computation.
a
Thank you.
👍 1