https://pinot.apache.org/ logo
#getting-started
Title
# getting-started
s

srikanth rangan

12/25/2021, 5:39 PM
Copy code
Hi team, I have JSON tags and metrics. the problem is that the data has METRIC JSON and the Pinot add schema does not allow me to specify metric field as json. How can I go ahead and define my schema ? :  
{ 
        "tags": { "real_name": "marvelmohinish99", "team": "TN4AF0V5W" },
        "metrics": { "salary": 8520, "performance": 22.5 },
        "mtime" : 1456249661342
}
k

Kishore G

12/25/2021, 6:53 PM
do you know set of metrics ahead of time?
I see salary and performance as metrics
s

srikanth rangan

12/25/2021, 8:39 PM
Hi @User, Thanks for your reply. I am actually subscribing to a kafka topic in which different kinds of metrics will come, both the tags and metrics are going to be heterogeneous.
heterogeneous data in topic "USER_DATA"
{
"tags": { "real_name": "marvelmohinish99", "team": "TN4AF0V5W" },
"metrics": { "salary": 8520, "performance": 22.5 },
"mtime" : 1456249661342
}
{
"tags": { "ID": 2244, "team": "TN4AF0V5W", "SOFTWARE": "AND-1" },
"metrics": { "usage": 33, "tables": 68 },
"mtime" : 1456249661346
}
@User: Your youtube video on linkedIn with apache pinot was awesome and inspiring!
🙏 1
for each "type" of message, I know the data that I need before hand.
k

Kishore G

12/25/2021, 9:13 PM
ok, you have multiple options on how you want to model the schema in Pinot Option 1 message_type| tags_json| metrics_json| mtime
Option 2 message_type| tags_json| salary | performance| other metric columns| mtime
s

srikanth rangan

12/25/2021, 9:28 PM
i was thinking a of using option1, but was stuck at schema definition on Pinot interface. It allows me to define dimensions (tags in my case) as JSON but the interface does not allow me to define Metrics as a JSON field.
just to illustrate:
k

Kishore G

12/25/2021, 9:56 PM
it does not matter in this case
you can say metrics_json as dimension for now and it should work
we will fix the UI, please file an issue
s

srikanth rangan

12/25/2021, 10:06 PM
sure, I will do that, Thanks a lot, Kishore
So I can perform aggregations on the metrics, even if I declare them as dimensions, right? just wanted to be sure.
k

Kishore G

12/25/2021, 10:37 PM
Yes
s

srikanth rangan

12/26/2021, 8:01 AM
thanks