Hi, I have started exploring apache pinot, there ...
# pinot-dev
m
Hi, I have started exploring apache pinot, there are few query regarding schema of apache pinot. I want to understand how apache pinot work with Kafka topic that has AVRO schema (schema includes nested object, array of object etc..) because i didn't find any resource or example that shows how we can inject data with Kafka that has avro schema with it. As per my understanding apache pinot we have to provide flat schema or other option for nested Json object we can use transform function. Avro schema
Copy code
{
  "namespace": "my.avro.ns",
  "name": "MyRecord",
  "type": "record",
  "fields": [
    {
      "name": "uid",
      "type": "int"
    },
    {
      "name": "somefield",
      "type": "string"
    },
    {
      "name": "options",
      "type": {
        "type": "array",
        "items": {
          "type": "record",
          "name": "lvl2_record",
          "fields": [
            {
              "name": "item1_lvl2",
              "type": "string"
            },
            {
              "name": "item2_lvl2",
              "type": {
                "type": "array",
                "items": {
                  "type": "record",
                  "name": "lvl3_record",
                  "fields": [
                    {
                      "name": "item1_lvl3",
                      "type": "string"
                    },
                    {
                      "name": "item2_lvl3",
                      "type": "string"
                    }
                  ]
                }
              }
            }
          ]
        }
      }
    }
  ]
}
Kafka Avro Message:
Copy code
{
  "uid": 29153333,
  "somefield": "somevalue",
  "options": [
    {
      "item1_lvl2": "a",
      "item2_lvl2": [
        {
          "item1_lvl3": "x1",
          "item2_lvl3": "y1"
        },
        {
          "item1_lvl3": "x2",
          "item2_lvl3": "y2"
        }
      ]
    }
  ]
}