Hello, how does Pinot handle processing incomplete...
# general
p
Hello, how does Pinot handle processing incomplete messages for realtime tables? If I have the following schema:
Copy code
{
  "schemaName": "hitexecutionview",
  "dimensionFieldSpecs": [
    {
      "name": "id",
      "dataType": "STRING"
    },
    {
      "name": "jobId",
      "dataType": "STRING"
    },
    {
      "name": "crowdMemberId",
      "dataType": "STRING"
    },
    {
      "name": "projectId",
      "dataType": "STRING"
    },
    {
      "name": "result",
      "dataTYpe": "STRING"
    }
  ],
  "dateTimeFieldSpecs": [
    {
      "name": "timestamp",
      "dataType": "LONG",
      "format": "1:MILLISECONDS:EPOCH",
      "granularity": "1:MILLISECONDS"
    }
  ],
  "primaryKeyColumns": [
    "id"
  ]
}
And the following kafka message:
Copy code
{
  "id": "19281-3123n1283-12312-312",
  "jobId": "245d-2334-fs33-23f4",
  "crowdMemberId": "xxxxxxxxxx",
  "projectId": "49mf-f39f-25v2-989m",
  "timestamp": "1238648237"
}
The field result is not passed, will pinot assume a null value? What happens if there are computed fields based on this non-existing result?
k
both result and other fields that are derived from result field will be null
null --> defaultNullValue
👍 1