:question:Looks like partition-based segment pruni...
# troubleshooting
n
❓Looks like partition-based segment pruning in Pinot can be configured for multiple columns for
offline
tables. Appreciate any help in confirming if the same applies to
real-time
tables as well 🙇‍♀️ Here is the associated reference for Kafka stream, but it doesn't explicitly mention if partition-based segment pruning can be set up for multiple columns in the corresponding Pinot table config: https://docs.pinot.apache.org/basics/getting-started/frequent-questions/ingestion-faq#how-do-i-enable-partitioning-in-pinot-when-using-kafka-stream I guess the pre-requisite is: Input Kafka stream needs to be configured with a custom partitioner to match the partition column(s), partition function and number of partitions set up in the Pinot table config.
cc: @Sai Tarun Tadakamalla
🍷 1
m
Yea, for RT upstream will need to take care of that
n
Thanks @Mayank!
Do we have to setup something like below in the RT table config involving >1 column?
Copy code
"tableIndexConfig": {
      ..
      "segmentPartitionConfig": {
        "columnPartitionMap": {
          "memberId": {
            "functionName": "Modulo",
            "numPartitions": 3 
          },
          "caseNumber": {
            "functionName": "Murmur",
            "numPartitions": 12 
          }
        }
      }