Hi Pinot heroes like <@UDQU92KBK>! I'm attempting...
# troubleshooting
t
Hi Pinot heroes like @Mayank! I'm attempting to use a minion merge rollup task (as @Mayank helpfully suggested), but it seems to do... nothing. The segments aren't getting merged and I'm not seeing any relevant logging in minion or controller logs. Details in thread... but here's what my OFFLINE table's task definition looks like:
Copy code
"task": {
      "taskTypeConfigsMap": {
        "MergeRollupTask": {
          "5min.bucketTimePeriod": "5min",
          "5min.bufferTimePeriod": "15min",
          "1hour.bucketTimePeriod": "1h",
          "1hour.bufferTimePeriod": "2h",
          "1day.bucketTimePeriod": "1d",
          "1day.bufferTimePeriod": "1d"
        }
      }
    },
Table:
Copy code
{
  "OFFLINE": {
    "tableName": "simple_OFFLINE",
    "tableType": "OFFLINE",
    "segmentsConfig": {
      "schemaName": "simple",
      "retentionTimeUnit": "DAYS",
      "retentionTimeValue": "1",
      "replication": "1",
      "segmentPushType": "APPEND",
      "segmentAssignmentStrategy": "BalanceNumSegmentAssignmentStrategy"
    },
    "tenants": {
      "broker": "DefaultTenant",
      "server": "DefaultTenant"
    },
    "tableIndexConfig": {
      "invertedIndexColumns": [
        "a",
        "b"
      ],
      "rangeIndexVersion": 2,
      "autoGeneratedInvertedIndex": false,
      "createInvertedIndexDuringSegmentGeneration": false,
      "loadMode": "HEAP",
      "enableDefaultStarTree": false,
      "enableDynamicStarTreeCreation": false,
      "aggregateMetrics": false,
      "nullHandlingEnabled": false
    },
    "metadata": {
      "customConfigs": {}
    },
    "task": {
      "taskTypeConfigsMap": {
        "MergeRollupTask": {
          "5min.bucketTimePeriod": "5min",
          "5min.bufferTimePeriod": "15min",
          "1hour.bucketTimePeriod": "1h",
          "1hour.bufferTimePeriod": "2h",
          "1day.bucketTimePeriod": "1d",
          "1day.bufferTimePeriod": "1d"
        }
      }
    },
    "fieldConfigList": [],
    "isDimTable": false
  }
}
Schema:
Copy code
{
  "schemaName": "simple",
  "dimensionFieldSpecs": [
    {
      "name": "a",
      "dataType": "STRING"
    },
    {
      "name": "b",
      "dataType": "STRING"
    }
  ]
}
controller config map, per docs, contains:
Copy code
# For segment merges:
controller.task.scheduler.enabled=true
controller.task.frequencyPeriod=5m
So why no task? What am I missing?
And yes, I have a minion k8s Pod running.
BTW: for this testing I've installed Pinot on k8s using the
pinot-quickstart
from https://docs.pinot.apache.org/basics/getting-started/kubernetes-quickstart . I'm assuming that since the result of this (which uses a Helm chart) includes running minion pods that everything needed for this merge task is supported by the version of Pinot I'm using — whatever version those instructions happen to use, which I don't see in the controller web UI anywhere. Is that a valid assumption?
m
@Haitao Zhang could you take a look?
thankyou 1
h
Ack
👋 1
thankyou 1
had 1:1 discussion, there are two issues: 1. "5min.bucketTimePeriod": "5min" should be
5m
2. we should add a time column to ensure MergeRollup can take filter segments into different time buckets
m
Can we update doc to ensure the format for various time periods (min, vs m, etc)?
t
Thanks @Haitao Zhang!
1. "5min.bucketTimePeriod": "5min" should be
5m
Already fixed.
2. we should add a time column to ensure MergeRollup can take filter segments into different time buckets
Working on it! The MergeRollup task docs were not, I think, clear on this one, and could be made so. Thanks!