Hello smile Any idea why the ` segments tableName metadata` Apache Pinot #troubleshooting

Hello :smile: Any idea why the `/segments/{tableNa...

Jonathan Meyer

07/02/2021, 3:44 PM

Hello 😄 Any idea why the

/segments/{tableName}/metadata

controller API endpoint only returns metadata of one segment (a random one, it seems) and not of all of them ? The description states

Get the server metadata for all table segments

Jonathan Meyer

07/02/2021, 3:51 PM

/segments/communities?type=OFFLINE

Copy code

[
  {
    "OFFLINE": [
      "communities_OFFLINE_0",
      "communities_OFFLINE_1",
      "communities_OFFLINE_10",
      "communities_OFFLINE_11",
      "communities_OFFLINE_12",
      "communities_OFFLINE_13",
      "communities_OFFLINE_14",
      "communities_OFFLINE_15",
      "communities_OFFLINE_16",
      "communities_OFFLINE_17",
      "communities_OFFLINE_18",
      "communities_OFFLINE_19",
      "communities_OFFLINE_2",
      "communities_OFFLINE_20",
      "communities_OFFLINE_21",
      "communities_OFFLINE_22",
      "communities_OFFLINE_23",
      "communities_OFFLINE_24",
      "communities_OFFLINE_25",
      "communities_OFFLINE_26",
      "communities_OFFLINE_27",
      "communities_OFFLINE_28",
      "communities_OFFLINE_29",
      "communities_OFFLINE_3",
      "communities_OFFLINE_30",
      "communities_OFFLINE_31",
      "communities_OFFLINE_32",
      "communities_OFFLINE_33",
      "communities_OFFLINE_34",
      "communities_OFFLINE_35",
      "communities_OFFLINE_4",
      "communities_OFFLINE_5",
      "communities_OFFLINE_6",
      "communities_OFFLINE_7",
      "communities_OFFLINE_8",
      "communities_OFFLINE_9"
    ]
  }
]

but

/segments/communities/metadata?type=OFFLINE"

Copy code

{
  "communities_OFFLINE_9": {
    "segmentName": "communities_OFFLINE_9",
    "schemaName": null,
    "crc": 3737963570,
    "creationTimeMillis": 1625099198953,
    "creationTimeReadable": "2021-07-01T00:26:38:953 UTC",
    "timeGranularitySec": null,
    "startTimeMillis": null,
    "startTimeReadable": null,
    "endTimeMillis": null,
    "endTimeReadable": null,
    "pushTimeMillis": -9223372036854776000,
    "pushTimeReadable": null,
    "refreshTimeMillis": -9223372036854776000,
    "refreshTimeReadable": null,
    "segmentVersion": "v3",
    "creatorName": null,
    "paddingCharacter": "\u0000",
    "custom": {
      "input.data.file.uri": "file:/communities/raw/608280708d8d6da460674ec0.parquet"
    },
    "columns": [],
    "indexes": {
      "parentDepartmentId": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "$segmentName": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "$hostName": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "entityType": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "entityId": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "communityId": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "$docId": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      }
    },
    "star-tree-index": null
  }
}

Kishore G

07/02/2021, 4:01 PM

are all the segments ONLINE?

Jonathan Meyer

07/02/2021, 4:02 PM

@Kishore G Yes

Jonathan Meyer

07/02/2021, 4:02 PM

image.png

Jonathan Meyer

07/02/2021, 4:22 PM

pinot_metadata-2021-07-02_18.20.20.mp4

Kishore G

07/02/2021, 4:26 PM

Thanks.. that was super useful.. we should upload this as an example for troubleshooting channel 😉

Kishore G

07/02/2021, 4:26 PM

Saves a lot of back and forth

Kishore G

07/02/2021, 4:37 PM

seems like a bug to me. can you file an issue

Jonathan Meyer

07/02/2021, 4:37 PM

I'll do that, thanks @Kishore G

Jonathan Meyer

07/11/2021, 4:41 PM

-> https://github.com/apache/incubator-pinot/issues/7151

Open in Slack

Previous Next