https://pinot.apache.org/ logo
j

Jonathan Meyer

07/02/2021, 3:44 PM
Hello 😄 Any idea why the
/segments/{tableName}/metadata
controller API endpoint only returns metadata of one segment (a random one, it seems) and not of all of them ? The description states
Get the server metadata for all table segments
/segments/communities?type=OFFLINE
Copy code
[
  {
    "OFFLINE": [
      "communities_OFFLINE_0",
      "communities_OFFLINE_1",
      "communities_OFFLINE_10",
      "communities_OFFLINE_11",
      "communities_OFFLINE_12",
      "communities_OFFLINE_13",
      "communities_OFFLINE_14",
      "communities_OFFLINE_15",
      "communities_OFFLINE_16",
      "communities_OFFLINE_17",
      "communities_OFFLINE_18",
      "communities_OFFLINE_19",
      "communities_OFFLINE_2",
      "communities_OFFLINE_20",
      "communities_OFFLINE_21",
      "communities_OFFLINE_22",
      "communities_OFFLINE_23",
      "communities_OFFLINE_24",
      "communities_OFFLINE_25",
      "communities_OFFLINE_26",
      "communities_OFFLINE_27",
      "communities_OFFLINE_28",
      "communities_OFFLINE_29",
      "communities_OFFLINE_3",
      "communities_OFFLINE_30",
      "communities_OFFLINE_31",
      "communities_OFFLINE_32",
      "communities_OFFLINE_33",
      "communities_OFFLINE_34",
      "communities_OFFLINE_35",
      "communities_OFFLINE_4",
      "communities_OFFLINE_5",
      "communities_OFFLINE_6",
      "communities_OFFLINE_7",
      "communities_OFFLINE_8",
      "communities_OFFLINE_9"
    ]
  }
]
but
/segments/communities/metadata?type=OFFLINE"
Copy code
{
  "communities_OFFLINE_9": {
    "segmentName": "communities_OFFLINE_9",
    "schemaName": null,
    "crc": 3737963570,
    "creationTimeMillis": 1625099198953,
    "creationTimeReadable": "2021-07-01T00:26:38:953 UTC",
    "timeGranularitySec": null,
    "startTimeMillis": null,
    "startTimeReadable": null,
    "endTimeMillis": null,
    "endTimeReadable": null,
    "pushTimeMillis": -9223372036854776000,
    "pushTimeReadable": null,
    "refreshTimeMillis": -9223372036854776000,
    "refreshTimeReadable": null,
    "segmentVersion": "v3",
    "creatorName": null,
    "paddingCharacter": "\u0000",
    "custom": {
      "input.data.file.uri": "file:/communities/raw/608280708d8d6da460674ec0.parquet"
    },
    "columns": [],
    "indexes": {
      "parentDepartmentId": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "$segmentName": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "$hostName": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "entityType": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "entityId": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "communityId": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      },
      "$docId": {
        "bloom-filter": "NO",
        "dictionary": "YES",
        "forward-index": "YES",
        "inverted-index": "YES",
        "null-value-vector-reader": "NO",
        "range-index": "NO",
        "json-index": "NO"
      }
    },
    "star-tree-index": null
  }
}
k

Kishore G

07/02/2021, 4:01 PM
are all the segments ONLINE?
j

Jonathan Meyer

07/02/2021, 4:02 PM
@Kishore G Yes
message has been deleted
k

Kishore G

07/02/2021, 4:26 PM
Thanks.. that was super useful.. we should upload this as an example for troubleshooting channel 😉
Saves a lot of back and forth
seems like a bug to me. can you file an issue
j

Jonathan Meyer

07/02/2021, 4:37 PM
I'll do that, thanks @Kishore G