https://pinot.apache.org/ logo
Join Slack
Powered by
# general
  • k

    Kishore G

    09/20/2019, 11:55 PM
    Yes. It’s possible but needs some tuning on stream side and Pinot side
  • a

    Andrew Greene

    09/20/2019, 11:59 PM
    Okay, thank you. 😃
  • a

    Andrew Greene

    09/20/2019, 11:59 PM
    I'll give it a shot and get back to you.
  • k

    Kishore G

    09/24/2019, 2:53 AM
    I am talking to Thomas later today at 8:30 pm. if anyone is interested in joining the call, let me know. They are trying to setup Pinot on Azure and facing some issues.
  • k

    Kishore G

    09/25/2019, 4:01 PM
    @User Do we have instructions on how to configure PinotFS for ADSL?
  • j

    Jennifer Dai

    09/25/2019, 4:22 PM
    yes, let me find the wiki
  • j

    Jennifer Dai

    09/25/2019, 4:23 PM
    wanted to clarify - you mean ADL/azure data lake and not azure blob storage, right?
  • j

    Jennifer Dai

    09/25/2019, 4:24 PM
    https://pinot.readthedocs.io/en/latest/pluggable_storage.html?highlight=pinotfs
  • j

    Jennifer Dai

    09/25/2019, 4:25 PM
    this is the wiki, let me know if it's unclear
  • m

    Mayank

    09/25/2019, 4:25 PM
    Thanks @User. Would be good to add a sample config in the doc.
  • j

    Jennifer Dai

    09/25/2019, 4:25 PM
    do you mean for client secret/client id, etc?
  • m

    Mayank

    09/25/2019, 4:28 PM
    Yes, a json sample of the config to provide.
  • k

    Kishore G

    09/25/2019, 4:53 PM
    A sample config along segment upload command assuming data will help a lot
    ➕ 1
  • j

    Jennifer Dai

    09/25/2019, 5:05 PM
    @User can you rephrase? do you mean http segment upload command?
  • k

    Kishore G

    09/25/2019, 5:07 PM
    Yes
  • k

    Kishore G

    09/25/2019, 5:08 PM
    A sample curl call
  • v

    vibhor

    09/27/2019, 5:40 AM
    my pinot setup is able to scan only 7 segments out of many segments..
  • v

    vibhor

    09/27/2019, 5:40 AM
    can someone help on this?
  • v

    vibhor

    09/27/2019, 5:45 AM
    <!here>
  • k

    Kishore G

    09/27/2019, 5:46 AM
    Sure
  • k

    Kishore G

    09/27/2019, 5:46 AM
    How many segments do you have
  • k

    Kishore G

    09/27/2019, 5:47 AM
    Can you paste the query and response
  • v

    vibhor

    09/27/2019, 5:50 AM
    {"pql":"select count(*) from dauTable"} .
  • v

    vibhor

    09/27/2019, 5:50 AM
    Copy code
    {
      "aggregationResults": [
        {
          "function": "count_star",
          "value": "12488040"
        }
      ],
      "exceptions": [],
      "numServersQueried": 1,
      "numServersResponded": 1,
      "numSegmentsQueried": 7,
      "numSegmentsProcessed": 7,
      "numSegmentsMatched": 7,
      "numDocsScanned": 12488040,
      "numEntriesScannedInFilter": 0,
      "numEntriesScannedPostFilter": 0,
      "numGroupsLimitReached": false,
      "totalDocs": 12488040,
      "timeUsedMs": 4,
      "segmentStatistics": [],
      "traceInfo": {}
    }
  • v

    vibhor

    09/27/2019, 5:50 AM
    no of segments : 600
  • k

    Kishore G

    09/27/2019, 5:52 AM
    How did you get the number of segments
  • k

    Kishore G

    09/27/2019, 5:52 AM
    From the response it looks like it queried all the segments that are ONLINE
  • v

    vibhor

    09/27/2019, 5:55 AM
    outdir has 600 folders with name dauSegments_600 (e.g) ..
  • v

    vibhor

    09/27/2019, 5:55 AM
    and we have data mismatch for the rows hence
  • k

    Kishore G

    09/27/2019, 5:56 AM
    Maybe all the segments were not uploaded?
1...868788...160Latest