https://pinot.apache.org/ logo
Join SlackCommunities
Powered by
# pinot-dev
  • m

    Mayank

    06/04/2020, 9:58 PM
    ack
  • s

    Subbu Subramaniam

    06/04/2020, 10:01 PM
    @User @User it failed the same way it seems https://travis-ci.org/github/apache/incubator-pinot/jobs/694797593
  • s

    Subbu Subramaniam

    06/04/2020, 10:02 PM
    Our daily build depends on this, so I hope we can get it fixed by today eve. I will start the daily buld job at 645 or so
  • m

    Mayank

    06/04/2020, 10:02 PM
    Scanning the log for errors
  • m

    Mayank

    06/04/2020, 10:02 PM
    Copy code
    WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
    WARNING: An illegal reflective access operation has occurred
    WARNING: Illegal reflective access by org.apache.pinot.spi.plugin.PluginClassLoader (file:/home/travis/build/apache/incubator-pinot/pinot-distribution/target/apache-pinot-incubating-0.4.0-dev-17411-bin/apache-pinot-incubating-0.4.0-dev-17411-bin/lib/pinot-all-0.4.0-dev-17411-jar-with-dependencies.jar) to method java.net.URLClassLoader.addURL(java.net.URL)
    WARNING: Please consider reporting this to the maintainers of org.apache.pinot.spi.plugin.PluginClassLoader
  • m

    Mayank

    06/04/2020, 10:07 PM
    Copy code
    INFO: [HttpServer-2] Started.
    ./.travis/.travis_quickstart.sh: line 102: 11571 Killed                  bin/quick-start-streaming.sh  (wd: ~/build/apache/incubator-pinot/pinot-distribution/target/apache-pinot-incubating-0.4.0-dev-17411-bin/apache-pinot-incubating-0.4.0-dev-17411-bin)
  • m

    Mayank

    06/04/2020, 10:11 PM
    ^^ are in passing builds as well
  • m

    Mayank

    06/04/2020, 10:32 PM
    Seems to be that
    curl -X POST --header 'Accept: application/json' -d '{"sql":"select count(*) from airlineStats limit 1","trace":false}' <http://localhost:8000/query/sql>
    returns nothing, which then leads to the script failing on syntax error
  • m

    Mayank

    06/04/2020, 10:33 PM
    But given that all components are started correctly (as per the log), It is not clear why this does not return anything (not even 0)
  • s

    Subbu Subramaniam

    06/04/2020, 10:33 PM
    did u bring up quickstart and running this?
  • m

    Mayank

    06/04/2020, 10:33 PM
    I am sure it will pass locally
  • s

    Subbu Subramaniam

    06/04/2020, 10:33 PM
    it does not look like a travis bug..
  • m

    Mayank

    06/04/2020, 10:34 PM
    given that it passes/fails intermittently
  • m

    Mayank

    06/04/2020, 10:34 PM
    I am suspecting issue in travis
  • s

    Subbu Subramaniam

    06/04/2020, 10:36 PM
    maybe @User can help, he set up the travis jobs..
  • m

    Mayank

    06/04/2020, 10:42 PM
    Ok, HybridQuickStart worked well on local. However RealtimeQuickStart that is used by above is stuck:
    Copy code
    2020/06/04 15:40:03.676 INFO [GroupCoordinator] [kafka-request-handler-2] [GroupCoordinator 0]: Assignment received from leader for group meetupRsvp_REALTIME_1591310399192_0 for generation 1
    2020/06/04 15:40:03.700 INFO [LeaderEpochFileCache] [kafka-request-handler-2] Updated PartitionLeaderEpoch. New: {epoch:0, offset:0}, Current: {epoch:-1, offset:-1} for Partition: __consumer_offsets-0. Cache now contains 0 entries.
    2020/06/04 15:40:03.719 INFO [AbstractCoordinator] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Successfully joined group with generation 1
    2020/06/04 15:40:03.721 INFO [ConsumerCoordinator] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Setting newly assigned partitions [meetupRSVPEvents-2, meetupRSVPEvents-3, meetupRSVPEvents-0, meetupRSVPEvents-1, meetupRSVPEvents-6, meetupRSVPEvents-7, meetupRSVPEvents-4, meetupRSVPEvents-5, meetupRSVPEvents-8, meetupRSVPEvents-9]
    2020/06/04 15:40:03.732 INFO [Fetcher] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Resetting offset for partition meetupRSVPEvents-2 to offset 0.
    2020/06/04 15:40:03.732 INFO [Fetcher] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Resetting offset for partition meetupRSVPEvents-3 to offset 0.
    2020/06/04 15:40:03.732 INFO [Fetcher] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Resetting offset for partition meetupRSVPEvents-0 to offset 0.
    2020/06/04 15:40:03.732 INFO [Fetcher] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Resetting offset for partition meetupRSVPEvents-1 to offset 0.
    2020/06/04 15:40:03.732 INFO [Fetcher] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Resetting offset for partition meetupRSVPEvents-6 to offset 0.
    2020/06/04 15:40:03.732 INFO [Fetcher] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Resetting offset for partition meetupRSVPEvents-7 to offset 0.
    2020/06/04 15:40:03.732 INFO [Fetcher] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Resetting offset for partition meetupRSVPEvents-4 to offset 0.
    2020/06/04 15:40:03.732 INFO [Fetcher] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Resetting offset for partition meetupRSVPEvents-5 to offset 0.
    2020/06/04 15:40:03.732 INFO [Fetcher] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Resetting offset for partition meetupRSVPEvents-8 to offset 0.
    2020/06/04 15:40:03.732 INFO [Fetcher] [Thread-37] [Consumer clientId=consumer-1, groupId=meetupRsvp_REALTIME_1591310399192_0] Resetting offset for partition meetupRSVPEvents-9 to offset 0.
    2020/06/04 15:41:02.223 INFO [Reflections] [grizzly-http-server-7] Reflections took 848 ms to scan 18 urls, producing 22 keys and 1367 values
  • j

    Jack

    06/04/2020, 10:46 PM
    I got this when I ran `quick-start-streaming.sh`:
    Copy code
    $ curl -X POST --header 'Accept: application/json' -d '{"sql":"select count(*) from airlineStats limit 1","trace":false}' <http://localhost:8000/query/sql> | trjson -p
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100   497  100   432  100    65   3375    507 --:--:-- --:--:-- --:--:--  3882
    {
      "exceptions": [
        {
          "errorCode": 410,
          "message": "BrokerResourceMissingError"
        }
      ],
      "minConsumingFreshnessTimeMs": 0,
      "numConsumingSegmentsQueried": 0,
      "numDocsScanned": 0,
      "numEntriesScannedInFilter": 0,
      "numEntriesScannedPostFilter": 0,
      "numGroupsLimitReached": false,
      "numSegmentsMatched": 0,
      "numSegmentsProcessed": 0,
      "numSegmentsQueried": 0,
      "numServersQueried": 0,
      "numServersResponded": 0,
      "segmentStatistics": [],
      "timeUsedMs": 0,
      "totalDocs": 0,
      "traceInfo": {}
    }
  • m

    Mayank

    06/04/2020, 10:47 PM
    That seems to be because it is stuck as I pasted above
  • m

    Mayank

    06/04/2020, 10:47 PM
    any realtime changes recently?
  • j

    Jack

    06/04/2020, 10:50 PM
    Wait, the table name is different
  • j

    Jack

    06/04/2020, 10:50 PM
    Copy code
    $ curl '<http://localhost:9000/sql>' --data-binary '{"sql":"select * from meetupRsvp limit 10","trace":false}' 
    {"resultTable":{"dataSchema":{"columnDataTypes":["STRING","STRING","LONG","STRING","STRING","LONG","STRING","LONG","INT","STRING"],"columnNames":["event_id","event_name","event_time","group_city","group_country","group_id","group_name","mtime","rsvp_count","venue_name"]},"rows":[["271034540","APIs 360, Gobierno de APIs. Los inicios. (ONLINE)",1591716600000,"Madrid","es",14950132,"APIAddicts Madrid",1591310557024,1,"Online event"],["270904119","AFTER WORK en Terraza  Atenas, Ven a conocer gente nueva.",1591376400000,"Madrid","es",23024216,"Diviértete",1591310558876,1,"Terraza Atenas"],["271069392","BREATHE AWAY YOUR ANXIETY: Online Meditation & Intro to Happiness Program",1591453800000,"New York","us",383040,"\"Art of Living\" NYC- Making Life a Celebration",1591310559114,1,"Online event"],["270914317","Business Analysis Basics for Business Intelligence Analysts ",1591743600000,"Neenah","us",21220336,"Fox Valley Business Intelligence And Analytics",1591310559220,1,"Online event"],["271069392","BREATHE AWAY YOUR ANXIETY: Online Meditation & Intro to Happiness Program",1591453800000,"New York","us",383040,"\"Art of Living\" NYC- Making Life a Celebration",1591310559000,1,"Online event"],["270912919","Cómo vender online",1591446600000,"Neuquén","ar",19760661,"GDG Neuquén",1591310561213,1,"Online event"],["271041784","National Trails Day Hike ~ Wa'ahila / Mt Olympus",1591465500000,"Honolulu","us",11084982,"The Hawai'i Hiking Group",1591310562548,1,"Wa'ahila Ridge State Recreation Area"],["271082438","Kristin Jackvony,  Get Organized for Postman Testing Success",1591819200000,"Boston","us",19054860,"Ministry of Testing: Boston",1591310563115,1,"Mabl"],["271037888","El ecosistema Laravel",1591815600000,"México City","mx",18259219,"PHP Mexico",1591310564870,1,"Online event"],["270999844","24 Hours of UX - Keynote with Indi Young and Brian Sullivan",1591815600000,"Dallas","us",4222412,"Dallas User Experience Group",1591310566703,1,"Online event"]]},"exceptions":[],"numServersQueried":1,"numServersResponded":1,"numSegmentsQueried":1,"numSegmentsProcessed":1,"numSegmentsMatched":1,"numConsumingSegmentsQueried":1,"numDocsScanned":10,"numEntriesScannedInFilter":0,"numEntriesScannedPostFilter":100,"numGroupsLimitReached":false,"totalDocs":389,"timeUsedMs":46,"segmentStatistics":[],"traceInfo":{},"minConsumingFreshnessTimeMs":1591310993491}
  • m

    Mayank

    06/04/2020, 10:50 PM
    meetUp vs airline?
  • j

    Jack

    06/04/2020, 10:51 PM
    yeah,
    meetUp
    is for realtime script
  • m

    Mayank

    06/04/2020, 10:51 PM
    let me see what changed
  • m

    Mayank

    06/05/2020, 1:49 AM
    So far it seems that the 30s wait may not be sufficient for hybrid table to be ready.
  • m

    Mayank

    06/05/2020, 1:49 AM
    From the failed runs, the queries are launched before I see that the table is created.
  • m

    Mayank

    06/05/2020, 1:54 AM
    Anyone know of any recent changes that would cause the cluster to slow down such that 30s is not good enough?
  • j

    Jack

    06/05/2020, 1:56 AM
    Plus, we encountered this message when running the following query:
    Copy code
    $ curl '<http://localhost:9000/sql>' \
    >   -H 'Connection: keep-alive' \
    >   -H 'Accept: text/plain, */*; q=0.01' \
    >   -H 'X-Requested-With: XMLHttpRequest' \
    >   -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36' \
    >   -H 'Content-Type: application/json; charset=UTF-8' \
    >   -H 'Origin: <http://localhost:9000>' \
    >   -H 'Sec-Fetch-Site: same-origin' \
    >   -H 'Sec-Fetch-Mode: cors' \
    >   -H 'Sec-Fetch-Dest: empty' \
    >   -H 'Referer: <http://localhost:9000/query>' \
    >   -H 'Accept-Language: en-US,en;q=0.9' \
    >   --data-binary '{"sql":"select count(*) from meetupRsvp limit 10","trace":false}' \
    >   --compressed
    {"code":500,"error":"Could not initialize class org.apache.pinot.common.function.FunctionRegistry"}
    which may be due to https://github.com/apache/incubator-pinot/pull/5440
  • m

    Mayank

    06/05/2020, 2:05 AM
    @User is the above issue consistently reproducible? What happens if you wait long enough before you manually issue a query?
  • m

    Mayank

    06/05/2020, 2:06 AM
    My guess is that the cluster was not setup and proper inits didn’t happen
1...121314...30Latest