https://pinot.apache.org/ logo
Join Slack
Powered by
# pinot-dev
  • k
    Copy code
    QueryExecutorTest
  • k
    Copy code
    PropertiesConfiguration queryExecutorConfig = new PropertiesConfiguration();
queryExecutorConfig.setDelimiterParsingDisabled(false);
queryExecutorConfig.load(new File(resourceUrl.getFile()));
_queryExecutor = new ServerQueryExecutorV1Impl();
_queryExecutor.init(new PinotConfiguration(queryExecutorConfig), instanceDataManager, _serverMetrics);
  • a
    Thank you!
  • a
    Here is a dumb question -- do we allow only a single column in DISTINCTCOUNT?
  • m
    Yes we do.
  • a
    Shouldn't multiple columns in Distinct Count be throwing a parser error then?
  • a
    Also, when using the dictionary, is our assertion that the query has only one column in its target list? It seems that we check only the first column for presence in the dictionary when deciding if the query can be planned only using the dictionary
  • a
    It also makes sense because we cannot maintain order amongst two dictionaries, but wanted to confirm
  • k
    I see that you can build Pinot using Java 8, via 
    mvn clean install -DskipTests -Pbin-dist -T 4 -Djdk.version=8
    . Is there any way to generate Eclipse projects that also use Java 8? Currently these project files are created with Java 11 as the target (compliance level, and generated .class files).
  • a
    Folks, here is a question. Why do we not represent Distinct as an aggregation? That will save us some special casing in the code and make the intermediate block format consistent
  • a
    Is there a reason why we have split it out?
  • j
    Modeling Distinct as aggregation limits the opportunity of optimizations. From query processing perspective, distinct works closer to selection, but with dedup. More context and performance numbers can be found here: https://github.com/apache/incubator-pinot/pull/6285
  • j
    The special casing in the intermediate block format is actually introduced by modeling it as aggregation. Ideally it should use the same format as selection query, but that will involve backward-incompatible change
  • a
    Thanks, that makes sense. I wanted to go in one direction, either have Distinct represented solely as an aggregation, or as a select query. In current form, it is a bit hard to fathom. Your explanation makes sense, thanks
  • k
    Hi, I had a query. In an instance where we loose configuration data in zookeeper in the pinot cluster, is there any recovery method in pinot? If the data in stored in deep store like S3, can we rebuild the cluster with data in the deep store?
  • k
    zookeeper can be configured to store the snapshot in deepstore like s3
    👍 1
  • k
    and you can revive restart zookeeper by providing that snapshot as input
  • k
    So if we save the configuration snapshot, the cluster can be revived. Is my assumption correct?
  • k
    Yes
  • m
    @User typically folks also maintain table config and schema under source control (say git) to track changes and use hooks to update the cluster via some hooks
    👍 1
  • k
    We have the table config maintained in git. What sort of hooks are we speaking of here? Any examples?
  • m
    I don’t have an example handy, but someway to ensure that git and Pinot cluster are always in sync, and also git push checks that updates are valid to be applied (without applying)
  • s
    Apache Pinot has recently been graduated to a top level project! We are currently working on the infra migration so some things have been changed. This can potentially break some of your scripts if those depend on fetching codes from git repo. • One of the most notable change is that our git repository became https://github.com/apache/pinot (previously, it was https://github.com/apache/pinot-incubator) • For contributors/committers, please change the remote url from the locally copied git repo
    Copy code
    git remote -v
upstream	git@github.com:apache/incubator-pinot.git (fetch)
upstream	git@github.com:apache/incubator-pinot.git (push)
->
upstream	git@github.com:apache/pinot.git (fetch)
upstream	git@github.com:apache/pinot.git (push)
    🎉 8
    🙌 5
  • x
    Trying to split tests to reduce ci time: https://github.com/apache/pinot/pull/7257
  • s
    @User is it possible to add the compatibility test also to run in parallel? We want to add it so that it does not block a merge, but the result is available to the reviewer. If this can be done, that will be great. @User is also working on it, so you may want to touch base with him, thanks
  • x
    Where are the compatibility test
  • s
  • k
    where do we use this?
  • m
    Hi Team, do we have any talks scheduled (or interest for talks) on Pinot scheduled for ApacheCon happening in Sept?
  • k
    Someone with admin rights for https://github.com/apache/pinot/ should eventually get rid of the “(Incubating)” bit from the description. I think that’s also being picked up by Github to set the window title.
1...242526...30Latest