https://pinot.apache.org/ logo
Join Slack
Powered by
# troubleshooting
  • x

    Xiang Fu

    04/28/2020, 11:25 PM
    got it
  • d

    Damiano

    04/28/2020, 11:25 PM
    ~37M are the docs, what about 75M?
  • x

    Xiang Fu

    04/28/2020, 11:25 PM
    that times the fields
  • d

    Damiano

    04/28/2020, 11:25 PM
    pardon? i do not understand
  • x

    Xiang Fu

    04/28/2020, 11:26 PM
    say if your query is
    select sum(met0) from myTable group by dim0
  • x

    Xiang Fu

    04/28/2020, 11:26 PM
    this query scans two columns
  • k

    Kishore G

    04/28/2020, 11:36 PM
    @Damiano If you are looking for faster latency for these queries, try enabling star-tree index
  • d

    Damiano

    04/28/2020, 11:41 PM
    @Kishore G if i enable it i should re-create the entire table right ?
  • k

    Kishore G

    04/28/2020, 11:42 PM
    Yes. There is a PR to create it on the fly without having to reload data
  • d

    Damiano

    04/28/2020, 11:43 PM
    not a big problem because as you told me when i run again the command:
  • d

    Damiano

    04/28/2020, 11:44 PM
    Copy code
    docker run --rm -ti 
    	--network=pinot-demo
    	-v /tmp/pinot-quick-start:/tmp/pinot-quick-start
    	--name pinot-data-ingestion-job
    	apachepinot/pinot:latest LaunchDataIngestionJob
    	-jobSpecFile /tmp/pinot-quick-start/docker-job-spec.yml
  • d

    Damiano

    04/28/2020, 11:44 PM
    the entire table will be overwritten again
  • k

    Kishore G

    04/28/2020, 11:44 PM
    Yes.
  • d

    Damiano

    04/28/2020, 11:45 PM
    @Kishore G i try to find on the doc how to activate that index type
  • k

    Kishore G

    04/28/2020, 11:48 PM
    @Jackie
  • k

    Kishore G

    04/28/2020, 11:51 PM
    Can you help Damiano ^^
  • d

    Damiano

    04/28/2020, 11:52 PM
    Thank you so much @Kishore G
  • d

    Damiano

    04/28/2020, 11:52 PM
    when i have created the table i used:
  • d

    Damiano

    04/28/2020, 11:52 PM
    Copy code
    {
      "tableName": "test",
      "tableType": "OFFLINE",
      "segmentsConfig": {
        "segmentPushType": "REFRESH",
        "segmentAssignmentStrategy": "BalanceNumSegmentAssignmentStrategy",
        "replication": "1"
      },
      "tenants": {},
      "tableIndexConfig": {
        "loadMode": "MMAP"
      },
      "metadata": {
        "customConfigs": {}
      }
    }
  • d

    Damiano

    04/28/2020, 11:53 PM
    so my query is not even using an index (inverted index)
  • k

    Kishore G

    04/28/2020, 11:54 PM
    Right
  • d

    Damiano

    04/28/2020, 11:54 PM
    am i wrong?
  • k

    Kishore G

    04/28/2020, 11:54 PM
    By default, no indexing is enabled
  • d

    Damiano

    04/28/2020, 11:54 PM
    wow, so basically it took 300 ms to group 37M docs so with an inverted index it should be faster
  • k

    Kishore G

    04/28/2020, 11:55 PM
    Not for your query as there is no filtering
  • k

    Kishore G

    04/28/2020, 11:56 PM
    If add where x=“...”
  • k

    Kishore G

    04/28/2020, 11:56 PM
    Then it will be faster
  • n

    Neha Pawar

    04/28/2020, 11:56 PM
    Here's a sample config for star-tree index : https://docs.pinot.apache.org/indexing#index-generation-configuration
  • d

    Damiano

    04/28/2020, 11:57 PM
    @Kishore G ah really? group by is not using index? so it is pointless to create an inverted index (https://docs.pinot.apache.org/indexing#inverted-index-only-available-with-dictionary-encoded-indexes) for the column i need to group
  • d

    Damiano

    04/28/2020, 11:57 PM
    @Neha Pawar thank you!
1...888990...166Latest