Abhishek Kedia
01/04/2022, 7:48 AMxtrntr
01/04/2022, 11:41 AMstorage
quota in a dimension table refer to the uncompressed size in memory, or the compressed segment size?Yash Agarwal
01/04/2022, 12:09 PMMap
01/04/2022, 1:35 PMt1
with 200+ columns A
, B
, `C1`…`C200`. When I do select max(A) from table group by B'
, I get the max A for each B. Is there to get the corresponding `C1`…`C200` for each max(A) per B? That is to say, how to achieve `select max(A), C1, ..., C200 from table group by B`without aggregating on C
or grouping by C
?Sergii Balganbaiev
01/04/2022, 2:23 PMThe updated index config won't be picked up unless we invoke the reload API.
I have added new index(inverted
) to table config and call reload API(with forceDownload
flag) but as I see index is not created(I checked in reload status
and even in metadata.properties
file of particular segment).
But after I re-ingest data(with spark job) to segment-store then index is created. All these operations are done for offline
table
So the question is: is it necessary to comply some additional conditions to make reload API working? Maybe, I missed somethingPriyank Bagrecha
01/04/2022, 11:36 PMSUM
in future?Priyank Bagrecha
01/04/2022, 11:38 PMAshish
01/04/2022, 11:39 PMDiana Arnos
01/05/2022, 2:44 PMDiana Arnos
01/06/2022, 1:59 PMEvan Galpin
01/06/2022, 5:56 PMJSON_MATCH
? Ex. Modifying example query from json-index docs[1]:
{
"name": "adam",
"age": 30,
"country": "us",
"addresses": [
{
"number": 112,
"street": "main st",
"country": "us"
},
{
"number": 2,
"street": "second st",
"country": "us"
},
{
"number": 3,
"street": "third st",
"country": "ca"
}
]
}
SELECT ...
FROM mytable
WHERE JSON_MATCH(person, '"$.addresses[*].number">2')
I would expect that this adam
doc/row would match. But I’m not sure if range queries are supported. Thanks!
https://docs.pinot.apache.org/basics/indexing/json-index#chained-key-lookupabhinav wagle
01/06/2022, 6:57 PMSheetal
01/06/2022, 10:52 PMWeixiang Sun
01/07/2022, 1:11 AMxtrntr
01/07/2022, 3:29 AMReported Size
in the table UI in bytes?Mohemmad Zaid Khan
01/07/2022, 5:25 AM0.9.3
and Hadoop Version 3.3.1
- Is there a compatibility matrix which I can refer?Dimitar Apostolov
01/07/2022, 4:34 PMChandan Choudhary
01/10/2022, 5:01 AMMohemmad Zaid Khan
01/10/2022, 9:22 AMmain
method not found exception and I checked the class LaunchDataIngestionJobCommand
code, it does not have a main method.
Exception in thread "main" java.lang.NoSuchMethodException: org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand.main([Ljava.lang.String;)
at java.base/java.lang.Class.getMethod(Class.java:2108)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:42)
at <http://org.apache.spark.deploy.SparkSubmit.org|org.apache.spark.deploy.SparkSubmit.org>$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Configuration -
#!/usr/bin/env bash
export HADOOP_CONF_DIR=/Users/$USER/servers/hadoop-2.7.1/etc/hadoop
export SPARK_DIST_CLASSPATH=$(/Users/$USER/servers/hadoop-2.7.1/bin/hadoop classpath)
export PINOT_VERSION=0.9.3
export PINOT_DISTRIBUTION_DIR="/Users/$USER/servers/pinot-${PINOT_VERSION}"
export SPARK_VERSION=3.2.0
export SPARK_HOME="/Users/$USER/servers/spark-${SPARK_VERSION}-bin-without-hadoop"
${SPARK_HOME}/bin/spark-submit \
--class org.apache.pinot.tools.admin.command.LaunchDataIngestionJobCommand \
--master "local[2]" \
--deploy-mode client \
--conf "spark.driver.extraJavaOptions=-Dplugins.dir=${PINOT_DISTRIBUTION_DIR}/plugins -Dlog4j2.configurationFile=${PINOT_DISTRIBUTION_DIR}/conf/pinot-ingestion-job-log4j2.xml" \
--conf "spark.driver.extraClassPath=${PINOT_DISTRIBUTION_DIR}/lib/pinot-all-${PINOT_VERSION}-jar-with-dependencies.jar" local://${PINOT_DISTRIBUTION_DIR}/lib/pinot-all-${PINOT_VERSION}-jar-with-dependencies.jar \
-jobSpecFile ${PINOT_DISTRIBUTION_DIR}/examples/batch/airlineStats/sparkIngestionJobSpec.yaml
Karin Wolok
Karin Wolok
abhinav wagle
01/10/2022, 8:46 PMorg.apache.hadoop.mapred.TextInputFormat
?Chengxuan Wang
01/11/2022, 2:24 PMJonathan Locke
01/11/2022, 6:33 PMbalci
01/11/2022, 10:07 PMDiogo Baeder
01/11/2022, 10:38 PMMap
01/12/2022, 1:38 AMAbhishek Tomar
01/12/2022, 11:13 AMBranden Colen
01/12/2022, 4:29 PMAbhishek Tomar
01/12/2022, 8:16 PM