Yupeng Fu
04/13/2021, 11:37 PMXiang Fu
- JSON index
- Lookup-based join support
- Geospatial support
- TLS support for pinot connections
- Introduced new APIs for segment management and offline table push.
- Various performance optimizations, improvements and bug fixes.
Please also see the full release notes here: https://docs.pinot.apache.org/basics/releases/0.7.1
The release can be downloaded at https://pinot.apache.org/download
Additional resources -
Project website: https://pinot.apache.org
Getting started: https://docs.pinot.apache.org/getting-started
Pinot developer blogs: https://medium.com/apache-pinot-developer-blog
Intro to Pinot Video:
Twitter: https://twitter.com/ApachePinot
Meetup: https://www.meetup.com/apache-pinotGabriel Lucano
04/14/2021, 5:23 PMAaron Wishnick
04/14/2021, 8:11 PMApr 14, 2021 3:16:33 PM INFO: org.apache.parquet.hadoop.InternalParquetRecordReader: time spent so far 0% reading (1854 ms) and 99% processing (311813 ms)
Is there a setting to use more cores to process segments in parallel or anything like that?Neil Teng
04/15/2021, 5:14 PMMing Liang
04/15/2021, 10:54 PM[INFO] Reactor Summary for Pinot 0.8.0-SNAPSHOT:
[INFO]
[INFO] Pinot .............................................. SUCCESS [ 15.773 s]
[INFO] Pinot Service Provider Interface ................... SUCCESS [ 2.762 s]
[INFO] Pinot Segment Service Provider Interface ........... SUCCESS [ 1.093 s]
[INFO] Pinot Plugins ...................................... SUCCESS [ 4.414 s]
[INFO] Pinot Metrics ...................................... SUCCESS [ 0.145 s]
[INFO] Pinot Yammer Metrics ............................... SUCCESS [ 4.240 s]
[INFO] Pinot Common ....................................... SUCCESS [ 16.085 s]
[INFO] Pinot Input Format ................................. SUCCESS [ 0.993 s]
[INFO] Pinot Avro Base .................................... SUCCESS [ 1.109 s]
[INFO] Pinot Avro ......................................... SUCCESS [ 0.902 s]
[INFO] Pinot Csv .......................................... SUCCESS [ 0.354 s]
[INFO] Pinot JSON ......................................... SUCCESS [ 0.344 s]
[INFO] Pinot local segment implementations ................ SUCCESS [ 6.542 s]
[INFO] Pinot Core ......................................... SUCCESS [ 9.313 s]
[INFO] Pinot Server ....................................... SUCCESS [ 4.548 s]
[INFO] Pinot Segment Uploader ............................. SUCCESS [ 1.528 s]
[INFO] Pinot Segment Uploader Default ..................... SUCCESS [ 15.425 s]
[INFO] Pinot Controller ................................... SUCCESS [ 50.838 s]
[INFO] Pinot Broker ....................................... SUCCESS [ 4.736 s]
[INFO] Pinot Clients ...................................... SUCCESS [ 0.117 s]
[INFO] Pinot Java Client .................................. SUCCESS [ 0.474 s]
[INFO] Pinot JDBC Client .................................. SUCCESS [ 0.555 s]
[INFO] Pinot Batch Ingestion .............................. SUCCESS [ 1.501 s]
[INFO] Pinot Batch Ingestion Common ....................... SUCCESS [ 0.393 s]
[INFO] Pinot Minion ....................................... SUCCESS [ 1.718 s]
[INFO] Pinot Confluent Avro ............................... FAILURE [ 0.586 s]
[INFO] Pinot ORC .......................................... SKIPPED
[INFO] Pinot Parquet ...................................... SKIPPED
[INFO] Pinot Thrift ....................................... SKIPPED
[INFO] Pinot Protocol Buffers ............................. SKIPPED
[INFO] Pluggable Pinot file system ........................ SKIPPED
[INFO] Pinot Azure Data Lake Storage ...................... SKIPPED
[INFO] Pinot Hadoop Filesystem ............................ SKIPPED
[INFO] Pinot Google Cloud Storage ......................... SKIPPED
[INFO] Pinot Amazon S3 .................................... SKIPPED
[INFO] Pinot Batch Ingestion for Spark .................... SKIPPED
[INFO] Pinot Batch Ingestion for Hadoop ................... SKIPPED
[INFO] Pinot Batch Ingestion Standalone ................... SKIPPED
[INFO] Pinot Batch Ingestion .............................. SKIPPED
[INFO] Pinot Ingestion Common ............................. SKIPPED
[INFO] Pinot Hadoop ....................................... SKIPPED
[INFO] Pinot Spark ........................................ SKIPPED
[INFO] Pinot Stream Ingestion ............................. SKIPPED
[INFO] Pinot Kafka Base ................................... SKIPPED
[INFO] Pinot Kafka 0.9 .................................... SKIPPED
[INFO] Pinot Kafka 2.0 .................................... SKIPPED
[INFO] Pinot Minion Tasks ................................. SKIPPED
[INFO] Pinot Minion Built-In Tasks ........................ SKIPPED
[INFO] Pinot Segment Writer ............................... SKIPPED
[INFO] Pinot Segment Writer File Based .................... SKIPPED
[INFO] Pinot Tools ........................................ SKIPPED
[INFO] Pinot Integration Tests ............................ SKIPPED
[INFO] Pinot Perf ......................................... SKIPPED
[INFO] Pinot Distribution ................................. SKIPPED
[INFO] Pinot Connectors ................................... SKIPPED
[INFO] Pinot Spark Connector .............................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:26 min
[INFO] Finished at: 2021-04-15T15:49:12-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project pinot-confluent-avro: Could not resolve dependencies for project org.apache.pinot:pinot-confluent-avro:jar:0.8.0-SNAPSHOT: Failed to collect dependencies at io.confluent:kafka-schema-registry-client:jar:5.3.1: Failed to read artifact descriptor for io.confluent:kafka-schema-registry-client:jar:5.3.1: Could not transfer artifact io.confluent:kafka-schema-registry-client:pom:5.3.1 from/to maven-default-http-blocker (<http://0.0.0.0/>): Blocked mirror for repositories: [confluent (<http://packages.confluent.io/maven/>, default, releases+snapshots)] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] <http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException>
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :pinot-confluent-avro
➜ incubator-pinot git:(master)
Badri Tripathy
04/15/2021, 10:55 PMGabriel Lucano
04/15/2021, 11:17 PMMohan Pandiyan
04/16/2021, 8:57 PMMatt
04/17/2021, 12:27 AMDileep Reddy
04/18/2021, 4:06 PMCharles
04/19/2021, 2:00 AMCharles
04/19/2021, 2:04 AMJohn Knapp
04/20/2021, 3:46 AMHector D
04/22/2021, 4:53 PMS Das
04/22/2021, 5:17 PMJosh Highley
04/22/2021, 5:25 PMYupeng Fu
04/22/2021, 9:17 PMPedro Silva
04/23/2021, 10:16 AMAkash
04/23/2021, 12:17 PMPedro Silva
04/23/2021, 2:00 PMArun Lakshman Ravichandran
04/24/2021, 11:50 AMErjan G.
04/26/2021, 12:56 PMJonathan Meyer
04/26/2021, 4:58 PMSELECT SUM(value) FROM values WHERE timeString BETWEEN '2021-01-01' AND '2021-01-08'
(ex: rolling week)
Can some configuration of StarTree index precompute this sort of query ? (or even part of it)
[I know this looks like a TSDB use case, but still, I'm hopeful 😄]Erjan G.
04/26/2021, 5:14 PMErjan G.
04/26/2021, 5:14 PMAmine Chraïbi
04/27/2021, 2:15 PMkauts shukla
04/28/2021, 6:36 AM{
"tenants": {
"DefaultTenant": [
{
"port": 8099,
"host": "Broker_1",
"instanceName": "Broker_1"
},
{
"port": 8099,
"host": "Broker_1",
"instanceName": "Broker_1"
}
]
},
"tables": {}
}
Pedro Silva
04/28/2021, 9:23 AM{
"schemaName": "hitexecutionview",
"dimensionFieldSpecs": [
{
"name": "id",
"dataType": "STRING"
},
{
"name": "jobId",
"dataType": "STRING"
},
{
"name": "crowdMemberId",
"dataType": "STRING"
},
{
"name": "projectId",
"dataType": "STRING"
},
{
"name": "result",
"dataTYpe": "STRING"
}
],
"dateTimeFieldSpecs": [
{
"name": "timestamp",
"dataType": "LONG",
"format": "1:MILLISECONDS:EPOCH",
"granularity": "1:MILLISECONDS"
}
],
"primaryKeyColumns": [
"id"
]
}
And the following kafka message:
{
"id": "19281-3123n1283-12312-312",
"jobId": "245d-2334-fs33-23f4",
"crowdMemberId": "xxxxxxxxxx",
"projectId": "49mf-f39f-25v2-989m",
"timestamp": "1238648237"
}
The field result is not passed, will pinot assume a null value?
What happens if there are computed fields based on this non-existing result?Xiang Fu