Elon
09/04/2020, 8:52 PM_OFFLINE
table. Then if we upload a more recent segment that segment becomes available but the newest uploaded segment is not available. Is there any way to set a segment as ready to be served or something similar?srisudha
09/08/2020, 1:16 PMElon
09/09/2020, 6:30 PMNeha Pawar
Yash Agarwal
09/09/2020, 6:32 PMRebalancing table: guestslslitm3years_OFFLINE with minAvailableReplicas: 1, bestEfforts: false
Found ERROR instance: Server_10.59.98.103_8098 for segment: guestslslitm3years_2018-01-01_2018-01-01_3, table: guestslslitm3years_OFFLINE
Caught exception while waiting for ExternalView to converge for table: guestslslitm3years_OFFLINE, aborting the rebalance
java.lang.IllegalStateException: Found segments in ERROR state
at org.apache.pinot.controller.helix.core.rebalance.TableRebalancer.isExternalViewConverged(TableRebalancer.java:538) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-701ffcbd5be5f39e91cea9a0297c4e8b0a7d9343]
at org.apache.pinot.controller.helix.core.rebalance.TableRebalancer.waitForExternalViewToConverge(TableRebalancer.java:480) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-701ffcbd5be5f39e91cea9a0297c4e8b0a7d9343]
at org.apache.pinot.controller.helix.core.rebalance.TableRebalancer.rebalance(TableRebalancer.java:344) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-701ffcbd5be5f39e91cea9a0297c4e8b0a7d9343]
at org.apache.pinot.controller.helix.core.PinotHelixResourceManager.rebalanceTable(PinotHelixResourceManager.java:2128) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-701ffcbd5be5f39e91cea9a0297c4e8b0a7d9343]
at org.apache.pinot.controller.api.resources.PinotTableRestletResource.lambda$rebalance$0(PinotTableRestletResource.java:530) ~[pinot-all-0.5.0-SNAPSHOT-jar-with-dependencies.jar:0.5.0-SNAPSHOT-701ffcbd5be5f39e91cea9a0297c4e8b0a7d9343]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_265]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_265]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_265]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_265]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_265]
Yash Agarwal
09/09/2020, 7:27 PMTim Chan
09/11/2020, 3:18 PMTim Chan
09/11/2020, 6:35 PM2020/09/11 18:26:00.615 WARN [LLRealtimeSegmentDataManager_sesame_person_features__4__6__20200911T1716Z] [sesame_person_features__1__7__20200911T1716Z] Commit failed with response {"isSplitCommitType":false,"streamPartitionMsgOffset":null,"buildTimeSec":-1,"status":"FAILED","offset":-1}
Ankit
09/14/2020, 7:08 AMYash Agarwal
09/15/2020, 1:44 PM{
"code": 409,
"error": "Failed to drop instance Broker_172.17.0.2_8099 - Instance Broker_172.17.0.2_8099 exists in ideal state for brokerResource"
}
How do i remove the instance from ideal states and also drop it ?Shen Wan
09/15/2020, 8:56 PMPradeep
09/16/2020, 1:40 AMShen Wan
09/16/2020, 3:39 PMBuchi Reddy
09/20/2020, 3:53 AMPUT
of the config and PUT also succeeded with http 200
but when I get back the tableConfig again, I don't see the STAR tree index config anymore.Yash Agarwal
09/21/2020, 8:04 PMJackie
09/23/2020, 5:32 PMPinotInstanceAssignmentRestletResource
for that. These APIs are detailed fine tuning APIs so we don't have them documented yet. Will enhance the documents. Let me know if you need more help understanding the APIsBuchi Reddy
09/25/2020, 12:46 AMTim Chan
09/29/2020, 5:11 PMPradeep
09/29/2020, 11:14 PMPradeep
09/30/2020, 12:12 AMPradeep
10/01/2020, 12:47 AMDan Hill
10/01/2020, 1:11 AMdelete
and insert
operations for group by results.
Here's a simplified version of my Pinot table to help illustrate my use case: (dimensions) utc_date, seller_id, content_id, (metrics) sum_clicks.Subbu Subramaniam
10/01/2020, 5:45 PMPradeep
10/01/2020, 7:55 PMElon
10/02/2020, 7:46 PMTim Chan
10/02/2020, 7:58 PMSubbu Subramaniam
10/02/2020, 8:28 PMDan Hill
10/03/2020, 5:57 AMmvn checkstyle:check -X
. It looks like the <http://repository.apache.org|repository.apache.org>
snapshots has not been updated for 0.6.0
. Also, I'm using openjdk 13.0.2 2020-01-14
. Should I change this?
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Failure to find org.apache.pinot:pinot-spi:jar:0.6.0-SNAPSHOT in <https://repository.apache.org/snapshots> was cached in the local repository, resolution will not be reattempted until the update interval of apache.snapshots has elapsed or updates are forced
Dan Hill
10/07/2020, 1:44 AMCaused by: groovy.lang.MissingPropertyException: No such property: PINOT_DATE for class: SimpleTemplateScript1
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53) ~[pinot-all-0.6.0-SNAPSHOT-jar-with-dependencies.jar:0.6.0-SNAPSHOT-8782e47b45c945cd02ccb8f06597b0ffa66a735a]
apiVersion: v1
kind: ConfigMap
metadata:
name: pinot-s3-data-config
data:
local_batch_job_spec.yaml: |-
executionFrameworkSpec:
...
inputDirURI: '<s3://promoted-event-logs/offlinepinot/dt=$JOB_DATE/>'
...
---
apiVersion: batch/v1
kind: Job
metadata:
name: pinot-populate-from-s3
spec:
template:
spec:
containers:
- name: pinot-populate-from-s3
image: apachepinot/pinot:0.6.0-SNAPSHOT-8782e47b4-20201006-jdk8
env:
- name: JOB_DATE
value: $(date -u +"%Y-%m-%d")
args: [
"LaunchDataIngestionJob",
"-jobSpecFile",
"/home/pinot/pinot-config/local_batch_job_spec.yaml"
]
volumeMounts:
- name: pinot-s3-data-config
mountPath: /home/pinot/pinot-config
restartPolicy: OnFailure
volumes:
- name: pinot-s3-data-config
configMap:
name: pinot-s3-data-config
backoffLimit: 100
Pradeep
10/14/2020, 12:09 AM