Elon
07/27/2020, 5:00 PMNeha Pawar
Elon
07/27/2020, 5:08 PMNeha Pawar
Elon
07/27/2020, 5:15 PMNeha Pawar
RealtimeSegmentValidationManager
Elon
07/27/2020, 5:16 PMERROR
state from external view. Deleted them. Could that be why the segment validator didn't run?Neha Pawar
Elon
07/27/2020, 5:49 PMNeha Pawar
Elon
07/27/2020, 5:50 PMNeha Pawar
Elon
07/27/2020, 6:07 PMtextPayload: "java.lang.RuntimeException: Mismatching schema/table config for oas_integration_operation_event_REALTIME
at org.apache.pinot.core.data.manager.realtime.RealtimeTableDataManager.addSegment(RealtimeTableDataManager.java:238) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.server.starter.helix.HelixInstanceDataManager.addRealtimeSegment(HelixInstanceDataManager.java:132) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeOnlineFromOffline(SegmentOnlineOfflineStateModelFactory.java:164) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeConsumingFromOffline(SegmentOnlineOfflineStateModelFactory.java:88) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at sun.reflect.GeneratedMethodAccessor110.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_262]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_262]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:404) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:331) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49) ~[pinot-all-0.4.0-SNAPSHOT-jar-with-dependencies.jar:0.4.0-SNAPSHOT-9e7da0349baa23dd02987a3142818dbc6a144fbe]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_262]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_262]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_262]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_262]
"
Neha Pawar
Elon
07/27/2020, 6:34 PMNeha Pawar
Cannot convert from incoming field spec:< field name: operation_ts, data type: LONG, time type: MILLISECONDS, time unit size: 1, time format: EPOCH > to outgoing field spec:< field name: operation_ts, data type: LONG, time type: SECONDS, time unit size: 1, time format: EPOCH > if name is the same
Elon
07/27/2020, 6:39 PMNeha Pawar
Elon
07/27/2020, 6:40 PMNeha Pawar
Elon
07/27/2020, 6:48 PMPUT
with the new schema?Neha Pawar
Elon
07/27/2020, 6:57 PMNeha Pawar
I create a new table with the correct table schema/realtime config
- if this is possible, that’s great.
though i didnt understand this - load the data into offline segments
Elon
07/27/2020, 7:01 PMNeha Pawar
Elon
07/27/2020, 7:05 PMNeha Pawar
Elon
07/27/2020, 7:10 PMNeha Pawar
Elon
07/27/2020, 7:11 PMNeha Pawar
Elon
07/27/2020, 7:14 PMNeha Pawar
We just use the uploaddownload client and push them to gcs
who/how queries these segments?Elon
07/27/2020, 7:14 PMNeha Pawar
Elon
07/27/2020, 7:18 PM