Channels
test-channel-bot
pw-pinot
debug_upsert
feat-presto-connector
pinot-helix
troubleshooting
pinot-sketches
resources
roadmap
discuss-validation
minion-improvements
community
config-tuner
test-channel
fix-numerical-predicate
kinesis_help
pinot-perf-tuning
feat-partial-upsert
pinot_website_improvement_suggestions
segment-write-api
releases
devrel
pinot-trino
pinot-k8s-operator
release-certifier
udf-type-matching
transform-functions
release060
latency-during-segment-commit
feat-text-search
pinot-realtime-table-rebalance
inconsistent-segment
feat-rt-seg-complete
pinot-power-bi
flink-pinot-connector
pinotadls
twitter
minion-star-tree
pinot-rack-awareness
jobs
fix_llc_segment_upload
metrics-plugin-impl
linen_dev
complex-type-support
pinot-s3
feat-geo-spatial-index
feat-compound-types
v2_engine_beta_feedback
inconsistent-perf
metadata-push-api
feat-pravega-connector
enable-generic-offsets
announcements
feat-upsert
multiple_streams
feat-better-schema-evolution
fraud
onboarding
docs
aggregators
query-latency
s3-multiple-buckets
presto-pinot-connector
multi-region-setup
apa-16824
events
segment-cold-storage
odsc-europe-2022-workshop
bangalore-meetups
new-office-space
lp-pinot-poc
time-based-segment-pruner
group-by-refactor
pql-sql-regression
random
custom-aggregators
order-by
pinot-website
qps-metric
product-launch
dhill-date-seg
thirdeye-pinot
introductions
pinot-dev
getting-started
pinot-docsrus
daily-digest
general
Powered by
#troubleshooting
Title
l
lâm nguyễn hoàng
12/28/2020, 7:09 PM
{ "REALTIME": { "tableName": "ERP_ERP_PM_INPUTVOUCHERDETAIL_REALTIME", "tableType": "REALTIME", "segmentsConfig": { "schemaName": "ERP_ERP_PM_INPUTVOUCHERDETAIL", "timeType": "MILLISECONDS", "retentionTimeUnit": "DAYS", "retentionTimeValue": "2", "segmentPushFrequency": "DAILY", "segmentPushType": "APPEND", "timeColumnName": "WARRANTYDATE", "replication": "4", "replicasPerPartition": "4" }, "tenants": { "broker": "DefaultTenant", "server": "inventory" }, "tableIndexConfig": { "streamConfigs": { "streamType": "kafka", "stream.kafka.consumer.type": "lowlevel", "stream.kafka.topic.name": "
PINOT.ERP.ERP.PM
_INPUTVOUCHERDETAIL", "stream.kafka.table.tablename": "
ERP.PM
_INPUTVOUCHERDETAIL", "stream.kafka.table.part.pattern": "_[0-9]{2}_[0-9]{4}", "stream.kafka.cdc.format": "CDC", "stream.kafka.decoder.class.name": "com.mwg.pinot.realtime.KafkaCDCMessageDecoder", "stream.kafka.consumer.factory.class.name": "com.mwg.pinot.realtime.KafkaCDCConsumerFactory", "notify.line.token": "aEZ1nmvqGjhDkuKMO0ghZaAFAVyvoszUjFYJG4Vobc9", "stream.kafka.broker.list": "datastore-broker01-kafka-ovm-6-76
9092,datastore broker02 kafka ovm 6 77
9093,datastore-broker03-kafka-ovm-6-78
9094,datastore broker04 kafka ovm 6 134
9095,datastore-broker05-kafka-ovm-6-135
9096,datastore broker06 kafka ovm 6 136
9097,datastore-broker07-kafka-ovm-6-120
9098,datastore broker08 kafka ovm 6 121
9099,datastore-broker09-kafka-ovm-6-122
9101,datastore broker10 kafka ovm 6 123
9102", "stream.kafka.consumer.prop.auto.offset.reset": "smallest", "realtime.segment.flush.threshold.rows": "0", "realtime.segment.flush.threshold.time": "900000", "realtime.segment.flush.threshold.segment.size": "50M", "group.id": "
ERP.PM
_INPUTVOUCHERDETAIL2-PINOT_INGESTION", "max.partition.fetch.bytes": "167772160", "receive.buffer.bytes": "67108864", "isolation.level": "read_committed", "max.poll.records": "5000" }, "autoGeneratedInvertedIndex": false, "createInvertedIndexDuringSegmentGeneration": false, "sortedColumn": [ "INPUTVOUCHERDETAILID" ], "loadMode": "MMAP", "enableDefaultStarTree": false, "enableDynamicStarTreeCreation": false, "aggregateMetrics": false, "nullHandlingEnabled": false }, "metadata": { "customConfigs": {} }, "routing": { "instanceSelectorType": "strictReplicaGroup" }, "upsertConfig": { "mode": "FULL" } } }
m
Mayank
12/28/2020, 7:12 PM
Why do you not expect it to change? Do you not have real-time data consumption?
l
lâm nguyễn hoàng
12/28/2020, 7:16 PM
since the message consuming from kafka doesn't change anymore ... but the table under pinot changes the number of lines continuously
Post