Hey :wave: I created a new Table in Pinot and it s...
# troubleshootingk
kaivalya apte
02/17/2022, 5:20 PMHey 👋 I created a new Table in Pinot and it starts off having 100 segments out of which only 51 are good. Is that expected? I haven’t pushed any data yet.
m
Mayank
02/17/2022, 7:19 PM
Can you check external view and ideal state?
k
kaivalya apte
02/18/2022, 10:12 AMyeah, there were servers in ideal state. I deleted those.
kaivalya apte
02/18/2022, 2:11 PMAlso many servers were dead because of OOM issues from the previous load test I ran.
p
Peter Pringle
02/19/2022, 1:31 PMrecall you are using kafka, are these are new topics? if not believe pinot will start from earliest so you might be processing existing data.
Peter Pringle
02/19/2022, 1:34 PMalso the flush threshold time/bytes and replica count can have an impact on memory of these are high and you have high ingestion rate
m
Mayank
02/19/2022, 2:41 PM
Table can be configured to consume from earliest or latest (in table config) https://apache-pinot.slack.com/archives/C011C9JHN7R/p1645277506989219?thread_ts=1645118415.518379&channel=C011C9JHN7R&message_ts=1645277506.989219
k
kaivalya apte
02/19/2022, 2:42 PMYes I have configured to consume from largest offset
kaivalya apte
02/19/2022, 2:42 PMFlush time is 24hrs (default) and not sure if thats right for my use case. Bytes is set to 200MB