https://pinot.apache.org/ logo
Join Slack
Powered by
# troubleshooting
  • p
    I haven’t configured deepstore and rebalance to offline yet
  • k
    did you suffix _REALTIME
  • k
    thats ok
  • p
    no
  • p
    I added a tenant with 
    DefaultTenant
    with one realtime, offline and broker
  • p
    and created a table to use this tenant
  • p
    *with only realtime config
  • p
    maybe because i didn’t configure deepstore? not sure how the the byte sizes are computed or when
  • k
    I think its more about segments are not flushed
  • k
    they are in consuming state
  • p
    yeah i see this corresponding to a segment metadata “segment.realtime.status”: “IN_PROGRESS”
  • k
    we can't compute the size while its progress
  • p
    got it, makes sense
  • p
    when do they move out of IN_PROGRESS ?
  • k
    when it reaches the threshold
  • p
    i guess 
    segment.flush.threshold.siz
  • p
    got it, didn’t realize that, that makes sense, thanks
  • k
    Copy code
    "realtime.segment.flush.threshold.size": "0",
  "realtime.segment.flush.threshold.time": "24h",
  "realtime.segment.flush.desired.size": "150M",
  • k
    first one to hit
  • p
    nice
  • k
    if its size, it takes few segments to get it right
  • k
    it learns over time since we have to reverse engineer the number of rows
  • p
    you mean if we specify size 0?
  • k
    no, if you specify size as 100mb
  • k
    first time it might flush at 50mb
  • k
    then 120mb
  • k
    then 105mb
  • k
    something like that
  • p
    oh
  • p
    got it, given it would be difficult to know the size before hand
1...112113114...166Latest