https://pinot.apache.org/ logo
Join Slack
Powered by
# general
  • k

    Kishore G

    04/02/2019, 1:19 AM
    Can you send hard copy
  • s

    Shireen Nagdive

    04/02/2019, 1:20 AM
    hard copy?
  • k

    Kishore G

    04/02/2019, 1:20 AM
    Sorry. Pdf
  • s

    Shireen Nagdive

    04/02/2019, 1:21 AM
    EASY Paper
    EASY.pdf
  • s

    Sunitha

    04/02/2019, 1:51 AM
    Shireen, if you are not seeing latency increase, your servers still have a lot of headroom - what was the server cpu?
  • s

    Shireen Nagdive

    04/02/2019, 2:15 AM
    you mean server cpu load?
  • s

    Sunitha

    04/02/2019, 2:42 AM
    Yes - cpu usage
  • s

    Seunghyun

    04/02/2019, 3:53 AM
    i also agree with @User. If your cpu utilization is low, 99th percentile latency can go down because you have more samples with normal latency.
  • s

    Shireen Nagdive

    04/02/2019, 6:31 PM
    Yes, I have increased QPS. Let’s see what we get
  • c

    Chinmay Soman

    04/03/2019, 6:33 PM
    Hey folks, wanted to understand the schema evolution bit. If we evolve the schema by adding a new column (with a default value) - will it automatically apply this default value to the old segments (created using old schema) ?
  • c

    Chinmay Soman

    04/03/2019, 6:33 PM
    We see a behaviour where old segments are discarded
  • c

    Chinmay Soman

    04/03/2019, 6:33 PM
    Is this a bug / expected behaviour ?
  • k

    Kishore G

    04/03/2019, 6:33 PM
    that seems to be a bug
  • c

    Chinmay Soman

    04/03/2019, 6:34 PM
    ok , Ting filed this: https://github.com/apache/incubator-pinot/issues/4029
  • k

    Kishore G

    04/03/2019, 6:34 PM
    if you restarted the serve or reloaded the segment, the default value
  • c

    Chinmay Soman

    04/03/2019, 6:34 PM
    I can take a look - but just wanted to know before I start to investigate
  • c

    Chinmay Soman

    04/03/2019, 6:34 PM
    yes restart was done
  • m

    Mayank

    04/03/2019, 6:37 PM
    As a side note, this is handled by
    DefaultColumnHandler.updateDefaultColumns()
  • c

    Chinmay Soman

    04/03/2019, 6:38 PM
    ack
  • k

    Kishore G

    04/03/2019, 6:38 PM
    ok. looks like a bug. do you have the schema?
  • m

    Mayank

    04/03/2019, 6:38 PM
    It is expected to be called when segment is loaded (that happens at restart)
    ImmutableSegmentLoader.load
  • c

    Chinmay Soman

    04/04/2019, 12:42 AM
    It may be that our servers are somehow configured without "enable.default.columns" (the default seems to be false: https://github.com/apache/incubator-pinot/blob/master/pinot-server/src/main/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManagerConfig.java#L160 )
  • m

    Mayank

    04/04/2019, 12:43 AM
    ah
  • c

    Chinmay Soman

    04/04/2019, 12:43 AM
    that seems to be necessary here:
  • c

    Chinmay Soman

    04/04/2019, 12:43 AM
    https://github.com/apache/incubator-pinot/blob/master/pinot-core/src/main/java/org/apache/pinot/core/segment/index/loader/SegmentPreProcessor.java#L87
  • m

    Mayank

    04/04/2019, 12:43 AM
    We can evaluate if the default should be true
  • c

    Chinmay Soman

    04/04/2019, 12:43 AM
    I'm curious, shoudl the default be true
  • c

    Chinmay Soman

    04/04/2019, 12:43 AM
    what are the implications ?
  • m

    Mayank

    04/04/2019, 12:45 AM
    @User ^^
  • j

    Jackie

    04/04/2019, 12:47 AM
    @User If default column is enabled, when the data you pushed does not match the schema, we will automatically generate default values for the missing columns
1...747576...160Latest