https://pinot.apache.org/ logo
Join SlackCommunities
Powered by
# general
  • n
    servers still need to download and keep segments locally. setting up the deep store as gcs will only offload the ones from the controller (by default controller’s local FS is used as deep store)
  • x
    @User you can configure the storage class to be:
    Copy code
    # ssd-storageclass.yaml
apiVersion: <http://storage.k8s.io/v1|storage.k8s.io/v1>
kind: StorageClass
metadata:
  name: faster
provisioner: <http://kubernetes.io/gce-pd|kubernetes.io/gce-pd>
parameters:
  type: pd-ssd
  • x
    then use this as pvc
  • x
    Copy code
    # ssd-claim.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-volume
spec:
  storageClassName: faster
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 30Gi
  • x
    also pd-ssd is expandable
  • x
    so you can update the pvc size if it’s out of disk space
  • x
  • x
    We recommend to use GCE PD for server serving directory: https://kubernetes.io/docs/concepts/storage/storage-classes/#gce-pd
  • t
    so if i am consuming from streaming data, how would you recommend managing storage?
  • x
    still, for pinot-server, you can check the pvc setup and use 
    pd-ssd
  • x
    then once the disk is full, you can expand the pvc
  • x
    another thing is to start with more pinot-servers
  • x
    just need to plan on the disk as well
  • x
    You can update the pvc size as well
  • x
    Copy code
    helm inspect values pinot/pinot > /tmp/pinot-values.yaml
  • x
    then modify the values in 
    /tmp/pinot-values.yaml
  • x
    Copy code
    helm install pinot pinot/pinot -n pinot --values /tmp/pinot-values.yaml
  • x
    then install with the command:
  • t
    so what is the advantage of setting up the deep store as GCS?
  • t
    Hello community, We are pleased to announce that Apache Pinot (incubating) 0.5.0 is released! Apache Pinot (incubating) is a distributed columnar storage engine that can ingest data in realtime and serve analytical queries at low latency. The release can be downloaded at: https://downloads.apache.org/incubator/pinot/apache-pinot-incubating-0.5.0/ The release note is available at: https://docs.pinot.apache.org/basics/releases/0.5.0 Additional resources - Project website: https://pinot.apache.org Getting started: https://docs.pinot.apache.org/getting-started Mailing list: dev@pinot.apache.org Slack channel: https://communityinviter.com/apps/apache-pinot/apache-pinot Twitter: https://twitter.com/ApachePinot Best Regards, Apache Pinot (incubating) Team
    🍷 8
    🎉 9
    👍 14
  • n
    Yet another amazing blog written by @User, this time using Pinot and Superset to visualize climate change: https://medium.com/apache-pinot-developer-blog/building-a-climate-dashboard-with-apache-pinot-and-superset-d3ee8cb7941d
    👏 2
    🍷 4
    🎉 7
  • n
    Please share it in your networks 🙂 🍷
  • j
    👋 is there an easy way to pass in the 
    instanceId
    variable to Pinot without hardcoding it into a 
    conf
    file? For example, passing it in as an env variable or CLI parameter. Context is the 
    conf
    file is being puppeted and the ID will be obtained during a launch script. Being able to plumb it through while calling 
    pinot-admin.sh
    would be nice convenient, but this if not it's probably just as easy to copy the configuration, edit it, and load the edited version.
  • x
    Right now 
    instanceId
    can only be set through 
    conf
    files. One way to workaround this is to have a wrapper start script to update the 
    conf
    file then start pinot.
  • k
  • k
    A future commit will enable properties loaded from PINOT_X environment variables.
  • j
    :P
  • n
    Hi all, Here’s the 0.5.0 release blog by @User with all the details of this release. Nicely done Ting! https://medium.com/apache-pinot-developer-blog/introducing-apache-pinot-0-5-0-e27983b6a678
    🎉 5
    🍷 5
    ❤️ 3
    👏 8
  • k
    Hi folks. @User has put together a really excellent video tutorial explaining segment assignment in Pinot. Thanks everyone for providing input and voting on upcoming video tutorials and kudos to Neha for putting this together.

    https://www.youtube.com/watch?v=HycNRCzkrjg&amp;feature=youtu.be▾

    🍷 5
    👏 2
    🎉 5
  • u
    In this link https://www.slideshare.net/jeanfrancoisim/intro-to-pinot-20160104, The author talks about a retention time. Does anyone know if this configurable and where we can set it?
1...143144145...160Latest