I am doing a PoC, Till now I used quick_start_stre...
# troubleshootingr
Ravikumar Maddi
04/23/2021, 4:32 PMI am doing a PoC, Till now I used quick_start_stream code. And did some work on that. But when I stop the Pinot, everything tables, schema and data is gone. How to make it all data should be in local permanently. And when I restart the pinot, how can I see tables, schema and data.
Need your help
m
Mayank
04/23/2021, 4:34 PM
If you delete the cluster (which will happen if you delete ZK as part of stopping Pinot) then everything is deleted.
r
Ravikumar Maddi
04/24/2021, 8:17 AMWhat should I do to stay all tables and data stay(store in secondary storage) even I stop and restart. Can you help please...
m
Mayank
04/24/2021, 4:50 PM
You can have data in deepstore and configs in ZK/helix. You can stop brokers and servers but don’t stop controller/zk, they will cause data loss
Mayank
04/24/2021, 4:51 PM
Why do you want to stop though? Is it for cost saving?
k
Kevin Vu
04/26/2021, 4:32 PMI also have this problem, I want to have all of data persisted in the case of a network outage for the ZK/Controller boxes. Is there any configuration required in order to have this data persisted to disk?
m
Mayank
04/26/2021, 4:32 PM
When you say data - do you mean Pinot segments, or the metadata/config in ZK?
k
Kevin Vu
04/26/2021, 4:33 PMpinot segments and the metadata/configs
m
Mayank
04/26/2021, 4:33 PM
Pinot segments should be persisted in deep-store
Mayank
04/26/2021, 4:36 PM
For metadata/configs, there should be a way to rebootstrap ZK from storage I suppose? If so, you can file an issue on Pinot to support that? Or give it a shot yourself, we welcome contributions 🙂
Mayank
04/26/2021, 4:37 PM
In production though, do you expect ZK/controllers to be completely stopped?
k
Kevin Vu
04/26/2021, 4:38 PMin production we can expect power outages so the linux boxes may go down.
Kevin Vu
04/26/2021, 4:38 PMwhat is deep storage? like HDFS?
m
Mayank
04/26/2021, 4:43 PM
yes, Pinot supports HDFS, S3, GCS, ADLS, NFS right now
Mayank
04/26/2021, 4:44 PM
You should also build in replication across fault domains to handle power outages.
k
Kevin Vu
04/26/2021, 6:56 PMAre there any guides for having the real time table write/save the segmentsto HDFS/NFS? I only see guides for batch job ingestion configs which read files from some file system and into another filesystem.
m
Mayank
04/27/2021, 2:39 PM
That happens under the hood based on what deepnstore you configured