Apache Pinot

Is there a way to recover the Pinot Cluster along with data if the zookeeper data is corrupt or lost and zookeeper is started with clean state.

It’s almost impossible, if you can recover all the table configs, then you can try to repush all the segments from deep to recover.
Otherwise I would always suggest to backup zookeeper periodically.

you can change all the segment file `myTable_xxx` to `myTable_xxx.tar.gz` , then run segmentUriPush job. Ref: <https://docs.pinot.apache.org/configuration-reference/job-specification>

<@UGRJA9TEH> I am also facing the same issue. I have created a segmentUriPush job using below configuration. But the job produces no output and completed in a few seconds. Have I missed anything? Thanks for the help.

```executionFrameworkSpec:
  name: 'standalone'
  segmentUriPushJobRunnerClassName: 'org.apache.pinot.plugin.ingestion.batch.standalone.SegmentUriPushJobRunner'

jobType: SegmentUriPush

inputDirURI: '<s3://d3-testing-bucket/metric_numerical_agg_1hour_partition/>'
outputDirURI: '<s3://d3-testing-bucket/tmp/metric_numerical_agg_1hour_partition/>'
searchRecursively: false
overwriteOutput: true
pinotFSSpecs:
  - scheme: s3
    className: org.apache.pinot.plugin.filesystem.S3PinotFS
    configs:
        region: 'us-east-1'
tableSpec:
  tableName: 'metric_numerical_percentile_1H'
  schemaURI: '<http://localhost:9000/tables/metric_numerical_percentile_1H/schema>'
  tableConfigURI: '<http://localhost:9000/tables/metric_numerical_percentile_1H>'
segmentNameGeneratorSpec:
  type: simple
pinotClusterSpecs:
  - controllerURI: '<http://localhost:9000>'
pushJobSpec:
  pushParallelism: 1
  pushAttempts: 2
  pushRetryIntervalMillis: 1000```

where are the segment files? are they located at `outputDirURI: '<s3://d3-testing-bucket/tmp/metric_numerical_agg_1hour_partition/>'` and all the files have `.tar.gz` extension?

I moved to segmentTarPush job and was able to make it work. But the values getting loaded to pinot for my data are all zeroes. I am not sure about the issue. All the metrics are getting populated as zero timestamp and dimension fields are there. Are you aware of this?

Please let me know if there is a way to read columns.psf file present in uncompressed segment file.

if you have code access, you can use `DefaultSegmentDirectoryLoader` to load the segment directory.

how do you define the metrics? are those derived columns or transform functions ?

Thanks for the info. I was able to fix the issue, there was a data issue. However SegmentURIPush job is not working with below error, it seems to be bug.

```500 (Internal Server Error) with reason: "Could not find paramName tableName in path or query params of the API: <http://localhost:9000/v2/segments>" while sending request: <http://localhost:9000/v2>```