Hi team, I'm trying to download segments from one ...
# troubleshooting
j
Hi team, I'm trying to download segments from one Pinot table and use them in a different table that has the same schema but a different retention configuration. Currently, I'm encountering an issue where the metadata doesn't match because the tables have different names.
m
If you are using upload api, you can specify the destination table name
j
To clarify, my process involves downloading a segment from deep storage (S3), tar it, and then attempting to upload it using the following command: Bash
Copy code
bin/pinot-admin.sh UploadSegment \
  -controllerHost <host> \
  -controllerPort <port> \
  -segmentDir /opt/pinot/tmp/ \
  -tableName test_stream3_full_event_OFFLINE
The original segment name is
test_stream1_full_event__0__0__20250604T1106Z.tar.gz
, and I am trying to upload it to the
test_stream3_full_event_OFFLINE
table. I am encountering the following error:
Copy code
Exception while uploading segment: Cannot find segment metadata file under directory: /tmp/pinot-tmp-data/untarredFileTemp/tmp-c594b424-52c5-4568-8495-7b85946fb039/pinot_job_output" while sending request: /v2/segments?tableName=test_stream3_full_event_OFFLINE&tableType=OFFLINE to controller: pinot-controller-0.pinot-controller-headless.mtr.svc.cluster.local, version: Unknown
I am uncertain if this operation is possible with the current Pinot API, and whether segments can be reused in this manner 🤔
m
The underlying rest API definitely supports it.
The error in your log is on the sender side, it is expecting metadata file but not finding in the directory.