Amit Chopra
12/11/2020, 4:50 PMXiang Fu
Amit Chopra
12/11/2020, 5:30 PMXiang Fu
Amit Chopra
12/11/2020, 5:39 PMXiang Fu
Amit Chopra
12/11/2020, 8:15 PMTing Chen
12/11/2020, 8:44 PMMahesh Yeole
12/14/2020, 7:20 PMXiang Fu
java.io.IOException: software.amazon.awssdk.services.s3.model.NoSuchKeyException: The specified key does not exist. (Service: S3, Status Code: 404, Request ID: E62169F11317304B, Extended Request ID: 3dlRY25FjPWIVJsA82PfQnhwlyp/26Nw1VM2xZCzlqEUvNSIXpFSexbvMewbLTR3ZuaDSHE6rq8=)
Mahesh Yeole
12/14/2020, 9:35 PMXiang Fu
Mahesh Yeole
12/14/2020, 9:52 PMXiang Fu
<s3://pinot-quickstart-s3/pinot-data/pinot-s3-example/controller-data>
pinot-quickstart-s3
?s3://<bucket>/pinot-quickstart-s3/pinot-data/pinot-s3-example/controller-data
Mahesh Yeole
12/14/2020, 9:59 PMXiang Fu
pinot-cluster-segment-s3
is the s3 bucket name?Mahesh Yeole
12/14/2020, 10:06 PMXiang Fu
Mahesh Yeole
12/14/2020, 10:08 PM{
"segment.realtime.endOffset": "9223372036854775807",
"segment.time.unit": null,
"segment.start.time": "-1",
"segment.flush.threshold.size": "10000",
"segment.realtime.startOffset": "10000",
"segment.end.time": "-1",
"segment.total.docs": "-1",
"segment.table.name": "pullRequestMergedEventsAwsMskDemo_REALTIME",
"segment.realtime.numReplicas": "1",
"segment.creation.time": "1607971877305",
"segment.realtime.download.url": null,
"segment.name": "pullRequestMergedEventsAwsMskDemo__0__1__20201214T1851Z",
"segment.index.version": null,
"custom.map": null,
"segment.flush.threshold.time": null,
"segment.type": "REALTIME",
"segment.crc": "-1",
"segment.realtime.status": "IN_PROGRESS"
}
{
"code": 404,
"error": "Segment: pullRequestMergedEventsAwsMskDemo__0__1__20201214T1851Z of table: pullRequestMergedEventsAwsMskDemo_REALTIME not found at: <s3://pinot-cluster-segment-s3/pinot-data/pinot-s3-example/controller-data//pullRequestMergedEventsAwsMskDemo_REALTIME/pullRequestMergedEventsAwsMskDemo__0__1__20201214T1851Z>"
}
Xiang Fu
Amit Chopra
12/15/2020, 6:15 PMTing Chen
12/15/2020, 6:19 PM"completionConfig": {
        "completionMode": "DOWNLOAD"
      }
by doing so, it can reduce the amount of downloads from s3.Amit Chopra
12/15/2020, 6:22 PMhttps://cwiki.apache.org/confluence/display/PINOT/By-passing+deep-store+requirement+f[…]145724933/Screen%20Shot%202020-02-05%20at%204.45.39%20PM.png▾
Ting Chen
12/15/2020, 6:36 PMAmit Chopra
12/15/2020, 6:37 PMJatin Yadav
11/15/2023, 7:24 AMTing Chen
11/20/2023, 9:06 PM