francoisa
06/23/2025, 9:19 AMgetting logs like software.amazon.awssdk.services.s3.model.S3Exception: The authorization header is malformed; the region is wrong; expecting 'eu-west-1'. (Service: S3, Status Code: 400, Request ID: 184BA1F70B628FA6, Extended Request ID: 82b9e6b1548ad0837abe6ff674d1d3e982a2038442a1059f595d95962627f827)
here is my server conf for the S3 part
# Pinot Server Data Directory
pinot.server.instance.dataDir=/var/lib/pinot_data/server/index
# Pinot Server Temporary Segment Tar Directory
pinot.server.instance.segmentTarDir=/var/lib/pinot_data/server/segmentTar
#S3
pinot.server.storage.factory.class.s3=org.apache.pinot.plugin.filesystem.S3PinotFS
pinot.server.storage.factory.s3.region=us-west-1
pinot.server.segment.fetcher.protocols=file,http,s3
pinot.server.storage.factory.s3.bucket.name=bucketName
pinot.server.storage.factory.s3.endpoint=URL_OF_MY_S3_ENDOINT
pinot.server.segment.fetcher.s3.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
pinot.server.segment.fetcher.s3.pathStyleAccess=true
Any ideas welcome 🙂Aleksandrs Vedernikovs
06/23/2025, 10:16 AM# Pinot Server Data Directory
pinot.server.instance.dataDir=/var/lib/pinot_data/server/index
# Pinot Server Temporary Segment Tar Directory
pinot.server.instance.segmentTarDir=/var/lib/pinot_data/server/segmentTar
# S3 Configuration
pinot.server.storage.factory.class.s3=org.apache.pinot.plugin.filesystem.S3PinotFS
pinot.server.storage.factory.s3.region=eu-west-1
pinot.server.storage.factory.s3.accessKey=YOUR_ACCESS_KEY
pinot.server.storage.factory.s3.secretKey=YOUR_SECRET_KEY
pinot.server.storage.factory.s3.disableAcl=false
pinot.controller.storage.factory.s3.disableAcl=false
pinot.server.segment.fetcher.protocols=file,http,s3
pinot.server.segment.fetcher.s3.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
pinot.server.instance.segment.store.uri=s3://bucketName
pinot.server.instance.enable.split.commit=true
pinot.server.storage.factory.s3.endpoint=URL_OF_MY_S3_ENDOINT
pinot.server.segment.fetcher.s3.pathStyleAccess=true
Controller
controller.data.dir=s3://bucketName
pinot.controller.storage.factory.class.s3=org.apache.pinot.plugin.filesystem.S3PinotFS
pinot.controller.storage.factory.s3.region=eu-west-1
pinot.controller.storage.factory.s3.accessKey=YOUR_ACCESS_KEY
pinot.controller.storage.factory.s3.secretKey=YOUR_SECRET_KEY
pinot.controller.segment.fetcher.protocols=file,http,s3
pinot.controller.segment.fetcher.s3.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
Lastly:
Verify S3 connectivity: Try to access your S3 bucket directly from the Pinot server using AWS CLI to confirm credentials and connectivity
Verify segment existence: Confirm that the segment absencesreport__1__5__20240527T2122Z actually exists in the expected S3 pathfrancoisa
06/23/2025, 10:18 AMfrancoisa
06/23/2025, 10:22 AMfrancoisa
06/23/2025, 12:30 PM