Hello, has someone success with Minio S3 as a deep storage backend?

I am not able to recollect but I remember someone getting it to work.. are you facing any issues?

I’m Im not sure how to test it, I don't see any error but neither any files in bucket when segment is completed

Have you configured the fs in controller?

extra: configs: |- pinot.set.instance.id.to.hostname=true controller.task.scheduler.enabled=true controller.enable.split.commit=true controller.data.dir=s3://pinot/ pinot.controller.storage.factory.s3.accessKey=* pinot.controller.storage.factory.s3.secretKey=* pinot.controller.segment.fetcher.protocols=file,http,s3 pinot.controller.segment.fetcher.s3.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher pinot.controller.storage.factory.class.s3=org.apache.pinot.plugin.filesystem.S3PinotFS pinot.controller.storage.factory.s3.endpoint=http://minio-democ.minio-tenant-1:9000 pinot.controller.storage.factory.s3.region=us-west-2 pinot.controller.storage.factory.s3.disableAcl=false

You might need to remove controller.extra.configs.controller.data.dir=s3://pinot and instead define the location of the s3 bucket using controller.data.dir. Refer to this thread for further details. I have Minio S3 working as a deep storage backend for my development environment.

Perfect! It’s working. Controller needs to have dir set to s3:// in helm values and in other config I have controller.local.temp.dir=/tmp/pinot-tmp-data/ …

@User seems like a good one to document.