I’ve been fooling around with how Pinot handles the “URI push” of segments. It seems like if I’m not using HDFS for deep storage, then the controller will download the segments before pushing to the server, which seems like it’s not a win. Is that correct? And (so far) I haven’t been able to configure the controller to successfully handle an HDFS URI push request, at least when I’m not using HDFS for deep storage - I see the msg when the controller starts up that the “hdfs file system” was initialized, but when it gets the URI push request, it fails with an error about the hdfs file system not being initialized. Any ideas?

URI push should work for all deep-storages that provide uri based access (HDFS/ADLS/GCP/S3), only exception is NFS I'd think

I was trying to figure out if you could do an HDFS URI push without enabled deep storage for the same. So instead of pushing actual segments through the controller to be stored locally by server processes, you’d push the URI and the server process would download locally. Sounds like that’s not supported.

you need to use URI with metadata push

right, metadata push might be what you want to take a look