Paweł Motyka
06/04/2024, 9:16 AMGian Merlino
06/05/2024, 4:39 AMGian Merlino
06/05/2024, 4:42 AMorg.apache.druid.data.SearchableVersionedDataFinder
the interface contemplates some form of searching for the latest file at a location, but i think for http (since there isn't a standard "list files" operation) it should skip that and just fetch the provided uri, assuming it matches the pattern.Gian Merlino
06/05/2024, 4:44 AMhttp input source periodically (using some external scheduler)
for an example check out "REPLACE an entire datasource" here: https://druid.apache.org/docs/latest/multi-stage-query/examples#replace-an-entire-datasource
then use standard SQL JOIN at query time.
the table will not be broadcast to all servers ahead of time (broadcast will need to be done at query time). but perhaps this is OK for your use case. you could try it out and see how well it performsPaweł Motyka
06/05/2024, 11:26 AM