Hi All, I was wondering whether the apache paimon ...
# random
s
Hi All, I was wondering whether the apache paimon connector supports
LookupSource
. Paimon is already a LSM based structure, so would be neat if queries can lookup from the paimon source itself, rather than creating another internal rocksDB/hashmap stateBackend key-value store for the same data
actually thinking more about it.. i guess its not possible because typical object stores (like S3) have too much latency to serve as a LookupSource (not to mention the cost of GET api requests πŸ˜› I guess i missed this point because I was always working with the local file system storage rather than cloud storage
g
@Sharath Gururaj paimon provides the lookup join, that uses its own LSM https://paimon.apache.org/docs/master/how-to/lookup-joins/#lookup-joins and there is also a Query Service in the making πŸ˜„
πŸ™Œ 1
s
haha awesome πŸ˜… I thought I had completely read the docs.. somehow I missed that.. thanks for the info!
βœ”οΈ 1