https://pinot.apache.org/ logo
#general
Title
# general
g

Grant Sherrick

10/22/2021, 3:02 PM
👋 Hi y’all, I’m taking the day off work to volunteer on pinot. I mostly work on streaming data related tasks day to day, would it be reasonable if I gave implementing a
KafkaThriftMessageDecoder
a try? I haven’t seen anyone clammering for it, but I thought it might be worth giving it a go.
👋 1
k

Kishore G

10/22/2021, 3:57 PM
thats amazing Grant!
go for it, I think protobuf is also another one that users have asked for
feel free to ping us if you need any guidance/help
g

Grant Sherrick

10/22/2021, 4:05 PM
Sounds good! We use a bit of protobuf around here. Now that it’s included in the schema registry, I am sure that it’s not too far off from how the KafkaAvroMessageDecoder works right now
k

Kishore G

10/22/2021, 4:05 PM
right.
g

Grant Sherrick

10/22/2021, 4:06 PM
I’ll start with thrift and keep y’all up to date with how it’s going. There’s no JIRA or any other work tracker for this project, right?
k

Kishore G

10/22/2021, 4:07 PM
there is, we use github
🙏 1
this was the record reader but that only works for batch ingestion
g

Grant Sherrick

10/22/2021, 9:52 PM
Okay, I’ve got a first pass here. I won’t have time to finish it up today, but I’ll give it some more polish in a few weeks: https://github.com/apache/pinot/pull/7620/files
k

Kishore G

10/23/2021, 4:34 PM
Thanks @User. that was quick. Did a quick review. It looks good, add the test case and we should be able to get the V0 merged