https://pinot.apache.org/ logo
#general
Title
# general
s

Sheetal

09/16/2021, 3:35 PM
Hi all, I want to do insert of a single record to Pinot from an application. Setting up Kafka for real time ingestion seems too complicated for very small volume insert calls. Is there some other way?
k

Kishore G

09/16/2021, 3:53 PM
Pinot does not support row level insert api as of now. You can use the batch api
n

Neha Pawar

09/16/2021, 5:00 PM
You can try out the SegmentWriter interface. There’s no document yet, but this test demonstrates how to use https://github.com/apache/pinot/pull/6718/files#diff-7f887860478edd25adc4277de42a38b84ee6e6571c89d0eb05172fa1b6073d4d But in any case, creating segments with single row isn’t the best idea
👍 1
s

Sheetal

09/16/2021, 5:47 PM
Thanks I will check it out. But yes the single row thing is what’s troubling me. I use the table for searchbox so I don’t want to batch process
k

Ken Krugler

09/16/2021, 5:48 PM
Are you trying to use Pinot for near real time search?
s

Sheetal

09/16/2021, 6:53 PM
Its a normal search bar. It will be a regex query on one of the columns. The reason to use Pinot is I have some dashboarding needs which require fast aggregations. Having a different database like Mysql for just one table (the search query one) seemed like an unnecessary layer. So I am thinking to use Pinot for the search
k

Ken Krugler

09/16/2021, 6:55 PM
OK, but normally adding row-by-row (not batch) means you want near-real time (NRT) search. As in, soon after data is available you want it to be searchable. Otherwise you could just use batch to generate segments every day (as an example).
s

Sheetal

09/16/2021, 7:47 PM
oh yes, I want it near real time. Data should available for all as soon as insert is done
k

Kishore G

09/16/2021, 8:36 PM
If you are planning to use this in production and expect strong guarantee.. it’s better to use Kafka
s

Sheetal

09/17/2021, 7:47 PM
I decided to go with SQL sever for this
a

Ashwin

09/18/2021, 8:09 PM
Have you looked into Apache pulsar instead?
s

Sheetal

09/23/2021, 9:54 PM
Have you looked into Apache pulsar instead?- No let me check
n

Neha Pawar

09/23/2021, 10:46 PM
@User how would introducing Pulsar be operationally very different than setting up Kafka for this?
a

Ashwin

09/24/2021, 1:53 AM
With Pulsar, the amount of operational steps you have to take to set up your clusters isn’t as extensive as Kafka.