This message was deleted.
# generals
Slackbot
02/07/2023, 10:23 AMThis message was deleted.
k
Kyle Hoondert
02/07/2023, 10:27 AMHi Ashok, this operation can be performed using a Druid datasource as the input to an ingestion - either using Native batch (https://druid.apache.org/docs/latest/ingestion/native-batch-input-sources.html#druid-input-source) or MSQ https://druid.apache.org/docs/latest/multi-stage-query/examples.html#insert-for-reindexing-an-existing-datasource
You might be able to bring in additional data by combining the Druid datasource and an external source with a JOIN as shown in https://druid.apache.org/docs/latest/multi-stage-query/examples.html#insert-with-join
a
Ashok Kumar Ragupathi
02/07/2023, 11:01 AMThanks Kyle
b
Ben Krug
02/07/2023, 11:16 AMOut of curiosity, what will be in the new column? Are you adding new, separate data to the existing data? If so, how would you match it up? Or is it something derived from the existing, or what?
a
Ashok Kumar Ragupathi
02/07/2023, 11:17 AMyes adding new column in the copied data-source.
Ashok Kumar Ragupathi
02/07/2023, 11:18 AMactually am using kafka stream...
Ashok Kumar Ragupathi
02/07/2023, 11:19 AMnow requirement is to add additional (new) column to the existing data-source
Ashok Kumar Ragupathi
02/07/2023, 11:22 AMwhich will have values for fresh streaming data / new insertion
Ashok Kumar Ragupathi
02/07/2023, 11:23 AMhow to do !?
b
Bibek Sahoo
02/07/2023, 12:02 PMFor adding new column, If you just update the SPEC and add the new column and resubmit it Druid should be able to fetch the new column
Bibek Sahoo
02/07/2023, 12:02 PMand once new data is available from kafka side druid will have the new data in the new column
a
Ashok Kumar Ragupathi
02/07/2023, 12:20 PMokay thank you Sahoo
Ashok Kumar Ragupathi
02/07/2023, 12:54 PMSahoo I checked as you mentioned above it working fine. I added new column and for new insertion am getting the value to the added new column. Is it possible to do update (similar to mysql) to the existing / previous records ?
Ashok Kumar Ragupathi
02/07/2023, 12:55 PMeither using druid SQL / native query ?
Ashok Kumar Ragupathi
02/07/2023, 12:55 PMor how to approach ?
Ashok Kumar Ragupathi
02/07/2023, 12:56 PMsorry using kafka stream (not using DSQL / native query)
b
Bibek Sahoo
02/07/2023, 12:56 PMUpdate is not possible i believe, you need to reingest as mentioned in this thread earlier
a
Ashok Kumar Ragupathi
02/07/2023, 12:57 PMok...