This message was deleted.
# generals
Slackbot
11/30/2023, 10:43 AMThis message was deleted.
j
John Kowtko
11/30/2023, 12:18 PMI don't think it's worker capacity ... I just ran this wikipedia join table create with 2 workers (see screenshot).
The error you are seeing is possibly related to this: https://github.com/apache/druid/pull/14042/ ... are you on an old version?
a
Ashok Kumar Ragupathi
11/30/2023, 6:00 PMHello John, thank you for your response first of all!
I am using V 28.0.0 (not older) yes it is working to me too for the wiki test data with 2 worker capacity.
But in this same single-server-instance I then migrated / imported other instance data-sources i.e. copied /var/druid/ folder (which holds 7 data-sources)
And then I tried to those copied datasources to merge (join) and store into in another data-source where it was not worked.
So I tried then again with the wiki test data, then it is not worked!
That's why I thaught would be the resource utilization issue.
Ashok Kumar Ragupathi
11/30/2023, 6:00 PM👆
Ashok Kumar Ragupathi
11/30/2023, 6:02 PMWell actually all I am trying is... using MSQ sortMerge algorithm I want to join and merge the more than 2 data-sources (normalized data) to the new datasource as de-normalized structured inorder to increase the Druid query fetch performance!
Ashok Kumar Ragupathi
11/30/2023, 6:02 PM👆
Ashok Kumar Ragupathi
11/30/2023, 6:04 PMSo while I trying the merge/join the data from existing data-sources I got that error saying failed to download the segment etc...
l
Laksh Singla
12/01/2023, 5:29 AMIs the data source present in the deep storage?
a
Ashok Kumar Ragupathi
12/01/2023, 6:11 AMyes Laksh
Ashok Kumar Ragupathi
12/01/2023, 6:11 AMit's present in deep storage
Ashok Kumar Ragupathi
12/01/2023, 6:12 AMBut I noticed in that server we have only 2 core cpu... and 8 GB ram only. So it may be the reason ? or something else !?
Ashok Kumar Ragupathi
12/01/2023, 6:13 AMBecause while I execute sortMerge REPLACE / INSERT SQL query it initiates (with 2 slots) for controller and worker and in the midway stop's and say error not able to download the segement. I mean SELECT is happening but not able write in the datasource... is my guess
Note: When I try only the SELECT query part in above it's working fine.
Ashok Kumar Ragupathi
12/01/2023, 6:16 AMnote : also all the segments are available.
j
John Kowtko
12/01/2023, 3:03 PMOkay my thought is then that the format of what you copied over is not quite compatible, at least not with MSQ. So my suggestion would be to first reindex each of the datasources into a fresh new one, so you know those segment files were created with your current cluster ... then try MSQ using your newly created datasources.
a
Ashok Kumar Ragupathi
12/01/2023, 9:54 PMThank you john for the suggestion.