Hello Friends,
I am facing error while ingesting data to new datasource from other datasource using BATCH-SQL-MSQ query (broadcast/sortMerge) option.
Steps I did as follows,
1. I have copied var/ folder files (i.e. metadata.db, segments, ...) from the older version 0.22.0 druid instance (single-server) -to-
new druid instance v26.0.0 (single-server)
2. once copied to new druid server, replaced the default var/ folder files by the copied one.
3. I restarted the new druid service
4. Now, I used BATCH-SQL-MSQ (join-queries) REPLACE INTO... SELECT query to ingest the data to the new flat structure datasource from the copied & existing normalized datasource
5. But while execute, query processed and at particular stage it's failed, saying "Failed to load segment"
In log am seeing below mentioned error...
2023-12-14T061958,127 ERROR [processing-0] org.apache.druid.segment.loading.SegmentLocalCacheManager - Failed to load segment in current location [/home/ubuntu/analytics/apache-druid-26.0.0/var/druid/task/query-c3316813-acb2-430e-822f-192decf79a18-worker5_0/work/indexing-tmp/segment-fetch],
try next location if any: {class=org.apache.druid.segment.loading.SegmentLocalCacheManager, exceptionType=class org.apache.druid.segment.loading.SegmentLoadingException,
exceptionMessage=Error loading [file:/home/ubuntu/Analytics/apache-druid-0.22.0/var/druid/segments/portal_customers/20210506T000000.000Z_20210507T000000.000Z/2023-01-23T17_20_49.673Z/0_index.zip], location=/home/ubuntu/analytics/apache-druid-26.0.0/var/druid/task/query-c3316813-acb2-430e-822f-192decf79a18-worker5_0/work/indexing-tmp/segment-fetch}
org.apache.druid.segment.loading.SegmentLoadingException: Error loading [file:/home/ubuntu/Analytics/apache-druid-0.22.0/var/druid/segments/portal_customers/20210506T000000.000Z_20210507T000000.000Z/2023-01-23T17_20_49.673Z/0_index.zip]
Question :
1. What is the root cause ?
2. Looks like error bco'z it still refer the older druid version instance segment file path! how to overwrtie / manage the metadata ?