https://linen.dev logo
Join Slack
Powered by
This message was deleted.
# troubleshooting
s
This message was deleted.
v
when you say "1 records out of 656" do you mean rows or segments?
a
Hi Vadim... total number of rows
h
Can you check any of the tasks which are successful and see the segments that got published? We can see the events processed and other metrics as well from each tasks which can let us know further
a
@Hemanth Patil to that particular suerpvisor (x - datasource), tasks are shows success in previous date time. & in one certain time the latest task id shows PENDING and then again RUNNING like that
h
@Ashok Kumar Ragupathi The tasks which are already finished, you can check the reports or the tasks log to see more info like published segments, number of events
a
ok
let me check that
h
can you also provide a screenshot of the druid console data source page ?
We can see if this data source is trying to load any segments.
a
👆 datasource page
Actually I synced last day 667 records / rows... and seen about 142 segments like that... but today it has only 1 rows 1 segment (no idea what happened in-between)
Please note I have only 7 worker capacity (slots) based on recommendation No., of CPU - 1. But I have 8 Tasks & (10 data-sources)
will this is the reason to disappear of those records?
h
From the screenshot, the datasource has only 1 segment which is loaded. Thats why we are seeing only 1 row for it.
The task slots should not be an issue if every hour you are seeing one task running for this DS
a
well, but it was 142 segment (approx.,) yesterday after I did sync using kafka stream (stream has not lost the data). In between something happened, and I could today only 1 segment and 1 row
h
can you check the metaDB for druid_segments table for this DS
DRUID_SEGMENTS
a
The task slots should not be an issue if every hour you are seeing one task running for this DS ------> here where I have concern, ofcourse it running but seeing often PENDING state and RUNNING state and the duration column shows (in tasks list page) very less time usually it will hit 1 hour.
How to check metaDB?
where ?
can possible to check in druid console?
h
Can you share the screenshot of tasks page after filtering this datasource ?
In the 
common.runtime.properties
you will see the information on the MetaData
Copy code
druid.metadata.storage.type
druid.metadata.storage.connector.connectURI
If you check the last success one, which was created around 6:30 What does the report and task logs says?
a
Copy code
{
  "ingestionState": "COMPLETED",
  "unparseableEvents": {},
  "rowStats": {
    "buildSegments": {
      "processed": 0,
      "processedWithError": 0,
      "thrownAway": 0,
      "unparseable": 0
    }
  },
  "errorMsg": null,
  "segmentAvailabilityConfirmed": true,
  "segmentAvailabilityWaitTimeMs": 0
}
Reports 👆
Status : { "id": "index_kafka_portal_closed_batches_bd2e5d643d6b8cc_pjiohnfn", "groupId": "index_kafka_portal_closed_batches", "type": "index_kafka", "createdTime": "2023-03-09T063018.988Z", "queueInsertionTime": "1970-01-01T000000.000Z", "statusCode": "SUCCESS", "status": "SUCCESS", "runnerStatusCode": "WAITING", "duration": 3606157, "location": { "host": "localhost", "port": 8103, "tlsPort": -1 }, "dataSource": "portal_closed_batches", "errorMsg": null }
h
processed is 0
so no new data
a
yes
currently no new data
h
You can check the metaDB to see how many segments got recorded for this DS
a
ok let me check
druid.metadata.storage.type=derby druid.metadata.storage.connector.connectURI=jdbc:derby://localhost:1527/var/druid/metadata.db;create=true druid.metadata.storage.connector.host=localhost druid.metadata.storage.connector.port=1527
h
So its local derby
a
yes
h
Try this from the installation directory
Copy code
./bin/derby-client
a
Tried... no result ubuntu@OPC-Analytics:~/opc-analytics/apache-druid-25.0.0$ ./bin/derby-client -bash: ./bin/derby-client: No such file or directory
where to try ?
h
From the installation directory, do you see 
bin
directory
Copy code
cd bin
a
yes i see bin
h
Inside bin, can you see 
derby-client
a
No
no such derby-client inside bin folder
h
Not sure tho, but can you check if you can read the file 
/var/druid/metadata.db
Thats the derby file which has the data
Also you can try to search for 
org.apache.druid.server.coordinator.duty.UnloadUnusedSegments
in the coordiantor logs to see if any segments are dropped
a
ok let me try on that
5 Views
Open in Slack
PreviousNext
Powered by