Hi Everyone, Can any one explain what would be the reason for segment not getting fully load. Attaching screen shots of datasource & Services tab of druid. How We can fix this issue?

Your historicals are full. Please try dropping some unneeded data, adding another historical or increasing the historical sizes.

We have 1 master (8Core 32GB RAM), 2 Datanodes(8 core 64gb RAM) and 1 query(8Core 32GB RAM) node setup. historical/runtime.properties druid.host=<CURRENT_SERVER_HOST> druid.processing.buffer.sizeBytes=500MiB druid.processing.numMergeBuffers=2 druid.processing.numThreads=7; druid.segmentCache.locations=[{"path":"var/druid/segment-cache","maxSize":"8g"}] druid.cache.sizeInBytes=1GiB Here We have set 8g. It is taking disk space right? Also if We increase this maxsize would it solve our problem for future?

Yeah

ok...My only concern is, If we get more data in datasources then might be this issue again occurs. FYI for deep storage we are using azure blob.

it will happen again if you add data and fill up the space, of course

@Jordi Escrich As per your last comment, If I will create one retention rule (see in screenshot), After that only 1 day segment will remains in historical disk rest will be free, right? As data is present in deep storage so if we do query for last 1 week records then it will pull data from deep storage and serve the request. Am I right or wrong on this. I am more concerned about data loss right now if we do this activity.

there is new behavior on the last versions of druid that is able to pull data from deep storage if it is not in the historicals, but haven't played with it yet

image.png

If you use a "Load" rule and specify 0 replicas, the segments will not load onto Historicals but will still be queryable via the MSQ engine: https://druid.apache.org/docs/latest/querying/query-deep-storage