Apache Druid

By default the druid core classloader is searched first for a class.

But you can try using `druid.extensions.useExtensionClassloaderFirst` (ref : <https://druid.apache.org/docs/latest/configuration/index.html#extensions>) which forces to check the extension CL first

<@U03B0DYSLUF> Are these guava deps by any chance ?
The other way is to shade the deps in your extension pom.

These are some hadoop/hive dependencies which are of a different version compared to the core druid ones. Let me test out the `druid.extensions.useExtensionClassloaderFirst` property and see how it goes, thank you!

just FYI, druid 25.0.0 also has a Hadoop-3 compatible build. <https://www.apache.org/dyn/closer.cgi?path=/druid/25.0.0/apache-druid-25.0.0-hadoop3-bin.tar.gz>

as this test is running, you can take a stack and see where its blocked

you could also run this particular test in intellij

Great idea, thanks! I actually ended up removing the tests internally, as it is a very old branch and we dont use this extension
```diff --git a/extensions-core/kafka-extraction-namespace/pom.xml b/extensions-core/kafka-extraction-namespace/pom.xml
index ab5d897d3a..9f218bacfa 100644
--- a/extensions-core/kafka-extraction-namespace/pom.xml
+++ b/extensions-core/kafka-extraction-namespace/pom.xml
@@ -181,4 +181,18 @@
       &lt;scope&gt;test&lt;/scope&gt;
     &lt;/dependency&gt;
   &lt;/dependencies&gt;
+
+  &lt;build&gt;
+    &lt;plugins&gt;
+      &lt;plugin&gt;
+        &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
+        &lt;artifactId&gt;maven-surefire-plugin&lt;/artifactId&gt;
+        &lt;version&gt;2.22.2&lt;/version&gt;
+        &lt;configuration&gt;
+          &lt;skipTests&gt;true&lt;/skipTests&gt;
+        &lt;/configuration&gt;
+      &lt;/plugin&gt;
+    &lt;/plugins&gt;
+  &lt;/build&gt;
+
 &lt;/project&gt;```

<https://cncf.pravega.io/docs/latest/javadoc/clients/io/pravega/client/stream/ReaderGroup.html>
Docs for Pravega ReaderGroup

assuming this is similar to kafka consumer groups, this thread <https://apachedruidworkspace.slack.com/archives/C0303FDCZEZ/p1663601546431449>  has some details of why kafka ingestion uses low level stream/partition positions

other abstractions are totally possible to define, i suggest updating <https://github.com/apache/druid/issues/13724> to elaborate on what the abstraction is if you’ve got something in mind to take the discussion there if the low level model of the ‘seekable stream’ abstraction isn’t what you plan to use

or it could probably live entirely in the extension if the model doesn’t really apply to other types of streams, pretty much everything is extendable with some varying amount of effort

Thank you for the information <@U030C8H59T8>. We took this into consideration and we are looking to see how we can still use the ReaderGroup abstraction level for the plugin and have it live entirely within the extension. Currently we were thinking about creating two  ReaderGroups (two partitions) so that Druid is able to use its own parallelism login within the Seekable stream supervisor. Initially we were just going to use one readergroup to manage all readers so we are still investigating

If your ReaderGroups are something like Kafka consumer groups, you may find that the SeekableStream abstraction is not a great fit. It really wants to think of things in terms of partitions and offsets, and it really wants to handle its own checkpointing. I bet you can come up with a simpler design by creating a new Supervisor and Task type

<@U0344FW86DD>
Sorry for the late reply, I am just seeing this now. Yes the readergroups abstraction are similar to consumer groups abstraction. We are currently trying our best to ingest the seekablestreamsupervisor code and see how we can make it work with readergroups but it is a little tough.

Would creating a new supervisor and task type compatible with a readergroup/consumergroup abstraction be more straightforward?

Because the Pravega client API isn't the best for accessing partitions and offsets at the lower level, readergroups are used to read streams and checkpoint at a higher level

<@U0344FW86DD> -- Yes, Pravega reader groups are similar to Kafka consumer groups, but slightly more powerful, supporting checkpoints across the whole reader group (like commitSync or commitAsync in Kafka, but committing offsets to a checkpoint across all readers in the group).

We are seeing that rewriting a new Supervisor would greatly simplify Druid's stream ingestion and autoscaling code, given the more complete abstractions provided by Pravega. We are new to Druid, so studying the SeekableStreamSupervisor ecosystem is how we're making our way through understanding the Supervisor/Task framework.

We are attempting to write an initial integration using a single reader in a reader group, thus it comes out as a single-task single-virtual-partition ingestion.

Being able to discuss issues with someone who knows the Supervisor/Task framework really well would likely accelerate our velocity.

there are some deep storage tests using an ancient s3 bucket unless someone removed it :sweat_smile:

iirc we do not have write access to that bucket (although i may not recall correctly…)

This is something that should be discussed over mailing list, as all PMC decisions should be formalized in such a way

Can you email dev@druid and we can discuss there?

We should consider 3 options

1) Use an ASF-supplied S3 bucket

2) Use an S3 bucket supplied by a 3rd party. I believe Imply would be willing to supply this

3) Use Minio (<https://min.io/>) or something similar instead of real S3

Personally, I feel (2) or (3) is a good option given the hesitation by Infra.

Anyway, please write to dev@druid, and link the thread here (using permalink from <https://lists.apache.org/list.html?dev@druid.apache.org>). I will reply with these thoughts and then we can see what others say as well :slightly_smiling_face:

Dev thread: <https://lists.apache.org/thread/vfob7hs20yv5oftl6o2n9olfgpdhfr9n>

i think it's historical artifact: it started off false by default when it was a new feature and we never made it true by default. It has been set to true in the default configs for a while now

I would definitely welcome a PR to change it to true by default

Thanks for the response . SGTM. I agree it should be true by default. :slightly_smiling_face:

I merged one just now and had a comment on an another one.

These PRs all aim to fix the helm chart being broken in general or on newer K8s versions

heads up to anyone with open PRs that modify stuff in druid-core, you will need to merge in master branch since <https://github.com/apache/druid/pull/13698> has been merged

github shows conflicts, however git recognizes them as file moves so there isn’t actually anything to do locally other than merge the changes in

unless you added a new file, which will need moved from core/ to the same place in processing/

Just merged <https://github.com/apache/druid/pull/13823>. If you are seeing static checks failures unrelated to your changes in PRs, Please pull the latest changes from master.

Ill trade ya for a review of <https://github.com/apache/druid/pull/13842|https://github.com/apache/druid/pull/13842>

Welcome! We sure do love DataSketches in Druid :heart:

Fun fact about DataSketches in Druid: Up until Druid 0.23 DataSketches was only used by Druid as user configurable data types. With the introduction of <https://imply.io/blog/multi-dimensional-range-partitioning/|multi-dimensional range partitioning> one year ago in Druid 0.23 DataSketches are now relied upon by 'core' Druid for accurately distributing data between segments. So now even if a user doesn't utilize DataSketches as part of their data schema they are still probably using them internally especially if they use <https://druid.apache.org/docs/latest/multi-stage-query/index.html|SQL based ingestion>, which uses range partitioning by default.

<@U030W50CNAC> thanks for the observation.  I assume you use one of our quantile sketches for the range partitioning.  I am curious: which one of the quantile sketches are you using? e.g., the traditional quantiles, KLL, REQ?  What size K are you configuring the sketch with? What is the data type you are feeding the sketch with: Strings or generic items, or ?  How big  can N be? What is the max # of splits across all dimensions?   Finally what is your tolerance for how equal the splits have to be in size?  e.g., 1%, 10%?  Understanding use cases like this is important for us.

All great questions! I am not the sketch expert to answer it - I am more of a sketch cheer leader. That said I believe it is some quantile sketch. I know that as part of SQL based ingestion (24.0) we added downsampling to the mix that was really cool. We also had an issue in a early dev version where the segment sizes were coming out really whacky and it was traced to us hilariously misconfiguring the sketches. I am sure <@U0318CX1NH4> and <@U0344FW86DD> could answer your questions more directly.

Configuring the sketches based on your application is where I might be able to help.

cc <@U03NCEXDNJK> Who has also done a lot work on data sketches  when used for partitioning.

• We are using an `ItemsSketch` with K = 32768 
• We are adding byte arrays to the sketch which are strings -&gt; bytes. 
• Theoretically, we donot have any upper bound on N 
• Number of splits is generally N/3_000_000 
We are downsampling the sketch. based on average element sizes. We store the the avg element size * sktech.getN() . If this crosses 300MB we downsample to the k/2.

That is a huge value of K and would correspond to a rank error of 64 parts-per-million or 0.0064% !

haha, I love parts-per-million. The best unit to measure error and CO2

So are you taking a single sketch and giving it ~ 3 million split-points?

I think it gets (N - number of rows) / 3000000 splits because it aims to make 3M rows per segment

so like if you have 90M rows in your dataset then 30 splits

&gt; We are downsampling the sketch. based on average element sizes. We store the the avg element size * sktech.getN() . If this crosses 300MB we downsample to the k/2.

Now I understand, I misread your bullet above.  but I still don’t understand the above sentence.  What are you doing when you “downsample the sketch” ?

```Downsample this sketch, dropping about half of the keys that are currently retained```
All records are pushed to the sketch.
Once the sketch reaches 300 MB we downsample it.
After all the records are read and a series of downsampling cycles, if required,  we get evenly spaced quantiles by `getQuantiles(sketch.getN()/target_row_batch_size())`

These data boundaries are then used to partition the data so that we get even cuts.

Are you going inside the sketch and removing half of the retained entries?

Or are you using the downSample(newK) method with a smaller K?

We call downSample(newK) method with a smaller K

Whew!  So why do you feel you need such a large K to start with?

If you start with a smaller K, it will speed up your processing :slightly_smiling_face:

We were initially on K = 4096 and then we saw usecases where 37B records were going into a single sketch

With a target size of 5M per partition, the error rate `0.063%` came out to be 3M so we had a distribution like :
```2M: 2, 4M: 320, 5M: 270, 6M: 6, 7M: 28, 8M: 4023, 9M: 24, 10M: 4, 11M: 4```
So we increased the K values so that the error rate reduces.

Since we donot know how many records will come into the system we went with the Biggest K value, and then downsampled stuff as we hit the memory limits.

But if you downsize 3 times, you will end up with a 4096 sketch, with the same error.  I’ll have to think about this.  Are the numbers above the resulting partition sizes of 18 partitions?

Do you have large numbers of duplicate rows?

And how successful is this strategy?  How much variance across the partition sizes are you getting?

The numbers above indicate num_rows_in_partition-&gt;num_partitions. So we had around 4.6k partitions .
Our accuracy for the smaller datasets has improved.

Regarding duplicate rows, we cannot control that. There may or may not be duplicates

Let me understand your numbers:  So at the end of processing one huge number of rows, your numbers represent the distribution of the resulting ~4.6K partitions?

So the range of partition sizes was from 2M rows to 11M rows

Hmm, That is not very good.  Let me think about this.    Could you send me a sample of what a single row might look like that is fed to the sketch?  Just one or two.

I will need to dig that up. Will send it to you in a few hours ?

no problem. Curious, where are you located? India? :slightly_smiling_face:

(sorry for the break, my wife called me for dinner :slightly_smiling_face: )

Your little distribution implies 62M rows and 4.68K partitions (like you said).

Yes located in India. We were able to repro the issue by pumping in longs (1B) to the item sketch .

I have a hunch that the variance you are seeing is not due to sketch error, but due to a likely power-law distribution of duplicates.  But I could be wrong. :slightly_smiling_face:

Hmmmm.   How did you generate the 1B longs?

We generated the longs serially in ascending order so that when we get the quantiles we know the segment sizes by just looking at the value.

and you got a similar distribution with a lot of variance?

Yes the variance we got:  rows_in_partition-&gt;num_partitions
`1M: 1, 4M: 261, 5M: 268, 7M: 3, 8M: 4091, 11M: 1`

Hmmm, I’m beginning to think we may have a bug in the item sketch.

A few more questions, if you don’t mind.  I’d like to reproduce your experiment.

Did you, by chance try feeding the longs into the DoublesSketch?

Did you configure the ItemSketch with a generic &lt;Long&gt;?

Also, the little distribution you show above must be a sampling of the output, because the total rows is only 36M.

I mean sampling of the resulting partitions.

Oh Right!   Stupid me, I was misreading you data!  :exploding_head:

It was most likely ItemsSketch&lt;Long&gt;

But if your input was just 1B rows, what is going on?

Oops I misread the notes. I donot have the variance for test data

OK, I think I have enough information to reproduce your experiment.  I’m also going to try the DoubleSketch to compare.  And your experiment also used a lgK of 15?

It is getting late for me, I probably won’t get much time to spend on this until Monday.

We set K=4096 for the above experimentation

No problem,  I hope you haven’t been struggling with this issue for very long.  Anytime you see something strange like this you should put an issue on our datasketches-java repo.  That way our whole team can see it.  :slightly_smiling_face:

<@U0344FW86DD> <@U030W50CNAC> <@U0315MQKEHK> <@U03NCEXDNJK>
Here is my initial analysis of what is causing the high variance in your partition sizes.

The short answer is that you are inadvertently requesting too many split points given the size of the maximum input stream and the size of the quantile sketch.   In other words, the implied resolution you are requesting is very close to or smaller than the error limit of the sketch, thus causing a lot of noise and variation in the resulting partition sizes.

To illustrate this I have run some simulations of what you have described to me about your use case and it comes down to these key parameters:
• Input N can be as large as 37B.
• Classic Quantile Sketch =&gt; ItemSketch&lt;String&gt; with K=2^15
• tgtPartSize: target partition size of 3M items. 
• Number of SplitPoints calculated by N/tgtPartSize.
I first decided to model this using the Classic Quantiles DoubleSketch, as handling strings is much much slower.  I felt that if I can reproduce the problem with the DoublesSketch then all I have to do is some spot measurements with the ItemSketch to eliminate anything peculiar with the ItemSketch.  ( I haven’t done this last step, because what I have so far is quite convincing, I think, but I will.)

My model has these parameters:
• lgK ranges from 10 to 15
• inputN ranges from 1B to 39B
• tgtPartSize = 2.5M items  (which is a little worse than (3M). 
Varying these parameters produces these metrics:
• NumPart: number of partitions = SplitPoints+1
• ImpRes; implied resolution = 1/numPart
• hMean: average resulting partition size
• hMax: maximum resulting partition size
• hMin: minimum resulting partition size
• hRange: hMax - hMin
• hRange% = hRange/hMean *100
• Variance; the computed variance of all the partition sizes
• StdDev: the computed standard deviation of all the partition sizes
• StdDev/Mean%: 
• SkErr: the error of the sketch determined by K
• Retained items: This gives a sense of the size of the sketch (in items, not bytes)
(to be continued)

yeah I think that makes sense: intuitively, we are asking for a lot of cut-points, meaning that we're very sensitive to errors. if we have 10 B input items, we're going to want about 2,000 cut-points, which means we want to get the 0.0005 ranked item, the 0.001 ranked item, the 0.0015 ranked item, and so on. so rank error has to be even smaller than that

is that matching what you are finding as well?

VersusLgK.png

(Continuing)

Let’s look at the following chart:

This first chart confirms that as we configure the sketch with larger K, Not only does the native error of the sketch go down (green), so does the range of the variation in sizes (hRange%) of the partition reduce as well as the standard deviation of that variation (StdDev/Mean%).  These two last metrics are normalized by the mean size of the partitions so we can get a sense of the impact.

As you can see from the title of the chart, N=1B, and with a target partition size of 2.5M, this results in only 400 partitions.  Thus, 1/400 =&gt; 2.5e-3 implied resolution (black line).  The dashed curves are plotted on the right vertical axis.  The key metrics of hRange% and StdDev/Mean% certainly improve with larger K.

VersusN.png

VersusN2.png

(Continuing)

Now let’s look at the next couple of plots:

The chart on the left is much more revealing.  Now I’m plotting against N keeping the other parameters constant.  Here N varies from 1e9 items to 39e9 items in steps of 2e9.  LgK is constant at 15, the TgtPartSize is constant at 2.5e6, and since the sketch is a constant size the SkErr is constant at 8e-5.

The black curve now decreases dramatically with each increase in N, where at the largest N, it actually crosses the green line and becomes less than the sketch error.  This is bad news. And you can see the results in the erratic plots of the hRange% and the StdDev/Mean%.  And as you can see at N=35B the variation in partition size is about 52% of the average size of the partitions!  This explains the wild variations you are seeing.  The StdDev/Mean% is smaller, because if you examine the Gaussian, +1 StdDev is only about 34% of the area under the curve.

The second chart just confirms the underlying cause:  If you always keep the target size of the resulting partitions the same size (here 2.5M items), as N increases, it demands more and more splits from the sketch, to the point where it demands more resolution from the sketch than it is designed to handle and you get very noisy results.

(Continuing)
We have had quite a bit of internal discussion in our team about this and here are some of our thoughts.
• Frankly, this use case has come as a bit of a surprise.  When we designed these sketches, we anticipated large N, and that’s not the problem. What we didn’t anticipate was large N and large number of split-points.  And this specific use-case requires values of K much larger than what we originally anticipated anyone ever needing.  
• We feel this needs a new sketch design specifically targeted for very large N, and very large # of split points, where K can scale much larger than 2^15.   This is why we would like to huddle with you folks to understand more about your environment and your constraints.
I hope you find this analysis interesting.

Cheers,

<@U0344FW86DD>
I think this analysis answers your question :slightly_smiling_face:

i find this very interesting! especially the tantalizing prospect of a sketch more closely suited to our needs :slightly_smiling_face:

<@U0315MQKEHK>
Question, to obtain the final partitions, what sketch functions are you using:  getQuantiles(…), getPMF(…), getCDF(…)?

And then what do you do to create the partitions?

&gt;  Question, to obtain the final partitions, what sketch functions are you using: getQuantiles(…), getPMF(…), getCDF(…)
We are using `sketch.getQuantiles()`
We get the various ranks and partition the data per worker according to them.

The `T[] sketch.getQuantiles(int evenlySpaced)`, returns an array of quantile values that define an array of buckets where each bucket is assigned a value from the quantile array.
Then you rescan the input stream and for each item you place it in a bucket where the value of the item is &lt;= a bucket value and &gt; than the bucket just below it.   Is this correct?

This will produce a quasi-sorted partition array, where the overall array is sorted but the values in each partition will not be sorted, but somewhat “close” together.

So now my question is. Why is this quasi-sorted property important?

Yes, that is correct. Druid uses primary partitioning by timestamp (the first value of each array is normally that, if that is relevant), so the property we need is to ensure that any change in timestamp value while getting quantiles would be a split point (I think this is something quantile sketches don't really handle well).

If we are say, partitioning by month, we use a different bucket for each month and use a different sketch to guarantee this property.

&gt; the property we need is to ensure that any change in timestamp value while getting quantiles would be a split point (I think this is something quantile sketches don’t really handle well).
This is the first I have heard of this issue. Has this been reported to us as an issue on our website?  If we don’t know about an issue, we can’t investigate it :slightly_smiling_face:  .  I can guess that it may be because quantile sketches are a stochastic sampling process, and in that process some date transitions get skipped over resulting in a transition that effectively occurs at some other nearby point but not exactly where the transition occurs in the raw data (but I can’t be sure).

If you know the first part of your input string is a date field, why don’t you just do an initial scan over your data and spray the items into date buckets?  You don’t need a sketch for that.

&gt; If we are say, partitioning by month, we use a different bucket for each month and use a different sketch to guarantee this property.
What different sketch?

&gt; If you know the first part of your input string is a date field, why don’t you just do an initial scan over your data and spray the items into date buckets?  You don’t need a sketch for that.
This is basically what we're doing today. The issue we have is we have a certain memory budget to use across _all_ buckets, so because we're managing one sketch per bucket, we need to manage that as we add things to the various buckets. Whenever we're about to exceed our memory budget, we pick a bucket and call `downSample` on the sketch for that bucket. The way we pick the bucket… "works" but is probably not ideal.

&gt; This is the first I have heard of this issue. Has this been reported to us as an issue on our website? 
No, since we figured it was out of scope for any of the sketches you have

Heads up everyone I made a new channel called <#C04TB8M30DV|> for everything DataSketches related, please join if interested.

Hey All,  still have two small helm chart modifications I'm looking to have reviewed.  Just merged in latest master.  They are relatively small changes that have been open for a few weeks:
<https://github.com/apache/druid/pull/13796>
<https://github.com/apache/druid/pull/13806>

It’s used by the overlord to offer segment allocation API ; It’s not meant for querying ; it basically ensures no 2 tasks are creating the same segment

Hi <@U04CGQ1P68Z>
thanks a lot for the info
is it mentioned in some doc as i was not able to find anywhere?

it's an internal implementation detail and that's why its not documented as such.

<https://github.com/apache/druid/pull/13854>

the most basic one is `DoubleSumBufferAggregator`, which does `SUM` of `double`

i would try starting there and see if that makes sense

<@U0344FW86DD> do you know if i can make my extension to be supported also in Sql?

yep, you can, you would need to define a SqlAggregator (`org.apache.druid.sql.calcite.aggregation.SqlAggregator`)

you could look at the datasketches extension for an example of how to do that in an extension

There are logs where it says task added to running then status updated task status failed intermittently after some time as failure

it can happen when a task is abandoned by its assigned worker somehow

i haven't seen this very often (or at all i think — i had to look up the msg in the code)

you might find some clues in overlord or MM logs

or maybe one of these processes is unstable for some reason (server going away?)

would love it

I think it would make sense as an InputSource?

sort of like the DruidInputSource: one that doesn't require a format. It would specify the iceberg table location and then figure out the format using iceberg APIs

Is there anything special that needs to be done for MSQ to use InputSources? I assume no as I thong see anything in DruidInputSource related to MSQ

nah nothing special, they can all be used through EXTERN

if we have to wire this input source to Catalog, then there may be extra work required. It can be done later though.

related or coincidence? <https://github.com/apache/druid/issues/13923>

sounds like there is a working extension, but not wired into msq yet maybe

though idk what special it would need tbh

commented <https://github.com/apache/druid/issues/13923#issuecomment-1466977105> and tagged you in <@U035FNDU6HH>

wondering if we'd like to add this to <https://github.com/apache/druid/issues/14157>?

We've recently experienced some drift in convention for the log and exception messages being added to the project.  I took some time to write down the convention so that it's easier to identify expectations: <https://github.com/apache/druid/pull/13916>

Thats amazing thanks, I'm getting your comments reviewed by the author :+1:

Cool. The developper seems to have replied to most! He’ll probably be able to address them all with some minor guidance

I added a comment to pr about a potential additional change. But also I'm running on 1.25 right now, what about the version of fabric needed updating? Everything seems okay to me, so far.

its unsupported for k8s 1.25, I also mentioned in the comments of the exact issue people experienced in k8s <https://github.com/apache/druid/pull/13804#discussion_r1112268989>

I ported your changes back to my druid and they worked. But I didn't require the fabric bump in version. I guess it doesn't really matter, doesn't hurt.

i don’t have a 1.25 cluster to test on, this is just debugging with some folks that were running this on 1.25, it makes sense from the documentation that the fabric client wouldn’t work as the way k8s tells you a job is complete has changed.

i'm not familiar with k8s but i can review from a generic project perspective— is that useful? (i would be trusting you that the k8s-specific stuff is good :slightly_smiling_face:)

it looks good to me. Though the code coverage check is failing because of lack of tests for DruidKubernetesClient class. And you have a dummy test specifically added to make code coverage happy.. If coverage is still failing, we don't need that dummy test either. Let me know if I missed something

that is not really a dummy test, it tests a getter i don;’t know how useful the test is, but it does “make code coverage happy”. The thing was that was making code coverage unhappy was the k8s client changed how they do deletes, they used to return a boolean, now it returns a list of objects and if that list is empty, the logic is the same as before.  Now the code has already been tested, but the code coverage tool thinks the logic has changed, but it hasn’t the client behavior is just different.  Don’t know how to test that and keep it useful.

since the code coverage is failing anyway, I suggest that this method be deleted. If the test has a purpose beyond making code coverage happy, then it should be renamed as such. When I see a test named "makeCodeCoverageHappy", I assume that its sole purpose is to get coverage passing.

you are right, it is meaningless, ill remove it and push up

<@U031A9D19NU> did you mean to :white_check_mark:  this one?
<@U04NHCE780G> — it looks like some conflicts crept in, mind fixing them?

<@U0344FW86DD> - will do once that test is removed.

IMO it's pretty annoying that we have to manually approve CI runs now — do people think we raise an infra ticket like this as well?

We should really get on the infra ticket route.  A PR  <https://github.com/apache/druid/pull/13934> which is raised by <@U03FPRT76AE> who contributes regularlty to druid should not block on a committer to approve CI runs.

Another PR in the same boat is <https://github.com/apache/druid/pull/13909> by <@U03NCEXDNJK>

Request the PMC to raise a similar ticket.

Could you please post in this thread: <https://lists.apache.org/thread/m1xq1wcgzd59l7g51btgd7npy4tws55f>

It'll help provide evidence that the project wants this, when we raise an Infra ticket

Hey Sergiu
Thanks for starting this discussion. This is the correct forum for starting this.

Yes I think druid as a project would benefit in moving to junit 5.

Yes please open up an issue: <https://github.com/apache/druid/issues> and add more details as to how are you planning to migrate.


Some points to think about:
1. Will it be a one shot migration as in you will completely remove junit 4 and move everything to junit 5. (it might be tough to review this)
2. If you one to migrate one module at a time, how would the migration look.
If you can mention the advantages of junit 5 over junit 4 in the same issue, that would help give context to PR reviewers.

Hey Karan,

Thank you for the response, I will come with a strategy for it and get back with an issue. My initial thought was to go with the second point but I haven't checked yet dependencies in between the tests, maybe rules ,or what else would be needed. In any case I will describe my thoughts in the issue.

Hey <@U0315MQKEHK> I already created a PR as suggested <https://github.com/apache/druid/issues/13948|here> is it possible to review it??

If you are looking to make more contributions, you can add a switch in SysMonitor to use `<https://github.com/oshi/oshi>` . Right now, SysMonitor uses Sigar which hasn't been upgraded in a while. And also doesn't work on ARM architecture machines.

Hi

I'm new to Druid dev, so might be missing something obvious...

I've been investigating the failure of this job: <https://github.com/apache/druid/actions/runs/4404793392/jobs/7795322362>

I can see there is a unit test that fails under `jdk8, sql-compat=true` but passes under `jdk8, sql-compat=false`

Trying to replicate that locally, I found this in `.github/workflows/reusable-unit-tests.yml`:
```if (${{ inputs.sql_compatibility }} == true); then
  echo "DRUID_USE_DEFAULT_VALUE_FOR_NULL=false" &gt;&gt; $GITHUB_ENV
else
  echo "DRUID_USE_DEFAULT_VALUE_FOR_NULL=true" &gt;&gt; $GITHUB_ENV
fi```
However, I'm seeing the test pass in a jdk8 container with this environment variable set: DRUID_USE_DEFAULT_VALUE_FOR_NULL=false

Might just be that I need to clear everything down, but I wanted to check if that env var is the only thing I should need to set to replicate `sql-compat=true` - or is there something else?

That PR was merged after the Druid 25 code freeze so it did not make it in there.

It will be available in the 26.0 release, planned for April.

If it’ll be widely available next month there’s no use in build an image from main branch, it’s best to wait a bit for it, thanks folks

We’re very eager to try window functions

Note that the code will be in 26.0 but the feature will still be undocumented/experimental

But it will be there for you to try and provide feedback on :slightly_smiling_face:

The experimentality and undocumentedness of the feature are because there's various known issues/gaps that we want to resolve prior to a proper release

FYI, I accidentally pushed a patch to master instead of a PR branch; I pushed a revert too so the effect should be nil. Just saying this in case anyone wonders where the odd commits come from

I know i have a blocker that I need to resolve to make the changes in <https://github.com/apache/druid/pull/13803> more compatible

which ive consolidated into another feature I’m trying to finish up asap

I am working on a big change to make the console work with the new schemaless ingest stuff... maybe you heard of it. I would call that a release blocker (I am almost done!)

It would be nice to get <https://github.com/apache/druid/pull/13804> merged.  It was closed by <@U04NHCE780G> but he has said that there is no opposition to giving write access on the branch/PR.

Without it Kubernetes 1.25+ will not work with apache druid using Kubernetes extensions

Here is the PR from my aforementioned comment: <https://github.com/apache/druid/pull/14017>. Would appreciate a review <@U030C8H59T8> (note that it is not as big as it looks, I just updated the snapshots with new sampler payloads)