thats ok, we should definitely pay attention to lambda's and functional style coding while reviewing our code

readability >>>>>>>>>>>>> optimizing lines of code

Verbosity is a feature 🙂

👋 as discussed created a new channel to discuss text search for Pinot. #CFQUJ9HGE 🙇

Created a new channel to discuss objectType/compoundType support: #CFQGALEPP

Merged the auto reformat PR: https://github.com/apache/incubator-pinot/pull/3739 In case your current working branch has conflicts with this one (which is very likely), to resolve conflicts, after rebasing on this change: - For any conflict, pick the change in your branch - Reformat the file using IntelliJ after all conflicts resolved From now on, please follow the Pinot Style introduced in PR: https://github.com/apache/incubator-pinot/pull/3705 When reformatting enum, be cautious because sometimes the result is not what we expected (Mac and Linux IntelliJ seems to have different behavior)

Does this work on eclipse?

We will be deleting existing snapshot releases/tags to be Apache compliant. I will be running this for each tag that shows up for `git tag`:

git push --delete origin <tag>

. I just deleted v0.002 and confirmed its gone as expected. If you have concerns with this, please reach out before noon

I am starting the delete of existing tags/releases

^ is done. https://github.com/apache/incubator-pinot/releases

Does anyone know if Druid supports star-tree like indexing? I couldn't find anything specific in their documentation

No it doesn’t

Cool, thanks for confirming!

Druid has sketches which can support approximate counts

Yes that’s true.

<!here> We are going to change Pinot version to

0.1.0-SNAPSHOT

as part of apache release preparation. https://github.com/apache/incubator-pinot/pull/3772 1. If your build script is hardcoding on

0.016

version, please make the change. 2. For creating a binary distribution tarbell, u need to run

mvn install -Pbin-dist -DskipTests

.

which java version does the latest Pinot code support?

latest java version*

version 8

Thanks

Will pinot still create segments and segment metadata、dictionary and index for HDFS raw data which is already columnar like ORC/Parquet ?

Question regarding CombineService (merging blocks) - if two blocks have different schemas although the schemas are backwards compatible (i.e. columns are added), currently this seems to throw an exception :

CombineService:141 - Data schema inconsistency between merged block schema:

Is this expected ?

DataSchema -> isTypeCompatibleWith

seems to check this:

if (!Arrays.equals(_columnNames, anotherDataSchema._columnNames)) {

which makes me believe it doesn't handle schema evolution. Just checking if this is a real issue or am I doing something wrong

IIRC, this is not the schema of the table, but of the blocks (ie projected columns in the query)

Schema evolution + auto backfill is supported.

Before addiing this exception, I believe the results were incorret, were non-mergable block was silently dropped

so from the info log, it seems the second block has one more column (than the first). I'm guessing the query is probably doing something like 'select *' and hence we cannot merge ?

should the solution to such issues be - query specific columsn instead ? (just trying to understand how the end user should query such datasets)

We either propose to auto-backfill, or add time filter in queries to protect against one query spanning across segments that don't have all have columns in the query.

I see