@User can you give read access to anyone with the link?

So I have shared with apache-pinot-dev@googlegroups.com and few specific folks whose email id I know. I obviously want to share with everyone but the sharing link in google doc written in your linkedin account does not give "allow anyone with the link to view/edit". It only gives "allow anyone at LinkedIn with the link to view/edit"

Can we add people to apache-pinot-dev google group

is there any reason that this has to be internal to linkedin?

I have also shared with dev@pinot.apache.org but it looks like our mailing list doesn't give automatic access

if not, you can transfer the ownership to your personal apache id

I added the document to my personal id. Please read that document -- https://docs.google.com/document/d/19uLti7wwl7nPlDuy6cUVnLOll2C8u3YtUITbNj0TT5o/edit?usp=sharing

<!here> I have created a new PR after cleaning up the POC code (merging some orthogonal part of it as separate PR and making some additional changes based on the design doc) and addressing the review comments.

Few things : The current PR controls the creation and loading of Lucene text index through knobs in TableConfig (IndexingConfig), SegmentGeneratorConfig and IndexLoadingConfig. As per the design doc, we plan to implement a new data type TEXT. So once the segment creator sees it in the schema, it automatically creates the text index on that column. I had intended to do this change in follow-up PR (soon after) which will add the new data type and also remove the knobs from TableConfig.

@User and I discussed this and undoing anything from TableConfig is not allowed. So this is the tentative sequence of steps

(1) Add a PR that just adds the new type -- keep this disabled

(2) Rebase my current PR on this to leverage the new type.

One question is that if we want to ever create a text index on STRING/BYTES column then we will still need the knobs in TableConfig, IndexingConfig

Or we can merge this PR (with feature disabled) and do the data type as follow-up

lets make sure we agree on the desired state

are we saying the only way to enable text indexing is mark it as TEXT column?

Yes, assuming users are probably not going to run standard filter operations on such data and the column will only be used in text_match filters

However, having text indexing support on existing STRING/BYTES column will let people use this feature without adding a new column and just a simple table config change.

we have never done anything like that until now, for e.g. marking something as dimension, metric, timefield does not automatically add indexes

👋 I’m here! What’d I miss?

should we change the topic of this channel and point to the docs now?

@User has left the channel

Hello, My text index somehow stopped working. it is now giving intermittent results. For eg: following is working

select * from mytable where regexp_likg(log, '0D82F520-62C8-9914-14B8-4C2331E54075')

But this one will not

select * from mytable where text_match('0D82F520-62C8-9914-14B8-4C2331E54075')

any pointers how to debug?

Can you post this on #C011C9JHN7R

ok will do

@User has left the channel

@User has left the channel