https://linen.dev logo
Join Slack
Powered by
This message was deleted.
# general
s
This message was deleted.
us_ecom_orders_com101.csv
a
It looks like a bug. Can you file a github issue? https://github.com/apache/druid/issues/new/choose
m
Hey @Abhishek Agarwal could you post here a quick summary of how you determined that this is probably a bug? I'd be interested in applying that pattern when encountering other issues... 🤠
j
It looks like it's interpreting the lower bound as "1 preceding" instead of "1 following" ... here's a quick reproducible test case on wikipedia:
Copy code
with tmp as (
 select channel, flags, count(*) cnt
   from wikipedia
  group by 1, 2
)
select channel, flags, cnt,
       sum(cnt) over (order by channel rows between 1 following and 2 following) "range 1f2f",
       sum(cnt) over (order by channel rows between 1 preceding and 2 following) "range 1p2f"
  from tmp
@Mike Sherman, the only way I found this stuff is to create a simple test case with relatively low numbers that are unique enough that you can figure out by inspection what's going on. A few months back I hit an offset error with the lower bound in the framing clause, but I'm pretty sure that one was already fixed. This one is new to me.
Btw, this issue appears to be "following -> preceding", not an offset of 2 ... to test that I used a larger offset:
Copy code
with tmp as (
 select channel, flags, count(*) cnt
   from wikipedia
  group by 1, 2
)
select channel, flags, cnt,
       sum(cnt) over (order by channel rows between 3 following and 2 following) "range 1f2f",
       sum(cnt) over (order by channel rows between 3 preceding and 2 following) "range 1p2f"
  from tmp
m
Woo hoo, @John Kowtko now I have something to play with. Getting reproducible use cases with the wikipedia datasource is exactly the approach I was thinking of...
Hey @John Kowtko did you modify the default wikipedia? I don't have a column called 
flags
in mine. I can load new data, that should add flags if it's not there. Not urgent, just wanted to follow up since I'm getting a better handle on window functions and use cases like this are great for learning!

https://media.giphy.com/media/8dYmJ6Buo3lYY/giphy.gif▾

j
Hi Mike, I'm using the default load wizard, pulling from this data file: "https://druid.apache.org/data/wikipedia.json.gz" ... here's what I have showing in web console ...
👍 1
a
@Mike Sherman - I looked at the CSV and druid results were obviously wrong.
✅ 1
m
Ya, John, I see the flags when I load from external source in the console. My existing wikipedia was created by following a tutorial. I'll see if I can merge this version, should be interesting...
Okay @John Kowtko and @Abhishek Agarwal this is SO cool! I specified the new wikipedia json gzip file that John pointed me to, went into the console and did a Connect external data to get the data. The SQL to load the new wikipedia version was generated and I ran it and voila - updated wikipedia datasource. Show me another database where things are that easy...👏
🙌 3
s
r
Window sql query is generated through another tool so we don't have control as explain by you making foo table. Please let know when you will be able to implement negative offset
2 Views
Open in Slack
PreviousNext
Powered by