This message was deleted.
# general
s
This message was deleted.
b
The startTime and endTime need to be start and end of some months, eg '2023-01-01' and '2023-02-28'. It sounds like they're not?
Otherwise it's stopping you because bad things will happen with existing data.
m
Even i tried with just dates its still same
j
The end time needs to be 12 o'clock midnight of the following day, because it's an excluded endpoint, e.g.
2020-02-01:00:00:00 <= time < 2020-03-01:00:00:00
. (my syntax not exact)
m
Thanks @John Kowtko I am passing right window for the interval. However i would like to load the data into MONTH partitions where it is complaining that it not aligned with interval values. If i give
PARTITIONED BY HOUR
it works fine. I do not want to load this data with too many segments by hour
For Example :
Copy code
REPLACE INTO "${target} OVERWRITE
WHERE __time >= TIMESTAMP '2022-12-01 00:00:00' AND __time < TIMESTAMP '2023-03-30 00:00:00'
SELECT * from "${source}" WHERE __time >= TIMESTAMP '2022-12-01 00:00:00' AND __time < TIMESTAMP '2023-03-30 00:00:00'
PARTITIONED BY MONTH
j
This works for me (on one of my test datasets) --
Copy code
REPLACE INTO test_mo OVERWRITE
WHERE __time >= TIMESTAMP '2022-12-01 00:00:00' AND __time < TIMESTAMP '2023-07-01 00:00:00'
SELECT * from test_mo WHERE __time >= TIMESTAMP '2022-12-01 00:00:00' AND __time < TIMESTAMP '2023-07-30 00:00:00'
PARTITIONED BY MONTH
The OVERWRITE clause should have both dates be on the 1st of the month ... in the SELECT clause you can see my time range was shorter, but it still worked. Your example about had the OVERWRITE end date on 3/30, which is not correct, it should be 4/1. If you use 4/1 and it still doesn't work, then the only other thing I can think of that could be an issue is timezone? Did you change default timezone on your cluster? I don't know if timezone actually matters here or if all is assumed UTC, but it is a thought of what to check next.
m
Thanks @John Kowtko for getting back to me. Let me try with that and see what happens. Appreciate your help !!