This message was deleted.
# general
s
This message was deleted.
c
Here is my query:
Copy code
REPLACE INTO hdsc_data
				OVERWRITE WHERE __time BETWEEN TIMESTAMP '2024-02-20' AND TIMESTAMP '2024-02-28 00:00:00'
				WITH data AS (
				   SELECT
				    '1' AS EventID,
				    'Events' AS ImportType,
				    'CoreMan.LimitChange',
				    DomainGroup AS DomainGroup,
						   SendingDomain AS SendingDomain,
						   IPV4_STRINGIFY(CAST(SendingIp AS INT)) AS SendingIp,
						   FLOOR(__time TO DAY) AS __time,
						   SUM(cnt) FILTER (WHERE EventType='Sent') AS Sent,
						   SUM(cnt) FILTER (WHERE EventType='FirstOpened') AS Opens,
						   SUM(cnt) FILTER (WHERE EventType='FirstClicked') AS Clicks,
						   SUM(cnt) FILTER (WHERE EventType='FirstUnsubscribe') AS Unsubscribes,
						   SUM(cnt) FILTER (WHERE EventType='FirstEmailDisabled') AS Disabled,
						   SUM(Revenue) AS Revenue,
						   SUM(cnt) FILTER (WHERE EventType='FirstReportedSpam') AS Complaints,
						   SUM(cnt) FILTER (WHERE EventType='MailBlock') AS Blocks,
						   SUM(cnt) FILTER (WHERE EventType='HardBounce') AS HardBounces,
						   SUM(cnt) FILTER (WHERE EventType='SoftBounce') AS SoftBounces,
						   SUM(cnt) FILTER (WHERE EventType IN ('FirstOpened','Opened')) AS RawOpens,
						   SUM(cnt) FILTER (WHERE EventType IN ('FirstClicked','Clicked')) AS RawClicks,
						   SUM(cnt) FILTER (WHERE EventType IN('FirstUnsubscribe','Unsubscribe')) AS RawUnsubscribes,
						   SUM(cnt) FILTER (WHERE EventType IN ('FirstReportedSpam','ReportedSpam')) AS RawComplaints,
						   SUM(cnt) FILTER (WHERE EventType IN ('FirstEmailDisabled','EmailDisabled')) AS RawDisabled
				   FROM    "esp-revenue-v2-prod"
				   WHERE   DomainGroup IN ('Gmail')
							AND SendingDomain IN ('<http://example.net|example.net>')
							AND DeliveryType IN ('VirtualCampaignDelivery')
							AND __time BETWEEN TIMESTAMP '2024-02-20' AND TIMESTAMP '2024-02-28'
				   GROUP BY FLOOR(__time TO DAY), DomainGroup, SendingDomain, SendingIp
				)
				SELECT * FROM data
				PARTITIONED BY DAY
				CLUSTERED BY EventID, ImportType
s
BETWEEN is inclusive which means you are including the start of the next day. i think that may be the issue. Try TIME_IN_INTERVAL( "__time", "2024-02-20/P7D")
j
... or
OVERWRITE WHERE __time >= TIMESTAMP '2024-02-20' AND __time < TIMESTAMP '2024-02-28'
... and a corresponding interval filter on the Select statement
c
I tried these this morning and it doesn't allow TIME_IN_INTERVAL in the OVERWRITE clause, says
Unsupported operation in OVERWRITE WHERE clause: TIME_IN_INTERVAL
Trying it with just in the
WHERE
had the same original issue
I thought I had pasted the original error, which is:
Copy code
Error: Plan validation failed
OVERWRITE WHERE clause contains an interval [2024-02-20T00:00:00.000Z/2024-02-28T00:00:00.001Z] which is not aligned with PARTITIONED BY granularity {type=period, period=P1D, timeZone=UTC, origin=null}
org.apache.calcite.tools.ValidationException
Interestingly it has .001Z at the end of the range
Hmm, found the solution, it required this:
OVERWRITE WHERE __time >= TIMESTAMP '2024-02-20' AND __time < TIMESTAMP '2024-02-28 00:00:00'
j
You can probably leave off the 000000 on the upper bound. Also make sure your select statement uses the same time boundaries (excluding the end time)