hey, I have synthesized some data, and generated d...
# getting-startedt
Tanmay Varun
04/03/2023, 8:16 PMhey, I have synthesized some data, and generated datetimecolumn as millis since epoch,but i keep getting this error while trying to ingest data using LaunchIngestionJob
Copy code
Caused by: java.lang.IllegalStateException: Invalid segment start/end time: 292278994-08-17T07:12:55.807Z/292278994-08-17T07:12:55.807Z (in millis: 9223372036854775807/9223372036854775807) for time column: transactionDate, must be between: 1971-01-01T00:00:00.000Z/2071-01-01T00:00:00.000Zn
Navina
04/03/2023, 8:19 PMIt looks like your epoch is in nanoseconds??
9223372036854775807t
Tanmay Varun
04/03/2023, 8:20 PMbut sample timestamps from my data are 1680552598625
1680552598630
1680552598633
n
Navina
04/03/2023, 8:22 PMSo Your raw data is that and For some reason, Pinot read it as
922337203685477580?t
Tanmay Varun
04/03/2023, 8:22 PMyeah..
Tanmay Varun
04/03/2023, 8:23 PMi am using cmd
apache-pinot-0.12.1-bin/bin/pinot-admin.sh LaunchDataIngestionJob -jobSpecFile jobspec.ymlTanmay Varun
04/03/2023, 8:24 PMdatetime field in schema —
Copy code
"dateTimeFieldSpecs": [
{
"name": "transactionDate",
"dataType": "TIMESTAMP",
"format": "1:DAYS:EPOCH",
"granularity": "1:DAYS"
}
]Tanmay Varun
04/03/2023, 9:39 PMthnx @Xiang Fu for helping resolve this, issue was format in dateTimeFieldSpecs - it needed to be milliseconds instead of days.. thanks @Navina for pointing out the time unit discrepancy!
👍 2
2 Views