Hello I am new Pinot. I have a question on ingest ...
# pinot-dev
r
Hello I am new Pinot. I have a question on ingest transformations. I have a string field with time value in yyyy-mm-ddThhmmssZ in the input csv file. I want to use this for time dimension string value and use superset for chart generation. My questions 1. can use the string value of time dimension for chart generation (superset shows as varchar) ? 2. I created two other fields timee_epoch field with timestamp datatype and timestamp_long with long data type for epoch seconds. In the table schema I have used transformation functions to convert into fromEpochSecondsfromEpochSeconds(fromDateTime(dateTimeStr, ’yyyy-MM-dd’’T’‘HHmmssZ’)) . After data ingestion the derived fields are showing negative value and 1970 Jan 01 000000 for long ad timestamp fields, which indicating the null value. The string field has expected correct times. could some one tell what could have gone wrong and how to debug?
m
@Seunghyun
s
@Raveendra Yerraguntla from your table, I would assume that `dateTimeStr`field is available. Can you try the following?
Copy code
select fromEpochSecondsfromEpochSeconds(fromDateTime(dateTimeStr, 'yyyy-MM-dd''T''HH:mm:ssZ')) from T limit 10
The ingestion side code will apply the same function. If works in the pinot query, it should work in the ingestion config. I feel that there may be the issue with the quotes. https://docs.pinot.apache.org/users/user-guide-query/querying-pinot#identifier-vs-literal