Hi fellows I m trying to build a schema to accommodate a new Apache Pinot #general

Hi fellows, I'm trying to build a schema to accomm...

luanmorenomaciel

08/16/2022, 5:17 PM

Hi fellows, I'm trying to build a schema to accommodate a new table using the following structure

Copy code

"event_time": "2021-07-30 12:45:16.668010"

trying to issue the following pattern

Copy code

"dateTimeFieldSpecs": [{
    "name": "event_time",
    "dataType": "STRING",
    "format": "1:DAYS:SIMPLE_DATE_FORMAT:yyyy-MM-ddTHH:mm:ss.SSS-SS:SS",
    "granularity": "1:DAYS"
  }]

however getting the following issue

Copy code

{
  "code": 400,
  "error": "Invalid schema: sch_output_faust_enriched_rides. Reason: invalid datetime format: 1:DAYS:SIMPLE_DATE_FORMAT:yyyy-MM-ddTHH:mm:ss.SSS-SS:SS"
}

is there someone here available to shed some lights?

Neha Pawar

08/16/2022, 5:32 PM

there could be 2 issues: 1.

2021-07-30 12:45:16.668010

should translate to

yyyy-MM-dd HH:mm:ss.SSSSSS

. the format in yout spec looks different 2. if you’re indeed having a eventTime with “T” inside, you’d need to addsigle quotes around it.

luanmorenomaciel

08/16/2022, 5:37 PM

seems that Swagger is showing me a healthy status now

Copy code

"format": "1:MILLISECONDS:SIMPLE_DATE_FORMAT:yyyy-MM-dd'T'HH:mm:ss.SSSZ",

luanmorenomaciel

08/16/2022, 5:37 PM

seems to be working!

luanmorenomaciel

08/16/2022, 5:38 PM

Copy code

(.venv) ➜  use-case-uber-rides-analysis git:(main) ✗ curl -X GET "<http://localhost:9000/tables/realtime_output_faust_enriched_rides_events/status?type=realtime>" -H "accept: application/json"
{
  "ingestionStatus" : {
    "ingestionState" : "HEALTHY",
    "errorMessage" : ""
  }
}%                                                                                                                                                                                        
(.venv) ➜  use-case-uber-rides-analysis git:(main) ✗

Neha Pawar

08/16/2022, 6:36 PM

great!

luanmorenomaciel

08/16/2022, 8:47 PM

Hi @Neha Pawar I thought that would work but now look what happened when it's trying to build the segment

Copy code

Could not build segment
java.lang.IllegalArgumentException: Invalid format: "2022-08-16 10:44:04.034154" is malformed at " 10:44:04.034154"

this is the format i'm using

Copy code

"format": "1:MILLISECONDS:SIMPLE_DATE_FORMAT:yyyy-MM-dd'T'HH:mm:ss.SSSZ",

Neha Pawar

08/16/2022, 9:40 PM

according to your example event_time, format should be

Copy code

yyyy-MM-dd HH:mm:ss.SSSSSS

fyi: https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html

luanmorenomaciel

08/17/2022, 5:51 PM

It's working like a charm, thank you @Neha Pawar

Open in Slack

Previous Next