This message was deleted.
# general
s
This message was deleted.
👀 1
v
this seems to be a timestamp in microseconds. Divide this by 1000 to get milisecond timestamp. If you are ingesting this field as __time than that should be enough. If you want to covert this to a string data the you can use timestamp_format
a
@Vijay Narayanan Yes. I am trying to ingest this as _time. when I select the date column for _time, it is showing another value not the actual date value.
v
does your column actually have 1539129600000000?
a
@Vijay Narayanan No, I actually gave an example. The actual BQ table has date values like "2022-01-01" and I exported table data to the GCS bucket using the BQ extract command: "bq extract --destination_format PARQUET ------". While ingesting this Parquet file, My date column data is showing as Int64, and due to that, the _time column is not getting with the proper date as shown in the previous screenshot. Hope you understood what my problem is.
🙌 1
v
if the value is like 1539129600000000 then you need to specify the format in ingestion as micro seconds.
a
@Vijay Narayanan I am pretty new to Druid. Could you please help me with where I need to specify.? Can we implement this formatting in the "*Parse Time*" section or Can I do it in Edit Spec Step?
v
you can do it in parse time.Just just micro from the format drop down
a
Sure. Let me try and get back to you.
g
note that in Druid there is no builtin timestamp type for columns other than
__time
you should store the timestamp as a
long
type, and then at query time, you can use functions like
MILLIS_TO_TIMESTAMP
to get a timestamp type and use time functions
there is no performance overhead to
MILLIS_TO_TIMESTAMP
so don't worry about that 🙂
it's best to store the number as milliseconds: so if it comes from parquet as millis, store it as-is; if it comes as microseconds, divide by 1000 (you can do this during ingest or query)
a
Thank you for your help @Gian Merlino and @Vijay Narayanan