https://linen.dev logo
Join Slack
Powered by
This message was deleted.
# general
s
This message was deleted.
c
a few questions, first, wondering how deeply nested was your data, was increasing the ulimit not an option?
just asking because nested columns are ever improving and i’m always looking for feedback to better support more use cases
besides that though, how did you end up storing it in string columns?
i imagine if just directly specifying string as the dimension type, some stuff might have come out.. strange
if you really want to store nested data in a string column, i would recommend using the 
to_json_string
expression in a transform to stringify the nested inputs
there is a 
try_parse_json
which can be used to turn it back into json types at query time if you wish to still use expressions like 
json_value
, but it would be significantly slower than if you were storing the data in nested columns
i’m not sure 
try_parse_json
would work very well if the data wasn’t run through 
to_json_string
on the input side
s
@Clint Wylie yes I was directly specifying string as the dimension type and thus was getting strange values in conversion. But with 
to_json_string
in expression work well for me. Thanks 👍
🎉 1
I am also observing another strange behavior, I ran Kafka ingestion and received 100k+ records yesterday and was able to query them. But today on UI console->Datasources, its showing 
Total rows
is 0 but 
Total data size
is still showing 69 MB. Also this datasource is not available on Query page. What could be the potential root cause for this?
Also under 
Availability details
on console its showing "1 segment to load"
c
is the realtime task still running perhaps related to the segment still loading? i suppose there might be some configuration issues with the historical servers loading the segments from deep storage and the task hasn’t successfully handed off the segment yet (it waits until it is loaded to continue serving queries)
s
Yes realtime task is in Running status
Is it possible to check somewhere (error log) related to, if loading segments from deep storage is failing
c
if the segment is in deep storage (and so published by the task) then the historical logs might help determine what is going on
s
ok will check
Open in Slack
PreviousNext
Powered by