<@U0344FW86DD> <@U030C8H59T8> Wanted to get some c...
# dev
j
@Gian Merlino @Clint Wylie Wanted to get some clarification on whether ingesting empty strings in both batch/streaming ingests causes the value to be inserted as
NULL
into the segment when
druid.generic.useDefaultValueForNull=true
. Is this the expected behavior? See the data loader photo attached, where it appears to show parsing row 3's
string_value
as an empty string. However, post-segment creation, I took a dump of the segment:
Copy code
{"__time":1704070860000,"title":"example_1","string_value":"some_value","long_value":1,"double_value":0.1,"float_value":0.2,"multi_value":["a","b","c"],"count":1,"double_value_doubleSum":0.1,"float_value_floatSum":0.2,"long_value_longSum":1}
{"__time":1704070920000,"title":"example_2","string_value":"another_value","long_value":2,"double_value":0.2,"float_value":0.3,"multi_value":["d","e","f"],"count":1,"double_value_doubleSum":0.2,"float_value_floatSum":0.3,"long_value_longSum":2}
{"__time":1704070980000,"title":"example_3","string_value":null,"long_value":0,"double_value":0.0,"float_value":0.0,"multi_value":null,"count":1,"double_value_doubleSum":0.0,"float_value_floatSum":0.0,"long_value_longSum":0}
{"__time":1704071040000,"title":"example_4","string_value":null,"long_value":0,"double_value":0.0,"float_value":0.0,"multi_value":null,"count":1,"double_value_doubleSum":0.0,"float_value_floatSum":0.0,"long_value_longSum":0}
you can see that in row 3, the
string_value
column has replaced what I'd expect to be an
""
with a
null
. This is running on version v31, with
druid.generic.useDefaultValueForNull=true
. I've tested that running on v33 with
druid.generic.useDefaultValueForNull=false
produces the expected result (
""
stored instead of
null
).
To make sense of the behavior above, from these docs, I read:
Copy code
String columns couldn't distinguish an empty string, '', from null. Therefore, Druid treated both values as interchangeable.
Does this also mean that
''
and
null
are both coerced to
null
at ingest time when
druid.generic.useDefaultValueForNull=true
?
g
With
druid.generic.useDefaultValueForNull=true
, empty strings and nulls both generally got mapped to
null
during ingest
👍 1
This no longer happens since we removed
druid.generic.useDefaultValueForNull
(behavior is locked to
false
now)