Ingesting JSON data into a realtime table. A field...
# general
j
Ingesting JSON data into a realtime table. A field in the JSON is a JSON string with leading spaces but is always numeric data otherwise:
Copy code
{ "account":"      123", .....}
If my realtime table defines the account column as DOUBLE, then the record loads with no issue -- the spaces appear to be ignored. However, if I define the column as INT then the record does not load. More troublesome, I can't find any error messages in any of the logs -- I would expect some kind of error message?
m
Thanks for reporting @Josh Highley, let me take a look at the code, and will get back to you
@Josh Highley I did a small experiment, Double.parseDouble can parse " 123", but Integer.parseInt throws NumberFormatException. I suspect that the exception is being swallowed. Either way, seems like a bug.
For a temporary work-around, is it possible for you to strip the leading spaces? And also file an issue, so we can fix this.
j
No, unfortunately, it's not practical for us to parse the record prior, modify the value, then write it out again.
using a Double column type will probably be our workaround
I submitted issue #6634
m
Thanks for submitting the issue.
k
@Mayank so jave trims the string for double parse but not for integer parse?
m
Yes
k
That’s bizarre. Should be a simple fix but might add perf overhead for things that are already trimmed may be try trim only on exception?
m
This was a standalone unit test that I did, I'll take a look at where in the code we do the type conversion a little later.
Copy code
Double.parseDouble("  123"); -> 123.0
Copy code
Integer.parseInt("   123"); -> NumberFormatException