Hi everyone. New here and I was just wanting to as...
# troubleshooting
r
Hi everyone. New here and I was just wanting to ask some questions on ingesting arrays, that is
INT_ARRAY
,
FLOAT_ARRAY
,
TIMESTAMP_ARRAY
, etc. I have found that I can ingest using JSON all types as multi-valued dimension columns with the exception of
BOOLEAN
,
TIMESTAMP
, and
BYTES
. I believe that
JSON_ARRAY
isn't a valid type, but I wasn't sure about
BYTES_ARRAY
. If anyone is about and can shed some light, I would be very appreciative. More info present in-thread
For BYTES_ARRAY I get:
Copy code
java.lang.UnsupportedOperationException: Unsupported data type : BYTES
For BOOLEAN_ARRAY I get:
Copy code
java.lang.ClassCastException: class [Z cannot be cast to class java.lang.Integer ([Z and java.lang.Integer are in module java.base of loader 'bootstrap')
For TIMESTAMP_ARRAY I get:
Copy code
java.lang.ClassCastException: class java.sql.Timestamp cannot be cast to class java.lang.Long (java.sql.Timestamp is in module java.sql of loader 'platform'; java.lang.Long is in module java.base of loader 'bootstrap')
So far I have in my docker contained pinot DB a table with all other types in singular and multi-valued form. The schema for the failing types are:
Copy code
"dimensionFieldSpecs": [
    {
      "name": "gamesWon",
      "dataType": "BOOLEAN",
      "singleValueField": false
    },
    {
      "name": "datesPlayed",
      "dataType": "TIMESTAMP",
      "singleValueField": false
    },
    {
      "name": "rawArray",
      "dataType": "BYTES",
      "singleValueField": false
    }
and the ingestion lines for these columns are: row 1
Copy code
"datesPlayed": ["2020-01-01 10:45:28", "2020-02-01 10:45:28", "2020-03-01 10:45:28"],
    "gamesWon": [true, false, true],
    "rawArray": ["cd", "ef"]
row 2
Copy code
"datesPlayed":[] ,
    "gamesWon": [],
    "rawArray": []
I have also tried ingestion of
TIMESTAMP_ARRAY
as long-typed epoch values, but it makes no difference.
Oh, and I'm using the
org.apache.pinot.plugin.inputformat.json.JSONRecordReader
m
@Ryan Ruane do you mind creating an issue for this?
r
Not at all.
I assume that would be on https://github.com/apache/pinot/issues
m
Thanks