Second question, I have incoming data in this very...
# troubleshooting
d
Second question, I have incoming data in this very strange format which is:
Copy code
{
 "meta": { "id": 0, "ingestion_time": "time" },
 "data": [{"id": 0, "sensor": 1, "timestamp": time1}, {"id": 0, "sensor": 1, "timestamp": time2}, {"id": 0, "sensor": 1, "timestamp": time3}]
}
I'm really struggling with how to create table with this input data from a stream without doing it in two queries, so my first query ends up being something like:
Copy code
CREATE TABLE {0} (
                  meta ROW(`id` VARCHAR, `ingestion_time` INTEGER),
                  data ARRAY<ROW(`id` VARCHAR, `sensor` DOUBLE, `timestamp` STRING)>
               )
What I actually need in my tables is one row per data array entry. So I have this second query:
Copy code
SELECT items.id,
       items.sensor,
     TO_TIMESTAMP(replace(replace(items.timestamp, '+00:00', ''), 'T', ' '))
 FROM
     {1} CROSS JOIN UNNEST(data) as items (id, sensor, timestamp);
This seems really inefficient and complicated, is there a way I can improve this and hopefully put in a watermark/partitioning schema in the first query?