what is the point of the

DataType

enum?

@charles It feels like there is a bit of a disconnect between

DataType

,

AirbyteType

, and the JSON schema types

https://github.com/airbytehq/airbyte/pull/609#issuecomment-711545129 makes it sound like the worker should be able to handle valid JSON Schema.

Does that mean we should allow integrations to emit

integer

https://json-schema.org/understanding-json-schema/reference/numeric.html#numeric?

Currently that fails in the

jsonSchemaTypesToDataType

conversion because there is no

DataType.INTEGER

Should

jsonSchemaTypesToDataType

map integer to

DataType.NUMBER

?

And aren't

DataType

and

AirbyteType

completely redundant?

cool. so the workers all use airbyte protocol structs but the API / UI still using

io.airbyte.config.Schema

. that just wasn’t in scope for 0.2.0. we need to figure out our story about how to express the airbyte protocol schema in the UI at some point and get rid of

io.airbyte.config.Schema

honestly i don’t think updating how we do the the schema in the UI right now is terribly high priority (as long as its not heinously wrong).

for the worker to succeed it needs to be able to parse valid jsonschema input (which includes type: integer), so Shrif is adding a mapping from integer to number DataType

does that clear up

DataType

. tl;dr. as long as we have

io.airbyte.config.Schema

in API/UI we need to shim back and for between whatever the airbyteprotocl is and that.

okay. that’s fine.

if you want to just change datatype to use integer that’d probably be fine too? 🤷 do whatever will be easiest?