MySQL has a BIT (0, 1) column, but for BigQuery ai...
# ask-ai
s
MySQL has a BIT (0, 1) column, but for BigQuery airbyte is converting that column to Boolean (true and false). Is there any way to ensure that it remains 0 and 1 instead of true and false? any way i can make change on CustomMySQLTinyIntOneToBooleanConverter.java or any other way to fix this ?
k
Based on the information provided in the knowledge sources, it seems that Airbyte's MySQL source connector treats
TINYINT(1)
as boolean by default, which is in line with MySQL's documentation. This behavior is due to the fact that in MySQL,
BOOL
and
BOOLEAN
are synonyms for
TINYINT(1)
. A value of zero is considered false and nonzero values are considered true. However, there is a configuration option in the MySQL source connector that allows you to change this behavior. The option is called
do_not_treat_tiny_int_as_boolean
and if enabled, properties of type
TinyInt(1)
are output as
TinyInt
, and not boolean. This option is set to
false
by default. Unfortunately, there is no direct information available on how to apply this setting in the context of BigQuery. However, you might be able to use this setting to prevent the conversion of
TINYINT(1)
to boolean, and then handle the conversion to the desired format in BigQuery. Please note that any changes to the source code of the connectors, such as
CustomMySQLTinyIntOneToBooleanConverter.java
, should be done with caution and thorough testing, as it might affect the overall functionality of the connector. For more detailed information, you might want to check the following resources: - [MySQL Source Connector type tinyint(1) > Answers](https://discuss.airbyte.io/t/2171) - [Github Issue (closed): Issue pulling spec for custom source > Original post Part 7](https://github.com/airbytehq/airbyte/issues/37751) - [Documentation > Integrations > Sources > MySQL > Changelog Part 8](https://docs.airbyte.com/integrations/sources/mysql#changelog)
s
hi @kapa.ai MySQL has a BIT (0, 1) column, but for BigQuery airbyte is converting that column to Boolean (true and false). Is there any way to ensure that it remains 0 and 1 instead of true and false?
k
Airbyte's MySQL source connector maps the
bit(1)
MySQL data type to a boolean when synchronizing data, as per the [MySQL Data Type Mapping](https://docs.airbyte.com/integrations/sources/mysql#data-type-mapping) in the Airbyte documentation. This is why you're seeing the BIT (0, 1) column in MySQL being converted to Boolean (true and false) in BigQuery. Currently, there's no built-in way to change this behavior in Airbyte. However, you might be able to achieve this by customizing the normalization process. You can read more about transformations and normalization in the [Airbyte documentation](https://docs.airbyte.com/operator-guides/transformation-and-normalization/transformations-with-sql). Please note that custom transformations are not available in Airbyte Cloud as mentioned in one of the [forum discussions](https://discuss.airbyte.io/t/2104). If you need a specific mapping, you might need to export the normalization module of Airbyte and modify it as per your requirements. Remember that any changes to the normalization process should be done with caution, as it could affect the data synchronization process.