This message was deleted.
# ask-ai
s
This message was deleted.
k
Yes, you can set up Airbyte to import parquet files from Google Cloud Storage (GCS). Here are the steps to do it: 1. Set up the GCS source in Airbyte UI: - Paste the service account JSON key to the
Service Account Information
field. - Enter your GCS bucket name to the
Bucket
field. - Add a stream: - Give a Name to the stream. - In the Format box, select the format of the files you'd like to replicate. In your case, it would be Parquet. - Optionally, enter the Globs which dictates which files to be synced. This is a regular expression that allows Airbyte to pattern match the specific files to replicate. If you are replicating all the files within your bucket, use
**
as the pattern. - (Optional) - If you want to enforce a specific schema, you can enter a Input schema. By default, this value is set to
{}
and will automatically infer the schema from the file(s) you are replicating. - Configure the optional Start Date parameter that marks a starting date and time in UTC for data replication. Any files that have not been modified since this specified date/time will not be replicated. - Click Set up source and wait for the tests to complete. Please refer to the [GCS source setup guide](https://docs.airbyte.com/integrations/sources/gcs#set-up-the-source-in-airbyte-ui) for more detailed instructions. 2. Once you have set up the GCS source, you can then set up a destination where you want to import the data to. Airbyte supports a large number of destinations, including cloud data lakes like Amazon S3, Google Cloud Storage (GCS), and Azure Data Lake Storage (ADLS). You can refer to the [Airbyte documentation](https://docs.airbyte.com/integrations/destinations) on how to set up your destination. Please note that the data stream is first converted to an Avro schema, then the Json object is converted to an Avro record, and finally the Avro record is outputted to the Parquet format. You can learn more about how source data is converted to Avro and the current limitations [here](https://docs.airbyte.com/understanding-airbyte/json-avro-conversion).