Hello- I have 5 CSVs from S3 and postgres is my de...
# replication-ask-ai
n
Hello- I have 5 CSVs from S3 and postgres is my destination. The connection works great, but the result is just one table. How can I make sure each csv gets its own table?
k
New message text here
n
What if I had 100,000 CSVs though? @kapa.ai
k
New message text here
n
@kapa.ai I don’t actually have that many, but you said to make new streams for each csv file. Which would seem pretty redundant
k
A message from kapa.ai
New message text here
n
@kapa.ai can I replicate a stream?
k
A message from kapa.ai
New message text here
n
['Traceback (most recent call last):\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py", line 79, in _check_parse_record\n record = next(iter(parser.parse_records(stream.config, file, self.stream_reader, logger, discovered_schema=None)))\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/file_types/csv_parser.py", line 183, in parse_records\n for row in data_generator:\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/file_types/csv_parser.py", line 52, in read_data\n headers = self._get_headers(fp, config_format, dialect_name)\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/file_types/csv_parser.py", line 97, in _get_headers\n headers = list(next(reader))\n File "/usr/local/lib/python3.9/codecs.py", line 322, in decode\n (result, consumed) = self._buffer_decode(data, self.errors, final)\nUnicodeDecodeError: \'utf-8\' codec can\'t decode byte 0xcc in position 644: invalid continuation byte\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py", line 58, in check_availability_and_parsability\n self._check_parse_record(stream, files[0], logger)\n File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py", line 86, in _check_parse_record\n raise CheckAvailabilityError(FileBasedSourceError.ERROR_READING_FILE, stream=stream.name, file=file.uri) from exc\nairbyte_cdk.sources.file_based.exceptions.CheckAvailabilityError: Error opening file. Please check the credentials provided in the config and verify that they provide permission to read files. Contact Support if you need assistance.\nstream=s3_files_to_postgres file=BigSupplyCo_Orders.csv\n'] @kapa.ai
k
A message from kapa.ai
New message text here
n
@kapa.ai I checked the encoding, it also said utf-8
k
A message from kapa.ai
Sorry @Natalie Crow, I seem to be having a bit of trouble. Don't worry though, I've pinged my support folks and they will try to fix me up! 🧑‍💻 Perhaps try again in 5mins?