Hello, is it possible to ingest multiple S3 buckets under one DBT source? I’m trying to achieve something like:

source:
    type: dbt
    config:
        aws_connection:
            aws_region: us-east-1
            aws_role: 'arn:aws:iam::************:role/DataHub-role'
        target_platform: s3
        manifest_path: 
		- '<s3://bucket1/manifest.json>'
		- '<s3://bucket2/manifest.json>'
        test_results_path: 
		- '<s3://bucket1/run_results.json>'
		- '<s3://bucket2/run_results.json>'
        sources_path: 
		- '<s3://bucket1/sources.json>'
		- '<s3://bucket2/sources.json>'
        catalog_path: 
		- '<s3://bucket1/catalog.json>'
		- '<s3://bucket2/catalog.json>'

Whenever I try to create a list of them (in

[]

brackets) ingestion stops working and throws:

Failed to configure source (dbt) due to \n'
           "\t\t'4 validation errors for DBTConfig\n"
           'manifest_path\n'
           '  str type expected (type=type_error.str)

but it’s perfectly fine with only one bucket configured. Am I missing something in my recipe?

The dbt source expects one each of manifest, catalog, run_results. If you want to ingest artifacts from multiple Dbt projects you need to run multiple recipes.

Understood, thank you