salmon-area-51650
06/15/2022, 5:09 PMdbt
ingestion. The cronjob which is executing the integration was ok (no errors) but I cannot see dbt platform in the UI.
This is the configuration of the job:
source:
type: "dbt"
config:
# Coordinates
manifest_path: "<s3://bucket/manifest.json>"
catalog_path: "<s3://bucket/catalog.json>"
sources_path: "<s3://bucket/sources.json>"
aws_connection:
aws_region: "eu-west-2"
# Options
target_platform: "snowflake"
load_schemas: True # note: if this is disabled
env: STG
sink:
type: "datahub-rest"
config:
server: "<http://datahub-datahub-gms:8080>"
And this is the output of the job:
'failures': {},
'cli_version': '0.0.0+docker.b4bf1d4',
'cli_entry_location': '/usr/local/lib/python3.8/site-packages/datahub/__init__.py',
'py_version': '3.8.13 (default, May 28 2022, 14:23:53) \n[GCC 10.2.1 20210110]',
'py_exec_path': '/usr/local/bin/python',
'os_details': '...-glibc2.2.5',
'soft_deleted_stale_entities': []}
Sink (datahub-rest) report:
{'records_written': 2048,
'warnings': [],
'failures': [],
'downstream_start_time': datetime.datetime(2022, 6, 15, 16, 28, 19, 669215),
'downstream_end_time': datetime.datetime(2022, 6, 15, 16, 29, 23, 420550),
'downstream_total_latency_in_seconds': 63.751335,
'gms_version': 'v0.8.38'}
Pipeline finished with 1061 warnings in source producing 2048 workunits
And there is no any `dbt`platform in the UI
Any idea?
Thanks in advance!bulky-soccer-26729
06/15/2022, 5:27 PMsalmon-area-51650
06/16/2022, 5:20 AMdbt
platformsalmon-area-51650
06/16/2022, 9:48 AM