@plain-balloon-39336 we probably want to look at the api for this use case at least then

@elegant-house-93198 so without the api its just reads the file on an ad-hoc basis. I inspected the file and i dont think they are any connection details and on prem so not worried about the IP address issues. But i did try uploading it and got some errors 😛

Dbt manifest extractor: 

'NoneType' object is not subscriptable

Traceback (most recent call last):

  File "/app/databuilder/job/job.py", line 75, in _launch_job

    self.task.run()

  File "/app/databuilder/task/task.py", line 67, in run

    record = self.extractor.extract()

  File "/app/databuilder/extractor/database/dbt/dbt_core_extractor.py", line 149, in extract

    return next(self._extract_iter)

  File "/app/databuilder/extractor/database/dbt/dbt_core_extractor.py", line 225, in _get_extract_iter

    manifest_list = get_manifest_list_from_machine_iam_role(

  File "/app/databuilder/utils/s3.py", line 78, in get_manifest_list_from_machine_iam_role

    if bucket_name[-1] != "/":

TypeError: 'NoneType' object is not subscriptable

Initialization: 

Job extraction failed, Error msg: 'NoneType' object is not subscriptable

Traceback (most recent call last):

  File "/app/databuilder/job/job.py", line 75, in _launch_job

    self.task.run()

  File "/app/databuilder/task/task.py", line 67, in run

    record = self.extractor.extract()

  File "/app/databuilder/extractor/database/dbt/dbt_core_extractor.py", line 149, in extract

    return next(self._extract_iter)

  File "/app/databuilder/extractor/database/dbt/dbt_core_extractor.py", line 225, in _get_extract_iter

    manifest_list = get_manifest_list_from_machine_iam_role(

  File "/app/databuilder/utils/s3.py", line 78, in get_manifest_list_from_machine_iam_role

    if bucket_name[-1] != "/":

TypeError: 'NoneType' object is not subscriptable



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

  File "/app/databuilder/models.py", line 105, in process

    getattr(Extractors, integration_type)(

  File "/app/databuilder/extract/database/dbt/dbt_extract.py", line 117, in _extract_dbt_core

    metadata_job.launch()

  File "/app/databuilder/job/job.py", line 123, in launch

    self._launch_job(

  File "/app/databuilder/job/job.py", line 87, in _launch_job

    raise JobFailedError(f"Job extraction failed, Error msg: {e}")

databuilder.utils.exceptions.JobFailedError: Job extraction failed, Error msg: 'NoneType' object is not subscriptable