@big-carpet-38439 need help in understanding datahub actions code what I am able to figure out is that when an ingestion is ran it executes and pipeline in datahub actions that triggers executor_actions which gets the config and run the act method I am unable to get two things 1. how the below code is able to get the task config

# Build default task config
        local_task_config = TaskConfig(
            name="RUN_INGEST",
            type="acryl.executor.execution.sub_process_ingestion_task.SubProcessIngestionTask",
            configs=dict({}),
        )

2. How act method calls the library in metadata-ingestion module ? please let me know in case my understanding is wrong or I am missing something

1. What do you mean by "able to get the task config" here? Access the type? The TaskConfig class is imported at the top of the file.

from acryl.executor.execution.task import TaskConfig

2. Not sure which exact method you're referring to, but similarly there are imported classes from the metadata-ingestion module that are used in datahub-actions. 3. SubProcessIngestionTask is currently closed-source

@orange-night-91387 I am referring to

datahub-actions/src/datahub_actions/plugin/action/execution/executor_action.py

and method is

_build_executor_config

basically I am facing error

'Pipeline finished with failures\n',
           "2022-06-21 12:04:16.182181 [exec_id=f3eab6dd-5bb1-4135-b06c-129209ddb1e3] INFO: Failed to execute 'datahub ingest'",
           '2022-06-21 12:04:16.182458 [exec_id=f3eab6dd-5bb1-4135-b06c-129209ddb1e3] INFO: Caught exception EXECUTING '
           'task_id=f3eab6dd-5bb1-4135-b06c-129209ddb1e3, name=RUN_INGEST, stacktrace=Traceback (most recent call last):\n'
           '  File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/default_executor.py", line 121, in execute_task\n'
           '    self.event_loop.run_until_complete(task_future)\n'
           '  File "/usr/local/lib/python3.9/site-packages/nest_asyncio.py", line 89, in run_until_complete\n'
           '    return f.result()\n'
           '  File "/usr/local/lib/python3.9/asyncio/futures.py", line 201, in result\n'
           '    raise self._exception\n'
           '  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 256, in __step\n'
           '    result = coro.send(None)\n'
           '  File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 115, in execute\n'
           '    raise TaskError("Failed to execute \'datahub ingest\'")\n'
           "acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"]}

not sure which code it throwing this error, while debugging I got stuck at

sub_process_ingestion_task

can help me understand where the code actually resided

The stack trace gives you the location of the task:

/usr/local/lib/python3.9/site-packages/acryl/executor/execution/sub_process_ingestion_task.py

@orange-night-91387 That i know but i wanted the code in project or library, not path of file in docker image, but never-mind, I found the package where the code exists "acryl-executor>=0.0.3rc2"