Hello. I run dbt in gitlab pipelines sometimes and...
# ask-elementary-ai
f
Hello. I run dbt in gitlab pipelines sometimes and I want to supply job information to elementary how can I do that
k
A message from kapa.ai
Hello! I can help you with supplying job information to Elementary when running dbt in GitLab pipelines. Elementary collects job metadata through environment variables or dbt vars. Here are your options: ## Using Environment Variables Elementary automatically collects these environment variables if they're set: -
ORCHESTRATOR
-
JOB_NAME
-
JOB_ID
-
JOB_URL
-
JOB_RUN_ID
-
JOB_RUN_URL
You can set these as [GitLab CI/CD variables](https://docs.gitlab.com/ee/ci/variables/) in your
.gitlab-ci.yml
file or in your GitLab project settings. ## Using dbt vars Alternatively, you can pass job information directly in your dbt command using the
--vars
flag:
Copy code
bash dbt run --vars '{"orchestrator": "gitlab", "job_name": "my_dbt_job"}'
## Example GitLab CI Configuration Here's how you might configure this in your `.gitlab-ci.yml`:
Copy code
yaml dbt_job:   script:     - export JOB_NAME="my_dbt_job"     - export ORCHESTRATOR="gitlab"     - dbt run
Or using dbt vars:
Copy code
yaml dbt_job:   script:     - dbt run --vars '{"orchestrator": "gitlab", "job_name": "my_dbt_job"}'
The job information will be stored in the
dbt_invocations
table in your Elementary schema, and you can also view it through the
job_run_results
view. For more details, see the [Collect jobs info from orchestrator](https://docs.elementary-data.com/oss/guides/collect-job-data) guide.
f
@kapa.ai which gitlab ci variables would you put in each of the env vars?
k
A message from kapa.ai
Based on the knowledge sources, I can provide guidance on mapping GitLab CI variables to Elementary's environment variables. ## GitLab CI Variable Mapping Here's how you can map GitLab's predefined CI/CD variables to Elementary's expected environment variables:
Copy code
yaml dbt_job:   script:     - export ORCHESTRATOR="gitlab"     - export JOB_NAME="$CI_JOB_NAME"     - export JOB_ID="$CI_PIPELINE_ID"     - export JOB_RUN_ID="$CI_JOB_ID"     - export JOB_URL="$CI_PIPELINE_URL"     - export JOB_RUN_URL="$CI_JOB_URL"     - dbt run
### Variable Explanations: - ORCHESTRATOR: Set to
"gitlab"
(static value) - JOB_NAME:
$CI_JOB_NAME
- The name of the job as defined in
.gitlab-ci.yml
- JOB_ID:
$CI_PIPELINE_ID
- The unique ID of the current pipeline - JOB_RUN_ID:
$CI_JOB_ID
- The unique ID of the current job - JOB_URL:
$CI_PIPELINE_URL
- The URL to the pipeline details page - JOB_RUN_URL:
$CI_JOB_URL
- The URL to the job details page These are GitLab's [predefined CI/CD variables](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html) that should be available in your pipeline by default. Note: While the knowledge sources don't explicitly list GitLab CI variable mappings, this approach follows the same pattern used for other orchestrators like [Jenkins](https://slack.com/archives/C02CTC89LAX/p1693381415910399), where users map orchestrator-specific variables to Elementary's expected environment variables.