DBT roadmap items Hi guys, I have two questions r...
# ingestion
m
DBT roadmap items Hi guys, I have two questions regarding the roadmap planned for DBT. I think there is some work planned for the DBT source for the near future and I'd like to know if the following items are covered, or if they have even been raised before: 1- I have raised this earlier and never really got closure on it, but I would like to know if the DBT source meta_mapping could be modified to support multiple matches (see this thread). TL;DR I would like to have the possibility to do something like this (
match this or that or this...
):
Copy code
meta_mapping:
      data_tier:
        - match: "Bronze"
          operation: "add_term"
          config:
            term: "Bronze"
        - match: "Gold"
          operation: "add_term"
          config:
            term: "Gold"
        - match: "Silver"
          operation: "add_term"
          config:
            term: "Silver"
          term: "Silver"
The current implementation only performs the last
match
and discards the previous ones (in this example, only
Silver
would be considered). 2- The DBT model support
meta
fields for columns (see docs), but the current code seems to only support
meta
information in the DBTNode (not in DBTColumn). I would like to be able to map terms to columns and not only for datasets. Was that ever considered?
m
Hey @modern-monitor-81461 for the first item, it is already supported. Our docs got lost in the shuffle. About to fix them up. Here is the snippet that will work for your scenario.
Copy code
meta_mapping:
  data_tier:
    match: "Bronze|Silver|Gold"
    operation: "add_term"
     config:
       term: "{{ $match }}"
❤️ 1
in the above example if you do:
Copy code
match: ".*"
then any data_tier you set will create a glossary term by that name
for the second question: supporting
meta
mapping for columns, this would be a great improvement on the current feature!
We don't have it currently
m
Thanks @mammoth-bear-12532! For the second item, I'll have a discussion with my users and we might contribute this back since this missing feature might be impacting our business. I'll report back if we ever decide to contribute it.