Hey all! does anybody know if there’s an existing dbt package for the github source? I want to contribute but avoid starting from scratch if possible I’ve tried searching for it but if I Google “airbyte github dbt package” it just can’t handle those keywords 😂 “you mean like the github page for airbyte?” “oh you mean the dbt directory in airbytes github?” I mean something like fivetran’s: https://hub.getdbt.com/fivetran/github/latest/

Got it! https://github.com/CerebriumAI/airbyte_dbt_github buried in that post 🙌

There isn’t a official dbt packages for Airbyte

Time for me to get started then!!! cracks knuckles

@Michael Louis getting some traction! 🙂

^ right but the title of that blog is not actually what I wanted. What I was searching for was a pre-built package rather than a tutorial on how to make my own!

Right! That's the 3rd result on my screenshot but anyway curious to know if you try the Cerebrium package and have some feedback over how reusable is for your use case 🙂

@Oliver cerebrium have a package but it assumes airbyte’s basic normalisation. I've built one internally directly over the RAW json. Was thinking of publishing it but it all feels too opinionated IMO to see a one-size-fits-all

Hey @gunu, curious to know why you preferedto build it from the raw data instead of normalization? :)

I find the basic normalisation sometimes too bloated e.g. the automatic table generation of nested JSON fields. also I think the default deduping method could be optimised

@Edward Gao (Airbyte) fyi this discussion

@gunu - some more context, I was thinking of adding some dbt metrics + Lightdash charts/dashboards as code, That way somebody can spin up airbyte+dbt+Lightdash for an instant github analytics stack. I’m not sure which would be most useful for the community. But if not I’ll use our internal one too, which is also quite opinionated I guess ha Do most airbyte users use the normalisation or not?

cc @Simon Späti (Airbyte)

so many people cc’d on this thread, what is going on 😂

I think Simon is also building some Github analytics as a sample project to showcase Airbyte 😉

Cool! Can you share more @Simon Späti?

Hi @Oliver Laslett 🙂 Interesting thoughts with

dbt metrics + Lightdash

, I have similar ideas as part of my example MDS project with

airbyte+dbt+dagster+(cube.js or metricflow or dbt metrics)+(Metabase or superset)

. My plan: I start with airbyte and dbt and use the basic normalization that airbyte provides. I'd add additional transformation logic and cleaning on top with standard dbt and schedule it with dagster, which would look something like the attached (see article here). Not sure if that helps at all. Unfortunately, I do not have a repo or code yet to share, but that is something I will work on pretty soon.

@User check this discussion

Hey @Simon Späti was wondering if you had any further thoughts on this idea or whether you started working on it? Would love to continue the conversation, particularly as I was separately planning on meeting up with @Justin Chau (Airbyte) anyway to talk about content ideas like this!