Hello team, I would like to ask if DataHub current...
# ingestion
s
Hello team, I would like to ask if DataHub currently support lineage for DockerOperator from airflow, I've been looking into some example but I couldnt find any.
b
Hi @straight-noon-75819! I don't believe we do at this time. What type of lineage would you be hoping to extract?
s
I meant pretty much like what we're currently have in BashOperator.. but i have quite a bit of usecase we baked the database transformation inside docker containers themselves, so each team can do their own things. and I just hope we might have some easier ways to just add inlet, outlet for those operators
g
You can actually pass the inlets and outlets to any airflow operator, including the DockerOperator
s
yah I tried that, seem the datahub didnt pick it up.. im not sure why.. anyway, I'll try again in a bit
g
what version of airflow are you using?
happy to hop on a call and debug if that’s easier
s
I'm using custom docker image built on top of 2.0
thanks @gray-shoe-75895, i'll play w/ it in a bit and ping you if I need more help, I do appreciate it
1