Hi team, is there is there any plan about supporti...
# feature-requestsn
numerous-planet-73590
10/19/2021, 7:34 AMHi team, is there is there any plan about supporting lineage of spark jobs/integration with openlineage? Thanks!
plus one 1
l
loud-island-88694
10/19/2021, 3:25 PM
We are working on providing Spark lineage support. ETA is about a month from now. Please stay tuned
🚀 2
🎉 2
n
numerous-planet-73590
10/19/2021, 4:41 PM❤️
r
refined-branch-44251
10/26/2021, 9:06 PM@loud-island-88694 would the spark lineage give fine grained (column/field level) lineage?
refined-branch-44251
10/26/2021, 9:07 PMalso would it work with delta tables?
l
loud-island-88694
10/26/2021, 9:07 PM
eventually there will be support for column-level lineage. we will start with table-level lineage
👍 1
loud-island-88694
10/26/2021, 9:08 PM
support for delta tables is on our roadmap for Q4. @mammoth-bear-12532 can comment more
l
little-megabyte-1074
12/01/2021, 12:40 AM
Hi folks! I’ve migrated this over to our new Feature Request board - you can upvote/provide additional context/subscribe to updates as work progresses.
• Spark Lineage https://feature-requests.datahubproject.io/b/feedback/p/spark-lineage
• Spark Delta Lake https://feature-requests.datahubproject.io/b/feedback/p/spark-delta-lake
2 Views