I have discovered a versioning in the programmatic pipelines feature. It's as follows: When running a pipeline with the SDK

pipeline = Pipeline.create(recipe)
pipeline.run()
pipeline.pretty_print_summary()

For example it throws this exception

if regex("LATERAL VIEW EXPLODE(col)"):
TypeError: 'str' object is not callable

This error comes from

sqllineage

because it uses the latest version of

sqlparse==0.4.4

pinning my version to

0.4.3

fixed the problem. I also noticed that the version of

sqllineage==1.3.6

uses the as present here I resolved it by moving my version of sqllineage to

1.4.2

. I am just putting this here in case anyone runs into this issue... I spent the better part of an hour or two getting to the bottom of this.

Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: 1️⃣ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? Yes button 2️⃣ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? Yes button

Hello Kwabena, thank you for the heads up ! @gray-shoe-75895 could you look into this?

We’re aware of this one - we fixed it in the latest 0.10.2.2 release