Got the same question as < strong analyst 47204> columns don DataHub #ingestion

Got the same question as <@U0167RC5Z7F>, columns d...

miniature-ability-75189

07/01/2020, 8:46 AM

Got the same question as @strong-analyst-47204, columns don't appear after mysql metadata was ingested, although they do appear in the produced message

swift-account-97627

07/08/2020, 7:44 AM

After updating

sql-etl/common.py

to set the tableSchema to something truthy (i.e. not null or empty), the schema then showed up in the front-end.

swift-account-97627

07/08/2020, 7:47 AM

Note that

mysql-etl/mysql-etl.py

already does this.

swift-account-97627

07/08/2020, 7:51 AM

(Also, the doc comment on tableSchema says it's human-readable JSON, but I don't think it's actually parsed as JSON anywhere. The

mysql-etl

script sets it to a string representation of a row from the information_schemas table. I just set it to the SQL DDL definition of the table.)

steep-airplane-62865

07/14/2020, 8:18 AM

Thanks for helping hand @swift-account-97627. Would you be interested in sending PR for this?

swift-account-97627

07/20/2020, 12:47 PM

Hi @steep-airplane-62865. Yes, definitely, but I'm not sure what's the most correct value to put in this field (see my last comment above, and my comment on https://github.com/linkedin/datahub/issues/1728). Would you be able to advise?

swift-account-97627

07/20/2020, 12:48 PM

(Actually, I'll tag you into the discussion there...)

aloof-fall-4769

09/17/2020, 2:21 AM

@miniature-ability-75189 did you get a resolution to this issue? Actually I faced this issue for mysql and the python script needs a few changes.

Open in Slack

Previous Next