Hi i am getting the following error when i try to ...
# ingestion
h
Hi i am getting the following error when i try to push my own data following the instructions from the metadata-ingestion README. Is there any other step that im missing out on apart from the ones given ?
b
Seems like your message content doesn't match the schema. Could you share the exact message you're trying to send?
h
how am i supposed to define the schema of my table in datahub ? i have just given the path to a postgres database and run this code
s
If you're using rdbms_etl without any modification, I guess it will not work out of the box, you need to do implementation specific to postgresql
You can refer to mysql_etl as an example to extract metadata you want https://github.com/linkedin/datahub/blob/master/metadata-ingestion/mysql-etl/mysql_etl.py
mysql_etl already extracts dataset schema
@worried-nightfall-77549 to confirm ^
w
Yes, the rdbms_etl provides postgres connector for extracting the data, you can transform the metadata according to the MCE, sample is in the mysql_etl with schema, owner …etc
f
For me this was a python version + avro package version problem
Couldn't use python 3 in the end, but python 2.7 + requirements.txt does it