Hi All, Could someone please confirm if ingestion ...
# ingestionc
curved-crayon-1929
03/16/2021, 5:16 AMHi All, Could someone please confirm if ingestion from Mongo DB is supported? if yes please help me with the respective YAML file thanks.
could someone help me as this is important for us to proceed further. Thanks
m
mammoth-bear-12532
03/16/2021, 4:05 PM
@curved-crayon-1929: I'm not aware of MongoDB support yet. @incalculable-ocean-74010 is working on a Druid integration here: https://github.com/linkedin/datahub/pull/2235 You can replicate that approach for Mongo.
i
incalculable-ocean-74010
03/16/2021, 4:25 PMNot sure if it will be trivial. What I did was use the Python 2.0 DB API spec for druid and wrap that in the objects that the metadata ingestion framework works in. Very similar to Postgres or even Hive. Mongo being a non-relational database may require a different approach
b
big-carpet-38439
03/16/2021, 4:34 PMwe need mongo 🙂 !!
m
mammoth-bear-12532
03/16/2021, 4:44 PM
Looks like this function would be the entry point -> https://pymongo.readthedocs.io/en/stable/api/pymongo/database.html#pymongo.database.Database.list_collection_names