Another issue while trying to ingest in another Li...
# getting-startedn
narrow-painting-12219
02/23/2021, 8:36 PMAnother issue while trying to ingest in another Linux env.:
Copy code
(venv) carr@carr-VirtualBox:~/Projetos/datahub/metadata-ingestion$ datahub ingest -c ./examples/recipes/example_to_datahub_rest.yml
Traceback (most recent call last):
File "/home/carr/Projetos/datahub/venv/bin/datahub", line 11, in <module>
load_entry_point('datahub', 'console_scripts', 'datahub')()
File "/home/carr/Projetos/datahub/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 480, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/home/carr/Projetos/datahub/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2693, in load_entry_point
return ep.load()
File "/home/carr/Projetos/datahub/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2324, in load
return self.resolve()
File "/home/carr/Projetos/datahub/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2330, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/home/carr/Projetos/datahub/metadata-ingestion/src/datahub/entrypoints.py", line 13, in <module>
from datahub.ingestion.run.pipeline import Pipeline
File "/home/carr/Projetos/datahub/metadata-ingestion/src/datahub/ingestion/run/pipeline.py", line 16, in <module>
from datahub.ingestion.source import source_class_mapping
File "/home/carr/Projetos/datahub/metadata-ingestion/src/datahub/ingestion/source/__init__.py", line 8, in <module>
from .ldap import LDAPSource
File "/home/carr/Projetos/datahub/metadata-ingestion/src/datahub/ingestion/source/ldap.py", line 4, in <module>
import ldap
ModuleNotFoundError: No module named 'ldap'm
mammoth-bear-12532
02/23/2021, 8:42 PM
@narrow-painting-12219: probably you need to install these additional requirements: https://github.com/linkedin/datahub/tree/master/metadata-ingestion#ldap-ldap
n
narrow-painting-12219
02/23/2021, 8:47 PMg
gray-shoe-75895
02/23/2021, 9:16 PMThis PR should fix the issue 🙂 - https://github.com/linkedin/datahub/pull/2134
n
narrow-painting-12219
02/24/2021, 2:06 AM@gray-shoe-75895 merged that branch with your postgis branch and ingestion succeeded, thanks
g
gray-shoe-75895
02/24/2021, 2:12 AMAwesome! I've raised a PR for the postgis branch here https://github.com/linkedin/datahub/pull/2136
n
narrow-painting-12219
02/24/2021, 2:13 AMDidn't tested as intended yet..got stuck on frontend container, which I'm trying to remove and quickstart.sh again 😅
g
gray-shoe-75895
02/24/2021, 2:14 AMAh yep the docker-compose stuff can be quite finicky - let me know if you need help with anything
n
narrow-painting-12219
02/24/2021, 2:15 AMDone! Removing container and running the script again did the magic
Now back to the GIS stuff 😄
narrow-painting-12219
02/24/2021, 2:58 AM@gray-shoe-75895:
Copy code
(venv) carr@carr-VirtualBox:~/Projetos/datahub/metadata-ingestion$ datahub ingest -c ./examples/recipes/my_postgis_to_datahub_rest.yml
[2021-02-23 23:56:41,621] DEBUG {datahub.entrypoints:64} - Using config: {'source': {'type': 'postgres', 'config': {'username': 'postgres', 'password': 'adm123', 'host_port': 'localhost:5000', 'database': 'postgres'}}}
1 validation error for PipelineConfig
sink
field required (type=value_error.missing)narrow-painting-12219
02/24/2021, 2:59 AMoh, nvm..forgot the sink
narrow-painting-12219
02/24/2021, 3:05 AMAwesome! I see the geometric details there 😄
🎉 1
narrow-painting-12219
02/24/2021, 3:06 AMI don't see Table Comment, is it supposed to be there, alongside Columns Comments?
g
gray-shoe-75895
02/24/2021, 4:11 AMAh we're not extracting the table comments right now - will fix that soon
👏 1
gray-shoe-75895
02/24/2021, 4:12 AMAnd also will be working on improving those error messages 🙂