Okay, pip3 got me a step further I think. I'm goi...
# ingestioni
icy-easter-2378
04/15/2021, 3:22 PMOkay, pip3 got me a step further I think. I'm going to push a little further there.
g
gray-shoe-75895
04/15/2021, 5:13 PMYes it should be pip3. To make sure the versions of python and pip match, I'd actually recommend using
python3 -m pipi
icy-easter-2378
04/15/2021, 8:38 PMI'm not familiar with that usage.
icy-easter-2378
04/15/2021, 8:39 PMIs that used when utilizing pip?
g
gray-shoe-75895
04/15/2021, 8:42 PMYep. It actually works with a fair number of other python projects as well, like
python3 -m blackpython3 -m datahubi
icy-easter-2378
04/15/2021, 9:05 PMI'm still a little confused
icy-easter-2378
04/15/2021, 9:05 PM Copy code
vmadmin@datahub-poc:~$ python3 -m acryl-datahub[mysql]
/usr/bin/python3: No module named acryl-datahub[mysql]g
gray-shoe-75895
04/15/2021, 9:06 PMtry
python3 -m pip install 'acryl-datahub[mysql]'i
icy-easter-2378
04/15/2021, 9:07 PMThanks for spelling it out for me 🙂
icy-easter-2378
04/16/2021, 1:04 PMHmm I messed up somewhere:
icy-easter-2378
04/16/2021, 1:04 PM Copy code
vmadmin@datahub-poc:~$ datahub ingest datahub-mysql.yaml
Traceback (most recent call last):
File "/home/vmadmin/.local/bin/datahub", line 7, in <module>
from datahub.entrypoints import datahub
File "/home/vmadmin/.local/lib/python3.6/site-packages/datahub/entrypoints.py", line 10, in <module>
from datahub.check.check_cli import check
File "/home/vmadmin/.local/lib/python3.6/site-packages/datahub/check/check_cli.py", line 8, in <module>
from datahub.ingestion.sink.sink_registry import sink_registry
File "/home/vmadmin/.local/lib/python3.6/site-packages/datahub/ingestion/sink/sink_registry.py", line 5, in <module>
sink_registry.load("datahub.ingestion.sink.plugins")
File "/home/vmadmin/.local/lib/python3.6/site-packages/datahub/ingestion/api/registry.py", line 55, in load
plugin_class = entry_point.load()
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2323, in load
self.require(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2346, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 783, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (avro-python3 file-.avro-VERSION.txt (/home/vmadmin/.local/lib/python3.6/site-packages), Requirement.parse('avro-python3>=1.8.2'))icy-easter-2378
04/16/2021, 2:04 PM Copy code
vmadmin@datahub-poc:~$ python3 -m pip check
avro-gen3 0.4.0 has requirement avro-python3>=1.8.0; python_version > "3.0", but you have avro-python3 file-.avro-VERSION.txt.
acryl-datahub 0.1.2 has requirement avro-python3>=1.8.2, but you have avro-python3 file-.avro-VERSION.txt.icy-easter-2378
04/16/2021, 2:04 PMI'm not good with python package management
g
gray-shoe-75895
04/16/2021, 6:19 PMAh I think you've run into a fairly esoteric bug - I think I've seen this one before as well. Can you try
Copy code
python3 -m pip uninstall avro-python3
python3 -m pip install --upgrade pip wheel setuptools
python3 -m pip install 'acryl-datahub[mysql]'python3 -m pip checki
icy-easter-2378
04/19/2021, 4:16 PM@gray-shoe-75895 you're brilliant. Apologies, I didn't see your response sooner. I did eventually work my way to the same conclusion, but the process would have been much smoother if I had seen your response before now 😄
g
gray-shoe-75895
04/19/2021, 5:41 PMGlad you figured it out! Although I’ve had that problem as well, I’m not quite sure what precise series of steps causes it - just curious if you have a reliable way to reproduce it?
i
icy-easter-2378
04/19/2021, 5:52 PMHmm I'm not sure. I think I initially installed using
pippip3python3 -m pipicy-easter-2378
04/19/2021, 5:53 PMI wouldn't be surprised if that sequence of events was problematic. Maybe using pip installed an older version that was not subsequently updated?
4 Views