https://datahubproject.io logo
Join Slack
Powered by
I ran `pip3 install --user -r common.txt -r snowfl...
# ingestion
l
I ran 
pip3 install --user -r common.txt -r snowflake.txt --no-cache-dir
before this
m
@gray-shoe-75895: you might be able to help with this
g
@lemon-analyst-37781 did the pip command run successfully?
l
Getting this
g
It seems to be installing the 
futures
package despite python 3.7 not needing it
what do you get if you run 
pip -V
?
l
getting back, checking
pip -v
Copy code
pip 20.3.4 from /usr/local/lib/python2.7/dist-packages/pip-20.3.4-py2.7.egg/pip (python 2.7)
g
Ah sorry I should have specified: pip3 -V
My current hypothesis is that pip3 is associated with a different version of python, and so python3.7 is unable to find the packages. You can get the correct pip by using 
python3.7 -m pip install ...
instead of 
pip3 install ...
l
Sure, let me try this
g
Is this an ubuntu machine? I believe you can get pip via 
sudo apt-get install python3.7-pip
l
I removed the version of confluent kafka, and pip3 install worked
but running the 
snowflake_etl.py
is throwing this error
not able to find the file specified in codebase
g
ah so the current etl scripts rely on generated files - running 
./gradlew build
in the root datahub directory should take care of that
l
let me try that
It worked, thanks @gray-shoe-75895
g
Awesome! Let me know if you have any other questions - the ingest scripts are quite finicky
🙌 1
Open in Slack
PreviousNext
Powered by