:wave: hopefully an easy one… I’ve forked the data...
# troubleshoot
f
👋 hopefully an easy one… I’ve forked the datahub project and am trying to install it via poetry but couldn’t find docs on how to do this. I’ve tried the following in pyproject.toml but it’s definitely not right…
Copy code
[tool.poetry.dependencies]
python = "^3.9"
acryl-datahub = { git = "https://github.com/FundingCircle/datahub.git", subdirectory = "metadata-ingestion", branch = "master", extras = ["datahub-rest","dbt","glue","kafka","postgres","tableau"] }
I say definitely because the datahub cli is broken
Copy code
$ datahub

Traceback (most recent call last):
  File "/Users/yusuf.mahtab/repos/fc-datahub/.venv/bin/datahub", line 5, in <module>
    from datahub.entrypoints import main
  File "/Users/yusuf.mahtab/repos/fc-datahub/.venv/lib/python3.9/site-packages/datahub/entrypoints.py", line 10, in <module>
    from datahub.cli.check_cli import check
  File "/Users/yusuf.mahtab/repos/fc-datahub/.venv/lib/python3.9/site-packages/datahub/cli/check_cli.py", line 7, in <module>
    from datahub.cli.json_file import check_mce_file
  File "/Users/yusuf.mahtab/repos/fc-datahub/.venv/lib/python3.9/site-packages/datahub/cli/json_file.py", line 3, in <module>
    from datahub.ingestion.source.file import GenericFileSource
  File "/Users/yusuf.mahtab/repos/fc-datahub/.venv/lib/python3.9/site-packages/datahub/ingestion/source/file.py", line 17, in <module>
    from datahub.emitter.mcp import MetadataChangeProposalWrapper
  File "/Users/yusuf.mahtab/repos/fc-datahub/.venv/lib/python3.9/site-packages/datahub/emitter/mcp.py", line 5, in <module>
    from datahub.emitter.aspect import ASPECT_MAP, TIMESERIES_ASPECT_MAP
  File "/Users/yusuf.mahtab/repos/fc-datahub/.venv/lib/python3.9/site-packages/datahub/emitter/aspect.py", line 1, in <module>
    from datahub.metadata.schema_classes import ASPECT_CLASSES
ModuleNotFoundError: No module named 'datahub.metadata'
d
Hi @flat-engineer-75197! Can you confirm that you run
poetry add acryl-datahub
? pyproject.toml should be like the following:
Copy code
[tool.poetry.dependencies]
python = "3.10.5"
acryl-datahub = "^0.10.0.5"
b
the error you encountered is because parts of the acryl-datahub library needs a extra step to generate, ie
./gradlew :metadata-ingestion:codegen
, which is run as part of the ci/cd when publishing.
f
@delightful-ram-75848 that installs the original datahub library. I’m trying to install my own fork. @better-orange-49102 thanks, I figured I was missing parts of the library. To confirm, I need to build the entire project and publish it to some repository? I can’t simply point to a git repo can I?
b
Not unless that repo contain those missing files. I normally clone the repo on local PC, generate the metadata files then pip install -e on them
You're not building the whole project, just parts of it pertaining to the python library