Hello Team, I am adding postgres db as ingestion source. I have deployed datahub in private kubernetes cluster. Getting error "could not find a version that satisfies the requirement wheel". Since the cluster is private, cannot install packages while adding the sources. Any idea which packages to be installed and to which docker image?

@brainy-tent-14503 Any thoughts on this? thinking

Are you using the UI ingestion or the datahub cli? The UI ingestion uses the actions pod, built here.

Hello @brainy-tent-14503 I am using UI ingestion

The design of the

actions

container is to take the datahub cli version from the UI’s configuration and install at runtime. This allows using pinned versions of older cli versions. Python doesn’t have a great way to list the dependencies exactly, however you should be able to match the version of the container with the version of the python cli package. Another way would be to use a python tool called

pipdeptree

after installing the desired version. An example of this with v0.10.4.3 is as follows

pipdeptree -p acryl-datahub
acryl-datahub==0.10.4.3
├── aiohttp [required: <4, installed: 3.8.4]
│   ├── aiosignal [required: >=1.1.2, installed: 1.3.1]
│   │   └── frozenlist [required: >=1.1.0, installed: 1.3.3]
│   ├── async-timeout [required: >=4.0.0a3,<5.0, installed: 4.0.2]
│   ├── attrs [required: >=17.3.0, installed: 23.1.0]
│   ├── charset-normalizer [required: >=2.0,<4.0, installed: 3.1.0]
│   ├── frozenlist [required: >=1.1.1, installed: 1.3.3]
│   ├── multidict [required: >=4.5,<7.0, installed: 6.0.4]
│   └── yarl [required: >=1.0,<2.0, installed: 1.9.2]
│       ├── idna [required: >=2.0, installed: 3.4]
│       └── multidict [required: >=4.0, installed: 6.0.4]
├── avro [required: >=1.10.2,<1.11, installed: 1.10.2]
├── avro-gen3 [required: ==0.7.10, installed: 0.7.10]
│   ├── avro [required: >=1.10, installed: 1.10.2]
│   ├── pytz [required: Any, installed: 2023.3]
│   ├── six [required: Any, installed: 1.16.0]
│   └── tzlocal [required: Any, installed: 5.0.1]
├── cached-property [required: Any, installed: 1.5.2]
├── click [required: >=7.1.2, installed: 8.1.3]
├── click-default-group [required: Any, installed: 1.2.2]
│   └── click [required: Any, installed: 8.1.3]
├── click-spinner [required: Any, installed: 0.1.10]
├── Deprecated [required: Any, installed: 1.2.14]
│   └── wrapt [required: >=1.10,<2, installed: 1.15.0]
├── docker [required: Any, installed: 6.1.3]
│   ├── packaging [required: >=14.0, installed: 23.1]
│   ├── requests [required: >=2.26.0, installed: 2.31.0]
│   │   ├── certifi [required: >=2017.4.17, installed: 2023.5.7]
│   │   ├── charset-normalizer [required: >=2,<4, installed: 3.1.0]
│   │   ├── idna [required: >=2.5,<4, installed: 3.4]
│   │   └── urllib3 [required: >=1.21.1,<3, installed: 2.0.3]
│   ├── urllib3 [required: >=1.26.0, installed: 2.0.3]
│   └── websocket-client [required: >=0.32.0, installed: 1.6.1]
├── entrypoints [required: Any, installed: 0.4]
├── expandvars [required: >=0.6.5, installed: 0.9.0]
├── humanfriendly [required: Any, installed: 10.0]
├── ijson [required: Any, installed: 3.2.2]
├── jsonref [required: Any, installed: 1.1.0]
├── jsonschema [required: Any, installed: 4.17.3]
│   ├── attrs [required: >=17.4.0, installed: 23.1.0]
│   └── pyrsistent [required: >=0.14.0,!=0.17.2,!=0.17.1,!=0.17.0, installed: 0.19.3]
├── mixpanel [required: >=4.9.0, installed: 4.10.0]
│   ├── requests [required: >=2.4.2, installed: 2.31.0]
│   │   ├── certifi [required: >=2017.4.17, installed: 2023.5.7]
│   │   ├── charset-normalizer [required: >=2,<4, installed: 3.1.0]
│   │   ├── idna [required: >=2.5,<4, installed: 3.4]
│   │   └── urllib3 [required: >=1.21.1,<3, installed: 2.0.3]
│   ├── six [required: >=1.9.0, installed: 1.16.0]
│   └── urllib3 [required: Any, installed: 2.0.3]
├── mypy-extensions [required: >=0.4.3, installed: 1.0.0]
├── packaging [required: Any, installed: 23.1]
├── progressbar2 [required: Any, installed: 4.2.0]
│   └── python-utils [required: >=3.0.0, installed: 3.7.0]
│       └── typing-extensions [required: >3.10.0.2, installed: 4.5.0]
├── psutil [required: >=5.8.0, installed: 5.9.5]
├── pydantic [required: >=1.5.1,<2,!=1.10.3, installed: 1.10.10]
│   └── typing-extensions [required: >=4.2.0, installed: 4.5.0]
├── python-dateutil [required: >=2.8.0, installed: 2.8.2]
│   └── six [required: >=1.5, installed: 1.16.0]
├── PyYAML [required: Any, installed: 6.0]
├── ratelimiter [required: Any, installed: 1.2.0.post0]
├── requests-file [required: Any, installed: 1.5.1]
│   ├── requests [required: >=1.0.0, installed: 2.31.0]
│   │   ├── certifi [required: >=2017.4.17, installed: 2023.5.7]
│   │   ├── charset-normalizer [required: >=2,<4, installed: 3.1.0]
│   │   ├── idna [required: >=2.5,<4, installed: 3.4]
│   │   └── urllib3 [required: >=1.21.1,<3, installed: 2.0.3]
│   └── six [required: Any, installed: 1.16.0]
├── ruamel.yaml [required: Any, installed: 0.17.32]
│   └── ruamel.yaml.clib [required: >=0.2.7, installed: 0.2.7]
├── sentry-sdk [required: Any, installed: 1.26.0]
│   ├── certifi [required: Any, installed: 2023.5.7]
│   └── urllib3 [required: >=1.26.11, installed: 2.0.3]
├── tabulate [required: Any, installed: 0.9.0]
├── termcolor [required: >=1.0.0, installed: 2.3.0]
├── toml [required: >=0.10.0, installed: 0.10.2]
├── typing-extensions [required: >=3.10.0.2,<4.6.0, installed: 4.5.0]
└── typing-inspect [required: Any, installed: 0.9.0]
    ├── mypy-extensions [required: >=0.3.0, installed: 1.0.0]
    └── typing-extensions [required: >=3.7.4, installed: 4.5.0]