Hi all, I'm new to Datahub & am struggling to...
# troubleshoot
f
Hi all, I'm new to Datahub & am struggling to understand how I can install the datahub plugins into a 2.4.3 MWAA Environment in AWS. Currently, I've tried adding the plugins I am interested in into the requirements.txt, however checking the requirement install logs, it is only grabbing and installing the airflow plugin and 'acryl-datahub', not necessarily the other plugins I have mentioned as sub-packages. Please see the below requirements.txt I am using for this:
Copy code
--constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.4.3/constraints-3.10.txt

acryl-datahub[glue, sagemaker, datahub-rest, datahub-kafka, datahub-business-glossary]==0.9.3
acryl-datahub-airflow-plugin==0.9.3
jq==1.4.1
I am unsure whether these plugins are supposed to be installed via the DAGs themselves etc, or if this method isn't supported. Any help would be appreciated.
🔍 1
📖 1
l
Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: ✅ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? ✅ button ✅ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? ✅ button Did you find a solution to your issue? ❌ Sorry you weren't able to find a solution. I'm sending you some tips on info you can provide to help the community troubleshoot. Whenever you feel your issue is solved, please react ✅ to your original message to let us know!
f
1.) 0.9.3 2.) MWAA Environment 3.)
Copy code
Successfully installed acryl-datahub-0.9.3 acryl-datahub-airflow-plugin-0.9.3 aiohttp-3.8.3 aiosignal-1.3.1 async-timeout-4.0.2 avro-1.10.2 avro-gen3-0.7.7 click-default-group-1.2.2 click-spinner-0.1.10 confluent-kafka-2.1.0 docker-6.0.1 entrypoints-0.4 expandvars-0.9.0 fastavro-1.7.0 frozenlist-1.3.3 humanfriendly-10.0 ijson-3.1.4 jq-1.4.1 lazy-object-proxy-1.8.0 mixpanel-4.10.0 multidict-6.0.2 mypy-extensions-0.4.3 progressbar2-4.2.0 pydantic-1.10.7 python-utils-3.5.2 pytz-deprecation-shim-0.1.0.post0 ratelimiter-1.2.0.post0 requests-file-1.5.1 requests-toolbelt-0.10.1 stackprinter-0.2.10 toml-0.10.2 typing-inspect-0.8.0 tzdata-2022.6 tzlocal-4.2 websocket-client-1.4.2 yarl-1.8.1
4.) Included requirements.txt for my environment in main message.
d
Hey @fancy-ambulance-73900, can you run & post result of
datahub check plugins
in your MWAA environment, to confirm which plugins are enabled? 🙂
f
Hey Hyejin, thanks for getting back to me. The customer I was working with has moved on with their progress on this 🙂 Thanks for reaching back out though.
h
@fancy-ambulance-73900 - Could you share instructions on how the sub-packages were installed ? We are trying to do the sam