Hi team, Happy New Year. Please is there a way to change the default account details, especially the password?

Hey, Happy New Year to one and all! Is there any way I can add a root CA certificate into datahub (running via

datahub docker quickstart

)?

Hello, can datahub manage the metadata of elasticsearch 6.x? It is possible to test and manage the metadata of elasticsearch 7.x.

Please reply to the information as soon as possible from the development team, thank you very much.

Hi Team, We are trying to integrate Great Expectation with Datahub for Snowflake source. And following the steps as mentioned in : https://docs.greatexpectations.io/docs/integrations/integration_datahub/. On running the checkpoint, It is running fine but the Validation tab in Datahub is still disabled.

Any leads would be much appreciated.

Do we need to change any settings as well ?

I wonder if there are any parameters that can be used to configure the password for new user registration, such as limiting the password to no less than 8 digits, no special characters, one uppercase and lowercase letter, etc

3

Hi! Is it possible to model a hierarchy using 'user groups' in datahub? I have tried to Google but not able to find relevant documentation. Thanks.

Hi I am trying enable metadata service authentication and have no idea how docker works. Can you please someone send steps to set metadata service auth enabled to true without data loss

Hi! While using quickstart, is there a way to restart datahub without losing the data?

Hi there, I know it's possible to download a list of datasets with their URNs, is there any way to generate a list of the field URNs in a container? Maybe via graphql? Basically, looking for all columns in a container so I can update metadata values in bulk using the CSV enricher and NOT on a per-dataset basis. Thanks! https://datahubproject.io/docs/generated/ingestion/sources/csv/

Hello, I use acryl-datahub-airflow-plugin. The plugin imports all executed dags in Datahub. https://datahubproject.io/docs/lineage/airflow/#using-datahubs-airflow-lineage-plugin It is possible to filter ?

Hi Everyone, Came across this slack board after watching a presentation by @brainy-tent-14503 on managing metadata with protobuf. This "Shift Left" philosophy really resonates with our extended team. One question we are struggling with is how we can leverage probuf when they are generic and not specific to each data source. For example, we leverage opentelemetry and define the proto once but use it for multiple instruments from different systems that may have different classification levels. How would we leverage this solution elegantly? Really intrigued by DataHub as we form our next metadata strategy and pick supporting technology. Any insights on how the community solves this protobuf approach would be helpful. Thanks!

Hi Team I am getting unable to emit metadata to Datahub gms error even after python version and packages version match

Hey all, Is there a way to upload fields/tables/Schemas descriptions using an excel file?

heya if we are using a corp certificate is there a way in the CLI to skip verifying it?

Hi team! There is an option in DataHub to add a link (as a separate "attribute") to the description of the Term in glossary through the UI. But is there any way to be able to add links through the ingsting of the terms through CLI? And in case there is no such option, what is the simpliest way I can use in order to automate addition of the links for the terms?

Hi! Just starting out with DataHub and have a quick question on relationships, specifically in regards to the Owner attribute on Datasets. Its possible to set multiple owners of different types, but it is also possible to limit it so that you for example only have one Business Owner for a given dataset? Please note I have only spent a very limited amount of time, but could not easily find this info anywere 🙂 We would like to keep the ability to have multiple Data Stewards for example for a dataset, so the relationship limitation should only be for a specific owner type.

Hello Team, is it possible to change DEFAULT_SESSION_TTL_HOURS from configuration? reading the code, it seems we can define SESSION_TTL_CONFIG_PATH = "auth.session.ttlInHours"; somewhere in the deployment variables https://github.com/datahub-project/datahub/blob/bacc2f957bfd3214d9cf2c1f57035f56d5b1fc39/datahub-frontend/app/auth/AuthUtils.java#L44

Another question: does sqlalchemy ingestion support pulling native comments of database entities?

Hi all, I’m running the DataHub quickstart on my local machine, and I’m noticing that not all of the sources that are listed in the DataHub documentation are available for UI ingestion by default. Is there something I need to enable? Or does it mean that those sources are only able to be ingested through the CLI?

Hi all, i want ingestion with mongodb . i was config succeed done but when check dataset then have not data (just have schema) . Who can help me pls ? Thanks !!

Hello Team, May I know when will be the datahub next release?

Hey guys, I’m facing an issue on the web interface and wanted to know if anyone has a clue. On the top menu, when I click on the profile icon, should have the datahub’s tag, but shows “null”. Is there a parameter that should have been set during deployment for it to work? Where does this value come from? Does anyone know where in the github repository can I find the creation of this component? Thanks!!

Hi there, I’m looking for a little advice on an issue I’ve been working on for a few days now. I have Datahub setup using SSO (via OIDC from Azure AD). When the users login a user is created e.g. “Foo.Bar@company.com”. I am ingesting Azure AD data also to try and assign groups etc. but when the users come in they are lowercase (e.g. foo.bar@company.com) and the system creates two separate accounts… I have IgnoreCase set to true but it doesnt seem to help. Am I missing something? Two different accounts:

<https://ui.datahubdev.company.com/user/urn:li:corpuser:foo.bar@company.com>
<https://ui.datahubdev.company.com/user/urn:li:corpuser:Foo.Bar@company.com>

Azure.dhub.yaml

source:
  type: "azure-ad"
  config:
    client_id: ${AZUREAPPID}
    tenant_id: ${TENANTID}
    client_secret: ${AZUREAPPSECRET}
    redirect: "<https://login.microsoftonline.com/common/oauth2/nativeclient>"
    authority: "<https://login.microsoftonline.com/${TENANTID}/oauth2/authorize>"
    token_url: "<https://login.microsoftonline.com/${TENANTID}/oauth2/token>"
    graph_url: "<https://graph.microsoft.com/v1.0>"

    ingest_groups: True
    ingest_group_membership: True

    groups_pattern:
      allow:
        - "DataHub-.*"
      ignoreCase: true

    ingest_users: True
    users_pattern:
      allow:
        - "[A-z]*\\.[A-z0-9]*company\\.com"
      ignoreCase: true

sink:
  type: "datahub-kafka"
  config:
    connection:
      bootstrap: "${MSK1},${MSK2}"
      schema_registry_url: "http://${SCHEMAPODURL}:8081"

Hello... I am trying to add a new ingestion source Kafka but I am running into an error:

Command failed: Failed to find a registered source for type kafka: kafka '
2023-01-12 12:42:37            'is disabled due to an error in initialization\n'
2023-01-12 12:42:37            'Traceback (most recent call last):\n'
2023-01-12 12:42:37            '  File "/tmp/datahub/ingest/venv-kafka-0.9.5/lib/python3.10/site-packages/datahub/ingestion/api/registry.py", line 97, in '
2023-01-12 12:42:37            '_ensure_not_lazy\n'
2023-01-12 12:42:37            '    plugin_class = import_path(path)\n'
2023-01-12 12:42:37            '  File "/tmp/datahub/ingest/venv-kafka-0.9.5/lib/python3.10/site-packages/datahub/ingestion/api/registry.py", line 32, in import_path\n'
2023-01-12 12:42:37            '    item = importlib.import_module(module_name)\n'
2023-01-12 12:42:37            '  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module\n'
2023-01-12 12:42:37            '    return _bootstrap._gcd_import(name[level:], package, level)\n'
2023-01-12 12:42:37            '  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import\n'
2023-01-12 12:42:37            '  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load\n'
2023-01-12 12:42:37            '  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked\n'
2023-01-12 12:42:37            '  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked\n'
2023-01-12 12:42:37            '  File "<frozen importlib._bootstrap_external>", line 883, in exec_module\n'
2023-01-12 12:42:37            '  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed\n'
2023-01-12 12:42:37            '  File "/tmp/datahub/ingest/venv-kafka-0.9.5/lib/python3.10/site-packages/datahub/ingestion/source/kafka.py", line 11, in <module>\n'
2023-01-12 12:42:37            '    from confluent_kafka.admin import (\n'
2023-01-12 12:42:37            "ImportError: cannot import name 'ResourceType' from 'confluent_kafka.admin' "
2023-01-12 12:42:37            '(/tmp/datahub/ingest/venv-kafka-0.9.5/lib/python3.10/site-packages/confluent_kafka/admin/__init__.py)\n'
2023-01-12 12:42:37            '\n'
2023-01-12 12:42:37            'The above exception was the direct cause of the following exception:\n'

This is using the docker container set up

Hey, is it possible to assign tags to Glossary Terms?