Hello Team, Not able to see the CSV Datasource ??

Hi everyone, I have questions. Can we migrate data from a datahub to another datahub? and How to migrate it? Thank you

Hello, I've deployed a datahub custom action by mounting a configuration and definition file inside my

datahub actions

container. To run the action, I exec into the container and run

datahub actions -c my_custom_action.yaml

. This starts up the action successfully. Can I run this same actions from outside the container ? ie without the need to exec into the container? Is this the recommended way to deploy actions? I followed the documentation here https://github.com/acryldata/datahub-actions/blob/main/docs/guides/developing-an-action.md#step-2-installing-the-action. thank you

This message was deleted.

while sending great expectations results to the datahub, getting error like no dataset found

any possible causes / fixes can be done???

Hello. Can I use LDAP based authentication of the UI after running docker quickstart? I made my own custom jaas.conf file and added the path of it to the volume of the container "datahub-frontend-react" in docker-compose.yml located in the quickstart directory. After doing this, I did docker-compose up datahub-frontend-react container and check login of my company's ldap/ad account but failed to login. What am I doing wrong?

I am trying to connect DataHub to my Snowflake account, however when I try to test the connection I get the error attached in the image. The data I have entered for the connection is as follows: source: type: snowflake config: account_id: CYFGSMO.RJ16003 warehouse: COMPUTE_WH role: datahub_role include_table_lineage: true include_view_lineage: true profiling: enabled: true stateful_ingestion: enabled: true username: XXXXX password: XXXX Currently, I'm pretty much stuck, so I would appreciate any help....

Hello, currently i am starting datahub using this command: docker-compose images quickstart/docker-compose-without-neo4j.quickstart.yml as far as I am aware the yml file will fetch the newest version of each image due to the ${DATAHUB_VERSION:-head} present within this file for each image. Question: is there a way (command or something) to see what is the current value for those images? I would like to know value for each ${DATAHUB_VERSION:-head} in my docker-compose.yml file

Hi, could somebody explane me how tableau ingestion should work. I got two sheets in tableau first one published but second marked as unpublished. so even second sheet unpublished i saw this sheet in Daahub. is that correct? second thinkg is in deletion. do the sheets shoud be deleted from datahub after it deleted in tableau? many thanks for explanation.

Hello everyone! How can I do the same as on https://demo.datahubproject.io so that without authorization it can be viewed? I have a fresh installation via helm and site always redirect to /login?redirect_uri=%2F when opening the main page

Hi Team, we are on 0.8.45 version of Datahub, we notice that for the validations, the column is displayed as undefined...

Hi team, we’re trying to build our forked datahub and noticed that the gradle-avro-plugin is failing to resolve, looking at the gradle status page (https://status.gradle.com/?utm_source=embed) they’ve stopped redirecting to jcenter where necessary, which means the relatively old plugin version (0.8.1) is failing to resolve. Does anyone happening to be looking at this and have a workaround? The same problem exists with the builds in master btw

Hello, sorry to repeat myself concerning my issue with datahub actions. I have successfully mounted my custom actions in

etc/datahub/actions/conf/

folder in the

data hub-action

container. However, I get a

module not found

error when the container starts. I'm following the instructions here https://github.com/acryldata/datahub-actions/blob/main/docs/guides/developing-an-action.md#step-2-installing-the-action where the definition file and the configuration file has the same name and in the same folder. Is there something I'm missing? Any help will be appreciated. Thank you

Hi everyone, I am trying to use the dataPlatformInfo aspect and, but I receive a 500 error (I am ingesting it with python ). This is how I use it:

mcp = MetadataChangeProposalWrapper(
            entityType="dataPlatform",
            changeType=ChangeTypeClass.UPSERT,
            entityUrn=make_data_platform_urn("TestPlatform"),
            aspect=DataPlatformInfoClass(name="TestPlatformName",type=PlatformTypeClass.OTHERS,datasetNameDelimiter=".",displayName="TestPlatformDisplayName"))

Hello everyone, How does DataHub create the lineage for Redshift objects? In particular, I want to know where DataHub retrieved the information for the lineage attached between the s3 files and the Redshift table. Is there a particular view that is being ingested? Or is DataHub parsing the queries on the table?

Hi everyone, I attempting to run datahub docker quickstart with the --quickstart-compose-file option and I got the following error "Unable to run quickstart - the following issues were detected: - quickstart.sh or dev.sh is not running." This has previously worked for me but this time as I was trying to go from v0.9.0 to v0.9.2 it didn't. I opened the log file in the error message but its blank and empty so there nothing for me to go off of.

Hello everyone, I developed a metadata emitter to emit ML Models metadata from Model Cards defined by a YAML file. I'm testing with mock data and the execution is fine. The problem is that some metadata is ignored by the front end (they are not appearing on the UI). One example is the 'InstitutionalMemory' aspect. Any one know what it could be? images in thread

Hi! I am having trouble setting up policies to allow groupA to be able to access domainA. Globally, users are not able to see any of the data assets but when assigning a user to groupA and then setting up a policy to give them permission to access domainA I am still met with "unauthorized access". Any help is appreciated, thanks!

Howdy everyone - would you please lend me a hand ? I am now deploying datahub with EKS / MSK / ES. However i met a wired issue, which was when i turned on multi partitions for MSK(aws managed kafka)

kafka:
    bootstrap:
      server: "xxx"
    zookeeper:
      server: "xxx"
    ## For AWS MSK set this to a number larger than 1
    partitions: 3
    replicationFactor: 3

however, that led to datahub-gms ran abnormally, kept reporting issues as below, which resulted in unable to create AccessToken/Glossary etc. The only way to resolve this was to comment the

partitions & replicationFactor

. May i seek help here - any chance to enable multi partitions along with gms work properly ? Thanks.

Hello. I want to find one environmental variable "auth.jaas.enabled" to change it to be true. I wonder where it is. Since I use docker quickstart, I found the directory /plugins/auth/resources is empty. I really appreicate it if anyone can help me. (I found it in the application configuration of the container "datahub-frontend_react", but it is meaningless to change in the container since it automatically changed to false if I run datahub quickstart.)

Hello Folks, any suggestion on this issue ? Already tried debugging guide and found no issues with the setup. To enable profiling on s3 parquet dataset do I have to do any additional setup that i am missing ?

I'm trying to use the remote debugging, but inteillj just keeps getting connection reset. I had this issue before, but I forget how I fixed it. Has anyone seen a similar issue?

Hi all We are facing issue with login to to Datahub from some users after enabling Google Auth. Datahub version : v0.8.43 We followed this Doc to setup google auth in our Datahub https://datahubproject.io/docs/authentication/guides/sso/configure-oidc-react-google/ Google auth is working for some Users and not working for some Error from Front end pod is

13:47:27 [application-akka.actor.default-dispatcher-1588] ERROR auth.sso.oidc.OidcCallbackLogic - Unable to renew the session. The session store may not support this feature
13:47:29 [application-akka.actor.default-dispatcher-1614] ERROR auth.sso.oidc.OidcCallbackLogic - Unable to renew the session. The session store may not support this feature

current Settings

- name: AUTH_OIDC_ENABLED
      value: "true"
    - name: AUTH_OIDC_CLIENT_ID
      value:
    - name: AUTH_OIDC_CLIENT_SECRET
      value:
    - name: AUTH_OIDC_DISCOVERY_URI
      value: <https://accounts.google.com/.well-known/openid-configuration>
    - name: AUTH_OIDC_BASE_URL
      value: <https://URL>
    - name: AUTH_OIDC_SCOPE
      value: openid profile email
    - name: AUTH_OIDC_USER_NAME_CLAIM
      value: email
    - name: AUTH_OIDC_USER_NAME_CLAIM_REGEX
      value: ([^@]+)

Do we need to add any additional permissions in GCP ?

Hello Team!!! I'm trying to use CSV with the S3 Module, getting 0 exported events to the datahub, what is the issue, Attaching Screenshot FYI.

image.png

#troubleshoot Does DataHub supports Azure Blob CSV and AWS S3 CSV as Ingestion???

Hey! Hopefully this is the correct channel to ask this - We’re using SSO with Okta and would like to be able to hyperlink to a specific dataset in the UI, allowing the user upon clicking the link to be taken straight to that dataset. We’re finding that if the user isn’t already logged into Datahub via Okta, clicking the link and inputting their credentials into Okta redirects them to the home page of Datahub rather than the dataset. This isn’t the case without SSO, where after inputting their credentials into the Datahub modal it will resolve to the full path in the url and present the dataset in the UI. Is this intentional/by design?

Hi we are experiencing an issue where users are logged in as wrong user when using google auth and OIDC: When users are logging in using SSO they are logged in as incorrect user. This started after we upgrade to

v0.9.2

This is only happening for a subset of our users. I have checked our OIDC configuration but i cant see anything that would lead to this behaviour. Datahub front-end logs dont seem to record any relevant events either. Has anyone experienced this issue?

Folks, I see that this PR is merged on Nov 4th . If I install using “datahub docker quickstart” will i get this fix ?