DataHub #troubleshoot

late-bear-87552

01/04/2023, 10:32 AM

Copy code

8.1.1

microscopic-machine-90437

01/06/2023, 5:34 AM

image.png

bored-dentist-25467

01/06/2023, 10:18 AM

Hi, May I know how and where did we configure meta_mappings to create glossary terms in dbt project? I configured meta_mappings in schema.yml of dbt. But it doesn't created glossary terms in dataset itself. Below is my Screenshot of datahub and code. Thankyou in advance for your kind suggestions.

adamant-van-21355

01/06/2023, 1:49 PM

Hi everyone @gray-shoe-75895 @bulky-soccer-26729 and best for the new year ahead! We upgraded to the latest version v0.9.5 but the above column-lineage issue is still there for merged *Snowflake*&*DBT* entities. Any updates on the above? Is there an ETA for this column lineage game changer fix? (currently we can not use column lineage at all). Thanks 🙏🏼

powerful-cat-68806

01/09/2023, 8:31 AM

Hi team, I need help with this pls.

salmon-jackal-36326

01/10/2023, 10:15 PM

Managed to solve it (for docker-compose) following exactly that we have here https://datahubproject.io/docs/authentication/guides/jaas/

🎉 1

thanks bear 1

datahubbbb 1

melodic-dress-7431

01/11/2023, 11:25 AM

Screenshot 2023-01-11 at 4.53.40 PM.png

microscopic-honey-35198

01/17/2023, 3:55 PM

Hi! I would like to know if there has been any progress in here. I am getting a similar error when executing the Tableau ingestion job:

Copy code

ValueError: source produced an invalid metadata work unit: MetadataChangeProposalWrapper(entityType='chart', changeType='UPSERT',…

but only when the warning about NODE_LIMIT_EXCEEDED appears. If the same recipe is run adjusting the

page_size

option, the warning stops appearing and the error also. Any clue about this? Thanks!

powerful-cat-68806

01/18/2023, 9:21 AM

Hi team, Can someone help with this pls.?

👀 1

proud-policeman-19830

01/19/2023, 9:47 AM

Just looking through your GH and the docker files for both

mysql-setup

and

elasticsearch-setup

include the following

Copy code

ENV DOCKERIZE_VERSION v0.6.1
RUN apk add --no-cache curl jq tar bash coreutils \
    && curl -L <https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz> | tar -C

Note that the arch of

dockerize

is set to

amd64

, could this be the cause of issues on

arm64

that I'm seeing (not sure how it musters through on my

M1

however.)

proud-policeman-19830

01/19/2023, 1:35 PM

So, I cloned the repo, and hacked the

Dockerfiles

for

mysql-setup

and

elasticsearch-setup

to build dockerize inline using an extra stage (tagged and stored the docker images locally), modified to compose file to use these images and it now works.

bland-balloon-48379

01/20/2023, 8:45 PM

I'm restarting this thread because I'm now seeing one of these issues again; note that statements regarding a lack of logs and memory errors above no longer apply. This issue at hand is that I'm receiving a 503 error on the dataset lineage tab. This error is occurring for the

searchAcrossLineage

graphql query which is responding with a generic "Error 503 Service Unavailable" HTML page. Looking in the gms logs I don't see anything unusual, just a few REST calls with 200 codes and 2 calls to neo4j to load assertions. I will say the lineage visualization works fine though. However, I noticed it uses the

getEntityLineage

graphql query instead.

cool-fireman-87485

01/23/2023, 6:25 PM

Datahub lineage exp.mp4

astonishing-answer-96712

01/24/2023, 10:40 PM

Hi @rich-policeman-92383, upgrade to 0.9.6.1, then run the job please!

best-umbrella-88325

02/07/2023, 11:57 AM

Anyone can help me out with this please?

glamorous-elephant-17130

02/09/2023, 7:58 PM

Would be great if anyone can help identify if anything is amiss

powerful-cat-68806

02/20/2023, 5:51 PM

Hi again, Can someone pls. advise for it? I kinda stuck 😞

glamorous-elephant-17130

02/21/2023, 10:21 AM

Stuck in this badly for a week. Guys please help!

acceptable-nest-20465

02/21/2023, 7:18 PM

hi @astonishing-answer-96712 i am using elastic search only for infrastructure and want to ingest metadata from neo4j

acceptable-nest-20465

02/21/2023, 7:41 PM

thank you @orange-night-91387

bitter-lawyer-49179

02/22/2023, 2:20 PM

Bumping this up again! Kindly help 🙏

polite-actor-701

02/24/2023, 12:59 AM

As I transcribed the code, one of '{' came out. It's written properly in the actual code, and it's not a query problem. I wrote it above, sometimes the results come out right. Could you review the errors again?

polite-actor-701

02/27/2023, 2:31 AM

yes. ES is healthy.

rich-pager-68736

02/27/2023, 7:00 AM

Hi DataHub-Team, anybody able to help with these "Duplicate key EntityAspectIdentifier" errors? 🙏 They are still there 😩

polite-actor-701

03/02/2023, 12:48 AM

I didn't measure the exact time, but every morning when I access the server and run test.py for the first time, an error occurs. If I run it continuously after that, the error keeps occurring. But if I execute it in about 10 minutes, the normal answer comes out. I must solve this problem as soon as possible, would you help me?

polite-actor-701

03/03/2023, 4:08 AM

The problem is only with 'Search' GQL queries. Test Case 1: Success

Copy code

{ tag (urn: "urn:li:tag:AUM추출") { urn } }

Test Case 2: Sometimes Error occurs

Copy code

{
  search(input: {
      type: CORP_USER, query: "TESTER"
      }) {

    searchResults {
      entity {
        ... on Tag { urn }
    
      }
    }
  }
}

Test Case 3: Sometimes Error occurs

Copy code

{
  search(input: {
      type: DATASET, query: "TEST"
      }) {

    searchResults {
      entity {
        ... on Dataset { urn }
    
      }
    }
  }
}

cuddly-butcher-39945

03/03/2023, 3:55 PM

Thanks @gray-ocean-32209! That was it. How did you know it should be

type: CORP_GROUP

? I did not see this in the returned types when I ran this

__schema {

types {

name

description

specifiedByUrl

. I saw

Copy code

{
                    "name": "CorpGroup",
                    "description": "A DataHub Group entity, which represents a Person on the Metadata Entity Graph",
                    "specifiedByUrl": null
                },

Thanks again!

witty-motorcycle-52108

03/08/2023, 4:18 PM

hello! was hoping to check in to see if there was any news on this search regression from

v0.10.0

. i saw a PR has been opened about adjusting stop words and wasnt sure if it was related to this, but i see that PR is closed now.

steep-waitress-15973

03/10/2023, 4:29 AM

@better-orange-49102 same here. Is there any fix other than workaround (removing the broker volume each restart datahub)?

kind-lifeguard-14131

03/10/2023, 10:36 AM

so, all the relevant privileges as granted. I have created a view based on the SDATA within the database, ingested everything with the following flags included ( ignore_start_time_lineage: True include_tables: True include_table_lineage: True include_views: True include_view_lineage: True) Yet still nothing to see in Datahub. Anyone got an idea?