Hey Everyone 👋, I am not able to ingest any glossary terms with the new datahubbbb versions

0.8.40

and

0.8.41

, is this a known issue with these version ? Local docker testing works alright for

0.8.42

but since helm chart is not yet available, i can not use it.

Hey Kamal! We've heard this just recently actually. could you share any error logs you may be seeing with us?

Hey @bulky-soccer-26729 , There is no error, logs shows ingest ran alright but glossary terms were not updated on UI. I tried adding glossary terms manually on UI but new ones were not visible on UI using this method as well. Here are output logs ( i shortened logs by removing most of glossary terms)

Source (datahub-business-glossary) report:
{'workunits_produced': 49,
 'workunit_ids': [
         'urn:li:glossaryTerm:PII.Country',
         'urn:li:glossaryTerm:PII.Race',
         'urn:li:glossaryTerm:PII.Place of birth',
         'urn:li:glossaryTerm:PII.Religion',
         'urn:li:glossaryTerm:PII.Age Range'],
 'warnings': {},
 'failures': {},
 'cli_version': '0.8.41',
 'cli_entry_location': '/datahub/__init__.py',
 'py_version': '3.9.0 (default, Nov 25 2021, 20:20:38) \n[Clang 12.0.5 (clang-1205.0.22.11)]',
 'py_exec_path': '.venv/bin/python',
 'os_details': 'macOS-12.4-x86_64-i386-64bit'}
Sink (datahub-rest) report:
{'records_written': 49,
 'warnings': [],
 'failures': [],
 'downstream_start_time': datetime.datetime(2022, 8, 4, 19, 23, 12, 939647),
 'downstream_end_time': datetime.datetime(2022, 8, 4, 19, 23, 40, 526068),
 'downstream_total_latency_in_seconds': 27.586421,
 'gms_version': 'v0.8.41'}

very weird, so you can't even create through the UI either? are you able to go directly to a term page to see if it has data there? try going to

<http://localhost:3000/glossaryTerm/urn:li:glossaryTerm:PII.Country>

and see if you can see documentation or owners or any other data associated with it to see if it exists

Yes i see the terms when i use the path with correct urn, just these terms are not shown on UI.

when you create a term via the UI are you specifying a Parent?

I deleted all the terms hoping a fresh ingest might fix it, now the existing glossary terms are also not visible. So to answer your question, no i did not test with assigning a parent on UI.

okay gotcha that's good to know. I'd still try it without the spaces when ingesting - still thinking about this/looking into it for you

Feedback received, thanks for highlighting this. and thank you so much for your response and looking into it. 🙂 🚀

after ingesting - can you check your document store (mysql or whatever you use) to ensure that the terms you ingested show up there? we can dynamically render a page when you pass the urn into the URL so technically it's not a guarantee that checking the page means the data exists (unless you saw descriptions, owners, metadata like that)

Hey @bulky-soccer-26729, I work with @wooden-pencil-40912 and have taken a look into this. I can confirm in the

glossaryTermInfo

it is showing

parentNode

. I’ve upgraded our staging instance to

v0.8.43

, I think there’s a few issues at play with our data. 1. All groups and terms ingested via recipe are not showing in the UI but are in the metadata and if I rebuild indices they still do not show 2. I can add new groups and terms via UI but they do not immediately show, I have to rebuild indices each time Looking at the metadata the difference I’m seeing in

glossaryNodeInfo

and

glossaryTermInfo

is that there’s a

name

element on the ones I created in the UI, whereas these don’t exist in the ingested ones, maybe because we’re setting the id and name the same, the ones created in the UI have a UUID for the ID, as there’s no advanced property to update the ID. I’m seeing some fails in our actions pod, might be nothing but I’m wondering if this is what’s used to trigger the indices builds when we create/ingest them?

hey Peter! so sorry for not getting back to you until now as I was on vacation. However this is all very interesting and great information - thanks so much for digging into this. So can you confirm that you're able to get your glossary in a state that you want now that you've set PLATFORM_EVENT_TOPIC_NAME?