A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

~Hi, I am having problem of ingesting LDAP users that has `.` in their user name. GMS says it's ingested and posted but they are not searched in elasticsearch and can't find ES index (used Kibana to find them on ES)~

~Some users are ingested. their user names don't include `.` I am using datahub 0.8.6.~

Solved. After deleting rows in Postgres, was able to index in ES again.

image.png

Seems problem of elasticsearch. All users are all persisted in DB. but not for ES only.

<@U0286L6BDD2> : would like to understand exactly how things got messed up. So we can prevent these in the future. 

Hi, I think elasticsearch index corruption might be the reason.

I reduced elasticsearch nodes (3 -&gt; 1) by mistake. Since all datahub indices set 1 replica by default, some documents would have been lost.

After then, re-ingestion didn't work at all. Even though GMS says it's ingesting. I guess mismatch between elasticsearch document and postgres DB rows results in the situation

Ok, so user names with . in them is working correctly for you? 