Hi everyone - is it possible to restrict / manage the ingestion privileges? In other words: With my locally installed version of Datahub, I don't need to authenticate to ingest data into Datahub. Is there a possibility to require username/passwort authentication before metadata can be ingested?

if you're using the rest api sink, dun think there is any way to restrict ingestion, since gms is open for anyone to access. John mentioned before they use their own proxy to limit access to GMS. i think kafka sink can implement ACL though (not familiar with kafka, sorry) source: https://datahubspace.slack.com/archives/CUMUWQU66/p1628772070237500

Hey there! xL is correct. We use a lightweight proxy sitting in front of GMS to validate requests across an untrusted network. Currently the ingestion API provides no authentication mechanisms. That being said, it is something on the core team radar and is in the early stages of thinking currently. We are considering which mechanism for authenticating, as well what the source of truth for authentication credentials, will be. Depending on the approach we decide on the level of work required can vary significantly. Here are a few things we're considering: • File-based username / password authentication (file w/ username and hashed passwords provided at deploy time) • Store-based username / password authentication (db storing username and salted hashed passwords, more work) • Token-based authentication: mechanisms to grant and validate OAuth access + refresh token by the backend APIs, with initial login happening either over OIDC, LDAP, or one of the username / pass mechanisms noted above. Which approach would best suit your use case?

joining this thread with a big plus one as we are currently also at the point were we would need authentication on GMS to restrict the access 🙂

Hey @bland-orange-95847 - Currently in the OSS, there is no purpose. It is not actively read or validated anywhere. In our Cloud (hosted) offering, this token is validated by the proxy service mentioned above^^

Okay thanks. Good to know, cause we may use both, your hosted and OSS. So for now we may need to build a proxy as well and extend the RestEmitter to support auth

As long as your proxy can verify Authorization Bearer tokens, you should not need to make any changes to RestEmitter

yeah right. This would be an option as well 👍