https://datahubproject.io logo
Join Slack
Powered by
Hello, I am having trouble getting the UI ingestio...
# troubleshoot
s
Hello, I am having trouble getting the UI ingestion working on datahub on docker-compose. Everything seems properly set up, but when I click on "RUN" it does not show me the output logs only the execution count increases with no status or outputs (image below). any suggestions? I used the the same recipe on datahub CLI 0.8.43.1 locally and the metadata was properly ingested. So it seems the problem is either in the UI or GMS. I'm using Datahub 0.8.43 I'll post the gms logs on the thread Edit: I'm using Google OIDC with 
METADATA_SERVICE_AUTH_ENABLED=true
GMS logs:
Copy code
9:54:39.917 [Thread-47243] WARN  c.l.m.s.e.q.r.SearchRequestHandler:444 - Found invalid filter field for entity search. Invalid or unrecognized facet ingestionSource
19:54:39.965 [qtp1830908236-18068] WARN  c.d.a.a.AuthenticatorChain:70 - Authentication chain failed to resolve a valid authentication. Errors: [(com.datahub.authentication.authenticator.DataHubSystemAuthenticator,Failed to authenticate inbound request: Provided credentials do not match known system client id & client secret. Check your configuration values...), (com.datahub.authentication.authenticator.DataHubTokenAuthenticator,Failed to authenticate inbound request: Authorization header missing 'Bearer' prefix.)]
19:54:40.198 [I/O dispatcher 1] INFO  c.l.m.s.e.update.BulkListener:28 - Successfully fed bulk request. Number of events: 4 Took time ms: -1
19:54:40.577 [Thread-47250] WARN  c.l.m.s.e.q.r.SearchRequestHandler:444 - Found invalid filter field for entity search. Invalid or unrecognized facet ingestionSource
19:54:40.577 [Thread-47249] WARN  c.l.m.s.e.q.r.SearchRequestHandler:444 - Found invalid filter field for entity search. Invalid or unrecognized facet ingestionSource
19:54:40.577 [Thread-47248] WARN  c.l.m.s.e.q.r.SearchRequestHandler:444 - Found invalid filter field for entity search. Invalid or unrecognized facet ingestionSource
19:54:40.621 [Thread-47255] WARN  c.l.m.s.e.q.r.SearchRequestHandler:444 - Found invalid filter field for entity search. Invalid or unrecognized facet ingestionSource
19:54:40.622 [Thread-47256] WARN  c.l.m.s.e.q.r.SearchRequestHandler:444 - Found invalid filter field for entity search. Invalid or unrecognized facet ingestionSource
19:54:41.125 [Thread-47259] WARN  c.l.m.s.e.q.r.SearchRequestHandler:444 - Found invalid filter field for entity search. Invalid or unrecognized facet ingestionSource
My recipe:
Copy code
sink:
    type: datahub-rest
    config:
        token: <MY_TOKEN>
        server: '<http://localhost:9002/api/gms>'
source:
    type: redshift
    config:
        table_lineage_mode: stl_scan_based
        include_table_lineage: true
        database: dev
        password: '${REDSHIFT_PASSWORD}'
        profiling:
            enabled: false
        host_port: 'REDSHIFT_HOST:5439'
        include_view_lineage: false
        stateful_ingestion:
            enabled: true
        username: '${REDSHIFT_USERNAME}'
b
hey! so what's most concerning to me here in your logs is this part:
Copy code
WARN  c.d.a.a.AuthenticatorChain:70 - Authentication chain failed to resolve a valid authentication. Errors: [(com.datahub.authentication.authenticator.DataHubSystemAuthenticator,Failed to authenticate inbound request: Provided credentials do not match known system client id & client secret. Check your configuration values...), (com.datahub.authentication.authenticator.DataHubTokenAuthenticator,Failed to authenticate inbound request: Authorization header missing 'Bearer' prefix.)]
i think when upgrading you experienced an issue with policies/authorization (and the invalid filter field for entity search is just a red herring)
s
Thank you @bulky-soccer-26729 I messed up the datahub-actions container by not properly setting the env vars for the secret so it could talk with the frontend.
b
Hi @boundless-sundown-7757 It appears to me that your DataHub System Client ID and System Client Secret config values are becoming inconsistent
Can you please a) make sure that your container tags for 
datahub-actions
, 
datahub-gms
, and 
datahub-frontend
are consistent? Let's start there
50 Views
Open in Slack
PreviousNext
Powered by