hallowed-gpu-49827
02/10/2022, 12:33 PMupstream connect error or disconnect/reset before headers. reset reason: connection termination/
The logs from frontend is:
datahub-frontend 12:27:41 [application-akka.actor.default-dispatcher-14] WARN akka.actor.ActorSystemImpl - Explicitly set HTTP header 'Content-Type: text/plain'
is ignored, explicit `Content-Type` header is not allowed. Set `HttpResponse.entity.contentType` instead.
datahub-frontend 12:27:41 [application-akka.actor.default-dispatcher-14] WARN akka.actor.ActorSystemImpl - Explicitly set HTTP header 'Content-Length: 95' is ign
ored, explicit `Content-Length` header is not allowed. Use the appropriate HttpEntity subtype.
gifted-queen-61023
02/10/2022, 3:20 PMCURL
POST
operation. When I define a MCE an try to use datahub ingest -c
, the following error arises:
• AttributeError: module 'avro.schema' has no attribute 'AvroException'
Not sure how to solve it.
Appreciate any help.
Thanks in advance 🙂numerous-eve-42142
02/10/2022, 6:00 PMnutritious-bird-77396
02/10/2022, 6:40 PM0.8.26
internally.
As i understand a new service datahub-actions
has been added to execute Datahub CLI and push events to Kafka
I see only the Docker env file but the Docker scripts or the project is missing...
The reason why i ask this is I would want to add MSK IAM Auth Jar to the project inorder for it to communicate with MSK.
Any help would be great. Thanks!strong-iron-17184
02/10/2022, 6:46 PMadorable-sandwich-55776
02/10/2022, 7:46 PMdatahub delete --urn 'urn:li:dataset:(urn:li:dataPlatform:kafka,-l,PROD)' --hard
, but the command line interface hangs adoes not exit. Is this a bug?
$ datahub delete --urn 'urn:li:dataset:(urn:li:dataPlatform:kafka,-l,PROD)' --hard
This will permanently delete data from DataHub. Do you want to continue? [y/N]: y
[2022-02-10 19:42:37,697] INFO {datahub.cli.delete_cli:126} - DataHub configured with <http://localhost:8080>
(... nothing else is printed, but it does not return)
ambitious-guitar-89068
02/11/2022, 4:49 AMnarrow-bird-99605
02/11/2022, 10:52 AMFailed to add owner: Unauthorized to perform this action. Please contact your DataHub administrator.
. I am performing this action on the test cluster and I am using root user datahub
. How can I find out what permission is missing and/or more detailed logs of the issue?
Thanks in advance!damp-queen-61493
02/11/2022, 7:54 PMgms.graphql
? Has this file been replaced by entity.graphql
?mysterious-lamp-73086
02/11/2022, 10:28 PMdamp-minister-31834
02/12/2022, 2:29 AMmodern-artist-55754
02/13/2022, 1:58 PMfew-air-56117
02/14/2022, 8:44 AMsource:
type: bigquery-usage
config:
# Coordinates
projects:
- <project_1>
- <project_2>
sink:
type: "datahub-rest"
config:
server: <ip>
But i got this error (for a table which is refreshed every 30 min)
'failed to match table read event with job; try increasing '
'`query_log_delay` or `max_query_duration`',
Do you have any ideea?blue-boots-43993
02/14/2022, 9:59 AMinfo_cache = {('get_schema_names', (), ()): ['name', 'name.other', 'name.other.dbo' ....]
as you can see there are some schema names that contain dots inside them. Usually they would be written as [Database].[name.other.dbo].[TableName].[Column]
right? Well I cannot seem to figure out how to force sqlalchemy (or the ingestion script) to not tread 'name.other'
as a schema and not database-schema combination. I tied with schema_pattern.allow properties but with no luckboundless-student-48844
02/14/2022, 11:41 AMdatahub-graphql-core
repo (Step 8). This doc seems to be outdated https://datahubproject.io/docs/datahub-graphql-core/#near-termcuddly-engine-66252
02/14/2022, 12:44 PMdatahub-actions_1 | 2022/02/14 11:04:50 Problem with request: Get "<http://datahub-gms:6667/health>": dial tcp: lookup datahub-gms on 127.0.0.11:53: server misbehaving. Sleeping 1s
Also after these errors start, I don't see datahub-gms in active docker containers(I attach the container log)
(final err from log - Failed to send HTTP request to endpoint: <http://schema-registry:6668/subjects/MetadataChangeLog_Versioned_v1-value/versions>
java.net.ConnectException: Connection refused (Connection refused))
What can be done with these?prehistoric-room-17640
02/14/2022, 2:09 PMalert-teacher-6920
02/14/2022, 5:28 PMplain-farmer-27314
02/14/2022, 6:56 PMError: secret "datahub-encryption-secrets" not found
Is this a new secret we need to add? I double checked the deploying with kubernetes docs and didn't see it mentionednumerous-guitar-35145
02/14/2022, 6:58 PMbland-barista-59197
02/14/2022, 10:23 PMmutation addTag {
addTag(input: { tagUrn: "urn:li:tag:NewTag", resourceUrn: "urn:li:dataFlow:(airflow,dag_abc,PROD)" }),
addTag(input: { tagUrn: "urn:li:tag:NewTag1", resourceUrn: "urn:li:dataFlow:(airflow,dag_abc,PROD)" })
}
few-air-56117
02/15/2022, 7:40 AMcurl -X POST '<link>/api/v2/graphql' \
--header 'Authorization: Bearer <token_from_datahub>' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"{\n me {\n corpUser {\n username\n }\n }\n}","variables":{}}'
But returns 401 :(modern-monitor-81461
02/15/2022, 3:50 PMgroups
claim in Azure OIDC and I can't figure out where my error is coming from. My Azure OIDC integration works until I enable groups
to be present in an ID token. When groups
claim is present, I get a 502 Bad Gateway
and I can't login to DataHub. I have looked at both frontend and gms logs (info & debug) and I can't see what would be causing a 502
.
If you look at the attached screenshot, the GET https://<dh_server>/authenticate?redirect_uri=%2F
returns a 303
to <https://login.microsoftonline.com/{tenant> id}/oauth2/v2.0/authorize?response_type=code&redirect_uri=https://<dh_server>%2Fcallback%2Foidc
, which returns a 302
to the expected callback URL https://<dh_server>/callback/oidc?code=0...
, which causes a 502
... (dh_server
is the DataHub server).
The authentication is a success since I can find the user with its profile. I added extra debug statements in auth.sso.oidc.OidcCallbackLogic
and all looks good. I thought that class would be the one handling the OIDC callback, but looks like I'm wrong. In order to debug further, can someone tell me which class is handling the https://<dh_server>/callback/oidc?code=0...
request? @big-carpet-38439 probably knows this, but I think he is on vacation 🥳. Anyone else?broad-thailand-41358
02/15/2022, 5:48 PM(base) scottlam@penguin:~$ datahub docker quickstart
Unable to run quickstart:
- Docker doesn't seem to be running. Did you start it?
(base) scottlam@penguin:~$
fresh-river-19527
02/15/2022, 7:18 PMProfiling exception No BigQuery dataset specified. Use bigquery_temp_table batch_kwarg or a specify a default dataset in engine url
. For some others it works. I'm of course passing the bigquery_temp_table_schema
setting. I've tried with max_workers: 1
but it still fails in some cases
2. The other one has been reported already. Looks like it is failing for columns with a REPEATED (ARRAY) type
3. A third one looks like it comes from GE itself. Looks like GE is not working with the GEOGRAPHY typewooden-football-7175
02/15/2022, 9:38 PMalert-teacher-6920
02/16/2022, 2:11 PMcool-painting-92220
02/16/2022, 7:26 PMmodern-artist-55754
02/16/2022, 11:20 PMbland-barista-59197
02/17/2022, 12:19 AM