better-orange-49102
07/21/2022, 8:27 AMpolicy = graph.get_aspect_v2(
entity_urn=policy_urn,
aspect = "dataHubPolicyInfo",
aspect_type=DataHubPolicyInfoClass
)
I will inevitably encounter an AvroException: ('Datum union type not in schema %s', 'filter')
for policies that are created in UI and have a filter condition. (and 1st time created without edits)
It is somehow resolved if i add more entities to the filter, then reverting to the original filter. Not sure why it is happening.cool-dinner-21847
07/21/2022, 11:48 AMwitty-painting-90923
07/21/2022, 1:09 PMdatahub delete --urn "urn:li:container:b2f754cac6699f0bd0688b5573eb2156" --dry-run
[2022-07-21 14:56:35,407] INFO {datahub.cli.delete_cli:134} - DataHub configured with <http://localhost:8080>
This urn was referenced in 5 other aspects across your metadata graph:
+----------------+---------------------------------------------------------------------------------------------------------------+-----------+
| relationship | entity | aspect |
+================+===============================================================================================================+===========+
{i removed this lines}
Do you want to delete these references? [y/N]: y
Failed to execute operation
java.lang.RuntimeException: Could not deserialize ByteString(length=4,bytes=6e756c6c) for aspect container
bitter-lizard-32293
07/21/2022, 1:44 PMIngestPoliciesStep
. I think it seems to be related to @early-lamp-41924’s PR - https://github.com/datahub-project/datahub/pull/4733 but for some reason we're still seeing the bootstrap fail (from what I understand that should allow us to be able to bootstrap). Some details in: 🧵silly-ice-4153
07/21/2022, 2:37 PMrhythmic-stone-77840
07/21/2022, 3:49 PMkind-whale-32412
07/21/2022, 7:41 PM- Docker doesn't seem to be running. Did you start it?
issue, here's how I managed to quickstart docker.
The main reason why I am getting that is because I'm not using docker-desktop (this is due to licensing issues for larger corporations)
The way to start is, clone datahub and then run this command on datahub folder:
docker-compose -p datahub -f docker/quickstart/docker-compose-without-neo4j-m1.quickstart.yml up -d
PS: I'm on Macbook M1, so that's why I'm using the one that has neo4j-m1
adjust that accordingly based on your devicepurple-whale-11305
07/21/2022, 8:10 PMambitious-cartoon-15344
07/22/2022, 1:40 AMcuddly-arm-8412
07/22/2022, 3:16 AMjolly-traffic-67085
07/22/2022, 4:08 AMjolly-traffic-67085
07/22/2022, 8:32 AMred-vr-34382
07/22/2022, 7:39 PMdatabase_name
is actually getting set to the display name thats in superset (see attached screenshots). is this intentional? i would have thought DataHub would want to grab the actual name of the database.breezy-shoe-41523
07/25/2022, 7:35 AMrich-policeman-92383
07/25/2022, 1:45 PMhandsome-football-66174
07/25/2022, 6:28 PMbest-umbrella-24804
07/25/2022, 11:24 PMbright-diamond-60933
07/26/2022, 3:28 PMCaused by: java.io.IOException: Cannot run program "C:\Program Files\jdk8\bin\java.exe" (in directory "C:\Users\sarat.lingamarl\Documents\datahub\datahub\metadata-models"): CreateProcess error=206, The filename or extension is too long
Is there an easy way to shorten the command line? I found a solution online but I dont know if this works: https://blog.jetbrains.com/idea/2017/10/intellij-idea-2017-3-eap-configurable-command-line-shortener-and-more/
The JetBrains Blog
IntelliJ IDEA: Configurable command line shortener | The IntelliJ IDEA Blog
In IntelliJ IDEA, it’s possible to configure a command-line shortener. Let’s take a closer look at when and how to use this feature. Configurable command line shortener When the classpath gets too l
https://blog.jetbrains.com/idea/2017/10/intellij-idea-2017-3-eap-configurable-command-line-shortener-and-more/
[11:14 AM] I was able to build datahub easily on Macbusy-airport-23391
07/26/2022, 4:50 PMtimestampMillis
?busy-dusk-4970
07/26/2022, 5:15 PMlemon-zoo-63387
07/27/2022, 6:11 AMError Msg: Could not find checkpoint 'xxxxxx' (or its configuration is invalid)
microscopic-mechanic-13766
07/27/2022, 10:51 AMERROR auth.sso.oidc.OidcCallbackLogic - Unable to renew the session. The session store may not support this feature
.
I have seen that the origin of this error used to be that the PLAY_TOKEN cookie was rejected due to userprofile exceeding 4096 chars. I have checked the length of this cookie and in my case is lower than 4096 chars.
The thing is that I get this error both when I login and when I try to logout (I say try to logout as it doesn't logout, it just keeps redirecting to the main page of Datahub, the one where you can see the datasets ingested,...)
I have also tried limiting the claims of the AUTH_OIDC_SCOPE
but haven't been lucky.
Any help would be appreciated!best-leather-7441
07/27/2022, 12:59 PMacceptable-baker-8114
07/27/2022, 3:15 PMkind-whale-32412
07/27/2022, 5:15 PMdocker-compose-without-neo4j-m1.quickstart.yml
as the file.
Wondering how can I enable debug mode?creamy-tent-10151
07/27/2022, 7:58 PM./gradlew :datahub-frontend:dist -x yarnTest -x yarnLint
To create the image
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker build -t linkedin/datahub-frontend-react:debug -f docker/datahub-frontend/Dockerfile .
Build and start datahub
datahub docker quickstart --build-locally
Replacing and running the local changes
(cd docker && COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -p datahub -f docker-compose-without-neo4j.yml -f docker-compose-without-neo4j.override.yml -f docker-compose.dev.yml up -d --no-deps --force-recreate datahub-frontend-react)
I did it in this order and at the log in screen it was fine but after that, I received error messages like these:
Validation error of type InvalidFragmentType
Failed to Complete Search
Any help would be appreciated, and let me know if my process for updating the source code is wrong, as I'm pretty inexperienced with development on containers, thanks!silly-wire-85016
07/27/2022, 9:15 PMRestliEntityClient
received ?narrow-apple-60403
07/28/2022, 8:23 AMdatahub-frontend-react | 08:23:23 [application-akka.actor.default-dispatcher-35] ERROR auth.sso.oidc.OidcCallbackLogic - Unable to renew the session. The session store may not support this feature
datahub/docker/datahub-frontend/env/docker.env:
AUTH_OIDC_ENABLED=true
AUTH_OIDC_CLIENT_ID=my-client-id
AUTH_OIDC_CLIENT_SECRET=my-client-secret
AUTH_OIDC_DISCOVERY_URI=<https://accounts.google.com/.well-known/openid-configuration>
AUTH_OIDC_BASE_URL=<http://localhost:9002>
AUTH_OIDC_SCOPE=openid profile email
AUTH_OIDC_USER_NAME_CLAIM=email
AUTH_OIDC_USER_NAME_CLAIM_REGEX=([^@]+)
docker-compose:
docker-compose -p datahub -f docker-compose-without-neo4j.yml -f docker-compose-without-neo4j.override.yml up
square-lawyer-36076
07/28/2022, 6:07 PMFailed to perform SSO callback. SSO is not enabled for protocol: oidc
do i have to specify env. vars to be added to a datahub front end in addition to options specified in datahub-frontend/values.yaml? Specifically, i see uri reference but no options for it in the values.yaml file. Pointers/ideas? thank you, Dima.handsome-football-66174
07/28/2022, 6:54 PM