https://datahubproject.io logo
Join Slack
Powered by
# troubleshoot
  • i
    Hey guys! How are you? Recently i was trying to run my Ingestion by the frontend but i got a secret error. (Image bellow). I've checked all my secrets and this secret was there and there was no clerical errors. To resolve this i had to recreate the secrets used by this ingestion and try to run again. Do you have idea why that happened? TY!
  • r
  • k
    I think it is just the backend going down and be unresponsive?
  • b
    Hi, I wanted to follow up on this. I reached out to our network guys and our nexus proxy should be able to pull anything that's part of the apache group as we have proxies set up for the major maven repositories. I also was not able to find any mention of 
    org.apache.commons.cli
    in the code or any variation of 
    commons-cli
    for that matter. I wonder if this is an underlying Java process that's having the issue. I have our nexus repo set up as a repository in the gradle config, but if java itself is trying to pull things without consuming that configuration, then maybe that could be the issue. Any insight or thoughts around this would be helpful. I've never worked with gradle before and have limited Java experience. Thanks!
  • n
    Hi all, after ingesting our glossary terms to DataHub we started adding them to the related datasets. However they don’t show up in the Related Entities tab of the the Glossary Term itself. What we’ve noticed is that while the Glossary Term (with its documentation) can be found behind a URL path containing the Node name, the related entity is attached to the same term without the node in the path (for which our ingested documentation is missing). Is this something we can easily fix?
  • a
    Has anyone had this before? not sure what wrong i am doing
  • c
    image_50358017.JPG
  • c
    And one more thing does I need to upgrade every image to 0.8.32 version or only acryl data hub-actions ?
  • c
    Yea I upgraded the version to 8.32 for acryl data hub Does I need to upgrade version of every image deployed in data hub to 0.8.32?
  • w
    Is there any chance of having a release with this hotfix soon?
  • s
    image.png
  • n
    e.g , i can give some other username/password for datahub UI , change the memory/cpu settings, replicas, etc
  • w
    Hi Team, I was trying to ingest a dataset into datahub using your java emitter library, and this results in failure with message "Reason: Aspect SCHEMA failed to update". I use breakpoint to debug and found out that in the RestEmitter.class, the method 
    public Future<MetadataWriteResponse> emit(MetadataChangeProposal mcp, Callback callback)
    failed to convert some special characters. For example, "Average temperature within the geohash (in ℃)" will be converted to "Average temperature within the geohash (in â)", and the ingestion of schema will fail. May I ask what method do you us to encode and decode the String in the java emitter? Also, are you planning on making this method accept wider range of special characters?
  • s
    Trying to set up like @swift-breakfast-25077 above here, just with Azure AD. Using quickstart, so dont have docker.env, but doing docker-compose. Restarting with "docker-compose -p datahub -f docker-compose.yml -f docker-compose.yml up datahub-frontend-react". 2nd picture shows the end of the error after the command runs. 3rd picture is shown when going into the datahub.
  • m
    I have tried the ES 8.1.2 and the problem still persists so I am thinking this might be compatibility problem with versions higher than 7.9.3 (since that is the only version (that I have tried) that doesn't give those errors)
  • c
    Kind off internal 500 error
  • m
    Copy code
    I want to connect to the Oracle database, but in the create new source stage it stops loading after selecting Oracle
  • m
    Hi, This is not an error but I have encountered with the following problem. If in the secrets tab you select some of the recommended text that appears as a recommendation in the textbox (see image 1), that secret doesn't appear but it exists. I am working with the 0.8.33 version of datahub. To reproduce this I just went to the mentioned tab, created a secret with both a name, value and description I once used (but that doesn't exist currently in datahub), it appears that the secret was succesfully created but it doesn't appear in the web.
  • i
    I’m facing the same issue with global: springKafkaConfigurationOverrides: max.poll.records : “100” 
    Error: UPGRADE FAILED: pre-upgrade hooks failed: warning: Hook pre-upgrade datahub/templates/kafka-setup-job.yml failed: Job in version "v1" cannot be handled as a Job: v1.Job.Spec: v1.JobSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.Containers: []v1.Container: v1.Container.Env: []v1.EnvVar: v1.EnvVar.Value: ReadString: expects " or n, but found 1, error found in #10 byte of ...|,"value":100}],"imag|..., bigger context ...|ame":"KAFKA_PROPERTIES_MAX_POLL_RECORDS","value":100}],"image":"linkedin/datahub-kafka-setup:v0.8.33|...
  • r
  • b
    I'm using Keycloak OIDC for users to login to Datahub. Am encountering a
    Copy code
    502 Bad Gateway
nginx/1.19.2
    error message in the UI sporadically especially when i leave a tab open with Datahub in it for a very long time (typically at least overnight), then come back to it and press F5 refresh or go to another Datahub page. There are no clear logs on this error except for a couple of lines in frontend debug log:
    Copy code
    [application-akka.actor.default-dispatcher-1290] DEBUG o.p.p.r.OidcRedirectActionBuilder - Authentication request url: <the long redirect url that I see when a user is logging in to Datahub>
[application-akka.actor.default-dispatcher-1290] DEBUG o.p.play.http.PlayHttpActionAdaptor - requires HTTP action: 302
    the final URL of the browser is "<datahub-url>/authenticate" to resolve this, I would need to clear the site's cache (F12-clear cache). Although its a bit of a long shot, was wondering if anyone knows what is happening here, because its rather tricky to reproduce, and im trying to stop this from happening because it is not intuitive to typical users to clear their cache when this happens.
  • w
    Hi Team, we notice that the datahub-gms use a re-instate machanism: if an entity's status is "removed=true", when we re-ingest any aspects of this entity later, the status will be change to "removed=false". Is there any way to disable this feature? Thanks
  • a
    Hi team, when will the next version be released?
  • b
    Sorry to revive this old thread, but we seem to be hitting similar issues while we're trying to get the frontend to talk to gms using our service mesh / proxy layer built on envoy. Trying to see if we can hack around using the suggestions posted above around directing specifying ips / such from the frontend -> gms, but I was trying to dig a bit into what a proper workaround would be (on our end we'd ideally want to use envoy as it gives us TLS between our services which is nice). I haven't been able to fix this but I'll throw what I've found out here so if it rings a bell for someone more knowledgeable we could try and build on it. The logs seem to suggest there's a 64 header limit on the server on the frontend. It looks as though the frontend play web-server uses Akka under the hood. The Akka docs indicate that there is a 
    max-header-count = 64
    config default. I was hoping that setting something like:
    Copy code
    # Set up akka max header value
akka.http.server.parsing.max-header-count=256
    In the datahub-frontend's application.conf would do the trick but unfortunately that hasn't helped. I still see the, 
    HTTP message contains more than the configured limit of 64 headers
    line in my logs even with this so I'm either chasing the wrong setting or I've not configured it correctly. If someones got a bit more experience with the Akka server / Play server piece has any ideas, that would be great.
  • b
    Is there a way to create domains using the python rest emitter?
    Copy code
    urn = builder.make_domain_urn(proj.get("name"))

        domain_info = models.DomainPropertiesClass(
            name=proj.get("name"),
            description=None
        )

        mcp = MetadataChangeProposalWrapper(
            entityType="domain",
            changeType=ChangeTypeClass.UPSERT,
            entityUrn=urn,
            aspectName="domainProperties",
            aspect=domain_info,
        )

        emitter.emit_mcp(mcp)
    Something like this? (THis doesn't seem to create it)
  • b
    Nix that, it works, just can't see them as there's nothing in them in the datahub home page
  • b
    My bad
  • m
    image.png
  • b
  • b
    I think I posted the message in the wrong sub. Sorry.
1...111112113...119Latest