https://datahubproject.io logo
Join Slack
Powered by
# troubleshoot
  • c
    Hi Team, I'm running a MWAA instance and trying to reach out the Datahub but I'm getting the following error: ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997) Does anyone know how to disable this SSL verification?
    ✅ 1
    g
    • 2
    • 1
  • h
    Can someone teach me the dataflow of Pyhton ( ingestion job) <----> graphql. <---> UI
    ✅ 1
    g
    • 2
    • 1
  • h
    I am not seeing any proper documents
  • a
    Hi team, please tell me how do I get the view table bottom or the view in superset bottom, I made some ingestions, but it has only the columns name of each table, it does not have available to see the data.
    g
    • 2
    • 2
  • g
    Hi there, I have been trying to find a way to execute some "data validation and quality check" scripts(python) in Dtahub or via Datahub over our catalogs (For now the source is AWS S3), Our use case is Data Governance, with a goal to automate this functionality, please consider us newbies in Datahub. I have tried Python Remitter but that is more for ingestion, If I am not mistaken. Any help is highly appreciated! Thank you all so much!
    g
    • 2
    • 1
  • b
    Hi #troubleshoot I'm trying to setup DataHub in an EKS cluster within a secure environment (no internet access). Is there a way to download locally all required files and upload them in a S3 bucket to access them through the local network? Any hints would be appreciated.
    ✅ 1
    b
    • 2
    • 1
  • a
    Hi! i installed python3 -m pip install --upgrade acryl-datahub library but when im importing Pipeline from this library i get ModuleNotFoundError: No module named 'datahub.metadata'. How can i solve this?
    ✅ 1
    g
    g
    a
    • 4
    • 6
  • f
    Hello! I am fairly new to datahub. I've installed datahub using docker-compose on an EC2 instance. I shut down this EC2 instance daily due to cost issues. So I stop datahub in the evening and start it back in the morning. However when I start datahub it always shows error in starting some service and then process goes into wait state undefinitely. I've to exit the instance and login again. It always takes many tries to get datahub instance up and running. How can I resolve this issue?
    ✅ 1
    plus1 1
    g
    • 2
    • 7
  • b
  • w
    Hello All. I've recently wanted to get into integrating great expectations with Datahub and I noticed that in the integration docs that there are a couple of configuration options that I don't quite understand. I'd appreciate if I could get some further explanation on the highlighted options (platform_alias, platform_instance_map). Furthermore, I am also interested to know what all options are available to be used in platform_instance_map. I look forward to the reply!
    ✅ 1
    b
    g
    h
    • 4
    • 4
  • b
    Hi Team, I am doing the ingestion for dbt artifacts from a S3 bucket. It was showing before the logic (queries) for the assertions. But all of a sudden it disappeared and is showing this error:
  • i
    Hello team! I have encountered a weird error after I tried to ingest snowflake metadata (the recipe worked just fine until yesterday, but today it is raising errors). Here's the error and abridged log:
    Copy code
    [2023-05-24 16:52:58,950] INFO     {datahub.cli.ingest_cli:173} - DataHub CLI version: 0.10.2.2
[2023-05-24 16:52:59,141] INFO     {datahub.ingestion.run.pipeline:204} - Sink configured successfully. DataHubRestEmitter: configured to talk to <http://localhost:8080> with token: eyJh**********O9bU
[2023-05-24 16:53:03,611] ERROR    {datahub.entrypoints:195} - Command failed: Failed to find a registered source for type snowflake: Updating forward references for asset model PandasCSVAsset raised TypeError: issubclass() arg 1 must be a class
    And here's the recipe:
    Copy code
    source:
  type: "snowflake"
  config:
    username: "${SNOWFLAKE_USERID}"
    password: "${SNOWFLAKE_PASSWORD}"
    account_id: "real_account_name"
    warehouse: "bi_transforming_development"
    database_pattern:
      allow:
        - "^ANALYTICS_DBT_PROD$"
    schema_pattern:
      allow:
        - "^PROD$"
    role: "analyst"
sink:
  type: "datahub-rest"
  config:
    token: "${TOKEN}"
    Using the UI with the same credentials is working, and the environment variables appear to be ingested correctly at least in the case of the token (also, hard-coding the variables in the recipe does not solve the problem either). Thank you for any help!
    ✅ 1
    • 1
    • 1
  • g
    hello everyone! I'm trying to follow the instruction to set up Data Hub in docker (Ubuntu), however, after issuing the command 'datahub docker quickstart' I keep getting this error - open /.datahub/quickstart/docker-compose.yml: permission denied Error while pulling images. Going to attempt to move on to docker compose up assuming the images have been built locally any idea what's causing this? tried chmod on the yml file but it didn't change anything.
    ✅ 1
    h
    g
    • 3
    • 2
  • a
    Hi all, how can we enable Prometheus monitoring on datahub i already tired to enable in values.yaml
    Copy code
    monitoring:
      enablePrometheus: true
    Like this but is there any docs that how we can setup whole monitoring process of enabling in datahub
    ✅ 1
    g
    • 2
    • 1
  • n
    Good day. Not sure if I am in the right channel. I am trying to use a shared ES instance for my datahub deployment. Before we proceed, I would like to check out the list of indices that datahub uses so that we can ensure we dont have a conflicting index already present in our current ES instance. Can someone point me to the resource?
    ✅ 1
    g
    • 2
    • 2
  • h
    I am not able to generate token
    ✅ 1
  • h
    Token based authentication is currently disabled. Contact your DataHub administrator to enable this feature.
    n
    g
    • 3
    • 2
  • g
    Hi Team, I am experiencing error on datahub-gms container. It keeps on showing 'unhealthy'. I've tried restarting the containers however, same issue again and again.
    n
    g
    +2
    • 5
    • 23
  • g
    image.png
  • g
    image.png
  • g
    Could anyone suggest a fix, please ? Thanks.
  • m
    We have installed datahub on AWS using https://datahubproject.io/docs/deploy/aws. I can access the UI. But would like to use CLI and python. Where do i get the GMS url and token as there is no server.
    ✅ 1
    g
    • 2
    • 2
  • a
    I am getting this error while doing BQ ingestion, I saw some thread which said point to other version. How do we do that and resolve this?
    Copy code
    datahub.ingestion.run.pipeline.PipelineInitError: Failed to find a registered source for type bigquery: 'str' object is not callable
    d
    h
    • 3
    • 2
  • e
    Hello, facing the issue when doing cluster upgrade: Logs from elasticsearch
    Copy code
    {
  "type": "server",
  "timestamp": "2023-05-25T18:01:45,859Z",
  "level": "ERROR",
  "component": "o.e.i.g.DatabaseNodeService",
  "cluster.name": "elasticsearch",
  "node.name": "elasticsearch-master-0",
  "message": "failed to retrieve database [GeoLite2-Country.mmdb]",
  "stacktrace": [
    "org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];",
    "at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:179) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:165) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.search.TransportSearchAction.executeSearch(TransportSearchAction.java:929) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.search.TransportSearchAction.executeLocalSearch(TransportSearchAction.java:763) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.search.TransportSearchAction.lambda$executeRequest$6(TransportSearchAction.java:399) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:112) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:77) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.search.TransportSearchAction.executeRequest(TransportSearchAction.java:487) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:285) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:101) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:179) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:53) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:177) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:145) ~[?:?]",
    "at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:177) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:154) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:82) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:95) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:73) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:407) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:57) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.client.OriginSettingClient.doExecute(OriginSettingClient.java:51) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:407) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:392) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:542) ~[elasticsearch-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.ingest.geoip.DatabaseNodeService.lambda$retrieveDatabase$11(DatabaseNodeService.java:367) [ingest-geoip-7.17.3.jar:7.17.3]",
    "at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:718) [elasticsearch-7.17.3.jar:7.17.3]",
    "at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]",
    "at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]",
    "at java.lang.Thread.run(Thread.java:833) [?:?]"
  ]
}
    ✅ 1
    • 1
    • 1
  • b
    Hi @here. We are getting this alert with datahub 0.10.0
    g
    a
    • 3
    • 7
  • b
    PyPi package potentially vulnerable to dependency confusion attack | acryl-datahub-actions datahub-prod-acryl-datahub-actions
  • b
    Anyone recieved this warning before?
  • b
    Hi all! I have a glue ingestion job that constantly fails. It's failing with this error and was wondering if anyone has came across it before and was able to fix it?
    Copy code
    'failures': {'<s3://aws-glue-assets-XXXXXX-us-west-2/scripts/Untitled> job.py': ['Unable to download DAG for Glue job from <s3://aws-glue-assets-XXXXXX-us-west-2/scripts/Untitled> job.py, so job subtasks and lineage will be missing: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.', 'Unable to download DAG for Glue job from <s3://aws-glue-assets-XXXXXX-us-west-2/scripts/Untitled> job.py, so job subtasks and lineage will be missing: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.']}
    I don't have that file in s3 nor a glue job called 
    Untitled job.py
    so I am trying to see what I can do to resolve. The rest of the metadata is being pulled over but it's annoying it's marking it as a failure.
    ✅ 1
    g
    • 2
    • 2
  • r
    There seems to be some problems with the demo UI, have anyone encountered it yet?
    ✅ 1
    g
    • 2
    • 2
  • n
    Hi team, Do we have support for citus db in datahub?
    h
    • 2
    • 2
1...979899...119Latest