https://datahubproject.io logo
Join Slack
Powered by
# getting-started
  • a
    Hi all! Is there a way to create glossaryTerm properties via the graphQL API? I have tried the createGlossaryTerm mutation, but it does not have any field for setting the properties of the entity! Thanks for your answer in advance!
    b
    • 2
    • 4
  • s
    Hello community ! Reaching out for help … please let me know if there is a better forum to share this question :) Appreciate any help!
    e
    • 2
    • 1
  • p
    Hi, I am trying to test out the 
    datahub-rest
    sink using the docker setup. I am using the 
    datahub ingest -c demo.yml
    command and can run the job with the 
    file
    based sink, but with 
    datahub-rest
    sink, I get 
    'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x13d2ba940>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /config
    Any suggestions on what should I do to debug this issue?
    e
    • 2
    • 23
  • s
    hi dears, pls tell me wich version of gradle need to install?
    m
    • 2
    • 3
  • s
    i want to review this
  • s
    ./gradlew metadata ingestion exampleskafka-etl:bootRun
  • s
    but something wrong
  • s
    ´Welcome to Gradle 7.4.2! Here are the highlights of this release: - Aggregated test and JaCoCo reports - Marking additional test source directories as tests in IntelliJ - Support for Adoptium JDKs in Java toolchains For more details see https://docs.gradle.org/7.4.2/release-notes.html To honour the JVM settings for this build a single-use Daemon process will be forked. See https://docs.gradle.org/7.4.2/userguide/gradle_daemon.html#sec:disabling_the_daemon. Daemon will be stopped at the end of the build Configuration on demand is an incubating feature. FAILURE: Build failed with an exception. * Where: Build file '/root/datahub/buildSrc/build.gradle' line: 8 * What went wrong: A problem occurred evaluating project ':buildSrc'.
    Could not find method compile() for arguments [io.acryljson schema avro0.1.5, build_1kqs8gk1jo9sbgzioo1f15l9v$_run_closure1$_closure2@4fdc76aa] on object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler.
    * Try:
    Run with --stacktrace option to get the stack trace.
    Run with --info or --debug option to get more log output.
    Run with --scan to get full insights.
    * Get more help at https://help.gradle.org BUILD FAILED in 11s´
    e
    • 2
    • 8
  • s
    Copy code
    > Task :metadata-ingestion-examples:mce-cli:bootRun
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::               (v2.5.12)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/.gradle/caches/modules-2/files-2.1/ch.qos.logback/logback-classic/1.2.9/7d495522b08a9a66084bf417e70eedf95ef706bc/logback-classic-1.2.9.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.25/110cefe2df103412849d72ef7a67e4e91e4266b4/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
12:32:41.283 [main] INFO  c.l.metadata.examples.cli.MceCli - Consuming records.
<============-> 99% EXECUTING [2m 55s]
> IDLE
> :metadata-ingestion-examples:mce-cli:bootRun
> IDLE
> IDLE
  • s
    error when put
  • s
    Copy code
    ./gradlew :metadata-ingestion-examples:mce-cli:bootRun
  • s
    freeze in 99%
  • s
    downgrade to java 8
  • b
    Hi everyone! Our team is looking at automating some tasks using the DataHub API. Is there a way to authenticate to the DataHub API using an OAuth token from our ID Provider (in our case, Azure AD)? We would rather avoid username/password (and PATs) if we can.
    e
    g
    s
    • 4
    • 12
  • m
    QQ: How to rank up the glossary terms in the search results?
    b
    • 2
    • 1
  • d
    Hi Everyone, If we want to replace datahub mysql container with our company dba's mssql db or oracle db, how should we adjust 'quickstart.sh' script or other config file to achieve that ? Thanks!
    e
    • 2
    • 3
  • w
    Hi everyone! I just learned about DataHub today, so I decided to deploy it to Google Cloud Platform (https://datahubproject.io/docs/deploy/gcp/) for some testing. I'm a bit stuck at the moment though as I'm trying to update the 
    METADATA_SERVICE_AUTH_ENABLED
    variable for the container / pod (as outlined here), but I don't know how to do it in Google Cloud. I thought I would have permissions to generate personal access tokens as the root user. Has anyone encountered this specific configuration before? If so, could you provide some guidance? It would be much appreciated. 🙂
    h
    b
    • 3
    • 3
  • l
    I understand we can ingest glossary data using YAML, is there a way to generate full YAML out for changes/additions done via UI by users?
    o
    • 2
    • 1
  • w
    Hi, I'm in the process of evaluating datahub. I have looked at the existing architecture, and I understand that the ingestion framework supports a deployment where Kafka is not required as a dependency : "The Ingestion Framework is a modular, extensible Python library for extracting Metadata from external source systems (e.g. Snowflake, Looker, MySQL, Kafka), transforming it into DataHub's Metadata Model, and writing it into DataHub via either Kafka *or using the Metadata Store Rest APIs directly*" In the quickstart I see a docker kafka broker image being pulled - is there an option to not require kafka for a datahub deployment?
    b
    m
    • 3
    • 13
  • c
    Hi, I'm starting with Datahub. I have a Trino DB that I load via DBT. The Trino DB has catalog called warehouse with three schemas : warehouse.bronze, warehouse.silver, warehouse.gold. I have configured ingestion and it runs fine. Then I configured ingestion from DBT with target_platform = trino. Unfortunately DH ingests the DBT tables without the catalog name ("warehouse") so I end up with two sets of tables: "warehouse.bronze.table1" from rino and "bronze.table1" from dbt. One set with schema but without lineage and another set with lineage but without any schema information. I need to configure the DBT to use the database name "warehouse". Any ideas?
    o
    • 2
    • 2
  • s
    Hi team, At the company, we're implementing a PoC with DataHub as our Data Catalog and for that end I am trying to create a new user and to manage its policies. I want this new user to have access only to some tables and some glossary terms. I'm basing myself on the following documentation: 1. https://datahubproject.io/docs/how/auth/add-users/ 2. https://datahubproject.io/docs/policies/ After adding a user by invitation link, I try to create a custom policy but the new user isn't proposed as an option for the new policy. I am able to apply it only to All Users. Do you know how can I fix that ? Thanks !
    o
    b
    • 3
    • 6
  • s
    Hi Team, Do you know if there is a way to block a user of seeing some tables or glossary terms ? I've made some tests limiting its rights, but the user can still see the name and schema of other tables. The user cannot indeed see the table metadata, but I was wondering if I could hide the table itself from the user. Thanks :)
    o
    • 2
    • 1
  • e
    Hello guys, my name is Nadia, I'm a data manager in a company that's currently starting with Datahub. I'm a bit confused about basic Datahub and what is called Acryl-Datahub configuration. Am I getting it right, that Acryl-Datahub provides certain paid functionality over the basic Datahub? If so, what are those features? For example, is Data Profiling available in a free version? It wasn't clear for me from the website.
    plus1 1
    l
    • 2
    • 2
  • r
    There's a line in 
    dbt
    source which describes how source freshness is ingested into DataHub. It says "We transfer dbt's freshness checks to DataHub's last-modified fields.". Can someone please explain what these "last-modified fields" are and where I can find them? So far I only found that for other sources your have a "Stats" tab where this info is showing, but for dbt it's greyed out.
    👍 2
    m
    • 2
    • 2
  • g
    Hi, I'm just getting started in Datahub and being a bit confused about the functionalities for graphql and mce. Graphql can also modify dataset metadata with "mutation". Is mce more powerful in changing metadata than graphql? ( currently I want to modify name and value in dataset's properties but cannot find appropriate interface in graphql)
    b
    • 2
    • 7
  • g
    I have a question about Neo4J viz Elasticsearch. Is anything lost if one uses ES and not Neo4J, functionally? Performance? Somewhat related, if someone has already invested in a different graph database, is there any support or planned support for something other than Neo4J?
    o
    • 2
    • 1
  • g
    Another question - any performance benchmark info about DH?
    m
    • 2
    • 2
  • r
    I have a reverse question to this one https://datahubspace.slack.com/archives/CV2KB471C/p1650645167453159 -- is there a way to hide the "Dataset" nodes (or better yet, not generate them at all)? They duplicate most of the information included in dbt datasets but are missing node type information (in node vizualization) and view definition (in dataset details), so they're just cluttering the lineage and the UI. I feel like I'm missing something, why are they even generated in the first place?
    m
    • 2
    • 6
  • g
    Is there any examples for java emitter to begin with? Moreover, I find DatahubGraph in python to fetch metadata. I wonder is there class with similar function in java?
    o
    • 2
    • 2
  • f
    Our organization does not allow node port service or ingress in production as we need ssl at the pod level. The best way to do is by adding side car container as proxy to the front end container(I'll be using nginx container). Problem here is that default datahub helm chart does not allow us to add extra containers from values.yaml it only allows for initcontainers. Can someone help on this please? Thread in Slack Conversation
    b
    i
    • 3
    • 2
1...323334...80Latest