https://datahubproject.io logo
Docs
Join the conversationJoin Slack
Channels
acryl-omnisend
advice-data-governance
advice-metadata-modeling
all-things-datahub-in-windows
all-things-deployment
announcements
authentication-authorization
chatter
column-level-lineage
contribute
contribute-datahub-blog
data-council-workshop-2023
datahub-soda-test
demo-slack-notifications
design-business-glossary
design-data-product-entity
design-data-quality
design-datahub-documentation
design-dataset-access-requests
design-dataset-joins
feature-requests
flyte-datahub-integration
getting-started
github-activities
help-
i18n-community-contribution
ingestion
integration-alteryx-datahub
integration-azure-datahub
integration-dagster-datahub
integration-databricks-datahub
integration-datastudio-datahub
integration-iceberg-datahub
integration-powerbi-datahub
integration-prefect-datahub
integration-protobuf
integration-tableau-datahub
integration-vertica-datahub
introduce-yourself
jobs
metadata-day22-hackathon
muti-tenant-deployment
office-hours
openapi
plugins
show-and-tell
talk-data-product-management
troubleshoot
ui
Powered by Linen
chatter
  • m

    modern-artist-55754

    12/24/2022, 7:27 AM
    Happy holiday everyone. Thanks all the contributors and Acryl team for the awesome project!
    s
    b
    • 3
    • 2
  • b

    big-carpet-38439

    12/28/2022, 10:42 PM
    Are many folks using Qlik Sense for BI analytics?
    r
    l
    a
    • 4
    • 3
  • m

    mysterious-carpet-43629

    01/02/2023, 8:34 AM
    Looks like the feature request page is down, at least since this morning. feature-requests.datahubproject.io
    This Serverless Function has timed out.
    Your connection is working correctly.
    Vercel is working correctly.
    504: GATEWAY_TIMEOUT Code:
    FUNCTION_INVOCATION_TIMEOUT
    b
    • 2
    • 2
  • s

    salmon-rose-54694

    01/04/2023, 2:15 AM
    Is there 2023 H1 roadmap? I can't find it.
  • s

    silly-energy-20436

    01/05/2023, 9:17 AM
    @little-megabyte-1074 @astonishing-answer-96712 Hello…Column level linage is great feature added to Datahub, As per the medium blog support for Redshift was scheduled for Q4 2022. Has this been rolled out or scheduled for later this year?
    b
    • 2
    • 2
  • n

    nutritious-gpu-34132

    01/05/2023, 9:51 PM
    What is the Northstar metric that your team follows in order to track success of DataHub / Data Catalog in your organization?
    g
    • 2
    • 2
  • l

    little-translator-96552

    01/11/2023, 8:43 PM
    A tough one, some people squirm when they hear schema and then I say table and they understand. Other times i need to say data definition etc etc. Would it be possible to somehow have these terms dependent on who logs in? As in change some of the terminology to cater to the audience, could be some kind of preference setting people set on first log in to figure out the specific user better (or maybe inferred from roles etc)? That would be the most user-friendly UI I could imagine without alienating some group of people without the right domain/technology knowledge.
    b
    e
    • 3
    • 6
  • e

    elegant-state-4

    01/12/2023, 5:03 PM
    I updated the DataPlatform entity and added DATA_MESH as a value. I am unable to run the command
    ./gradlew :gms:impl:build -Prest.model.compatibility=ignore
    as recommended in the developer guide. I am getting the following error:
    Project 'gms' not found in root project 'datahub'.
    Is the guide out of date?
    i
    • 2
    • 4
  • s

    square-australia-53660

    01/18/2023, 2:52 PM
    Hi, Does DataHub have a way to keep track of successful job runs? I’ve been looking around on the website but haven’t found anything like that. I use Airflow, Spark, and S3 mostly and was trying to see if I could report the successful runs to DataHub
    d
    b
    • 3
    • 7
  • a

    ambitious-flower-45028

    01/18/2023, 9:34 PM
    Hi guys. I just recently got to know DataHub and by reading docs , seems like if you want to deploy production level DataHub , you better have Kubernetes set up ? Is it a must have or nice to have thing ?
    g
    l
    • 3
    • 7
  • j

    jolly-traffic-67085

    01/20/2023, 10:08 AM
    Hi guys. is current version of DataHub can limit the permission of user to only see the database name, but can't see all contents inside [like schema, table, column etc.]?
    b
    b
    • 3
    • 3
  • e

    elegant-salesmen-99143

    01/25/2023, 11:17 AM
    a bit random question, but is it only me who gets logged out from Datahub Feature Requests page every day? Can't it remember me logged in? or is the problem on my side?
    g
    l
    • 3
    • 6
  • e

    elegant-state-4

    02/01/2023, 3:11 PM
    Hey folks! I would like to be able do CRUD operations on all entities (e.g. datasets, data jobs, data lineages, charts, etc) programmatically using the OpenAPI interface. Is this possible? If not what are the restrictions?
    a
    • 2
    • 1
  • a

    astonishing-answer-96712

    02/07/2023, 6:22 PM
    Does anyone have tips/tricks or good stories about how you onboarded new users into DataHub after getting up and running? We’re writing a blog post and want to showcase community examples!
    c
    • 2
    • 2
  • i

    important-rainbow-77301

    02/08/2023, 5:53 PM
    Hi, dear Datahub team. we tried several datahub images (like in the attachment) in our deployments, but almost all the tags have vulnerability issues and fail to get scanned. Could you please let us know which tags are vulnerability free? We have tried from version 0.8.45 to 0.10.0. None of them passes through a vulnerability check. Thank you
    b
    • 2
    • 1
  • b

    billions-family-12217

    02/10/2023, 7:22 AM
    datahub-ingestion-cron: enabled: true crons: mysql: schedule: "0 * * * *" # Every hour recipe: configmapName: recipe-config fileName: mysql_recipe.yml is this not working can any One help me out
    b
    • 2
    • 2
  • m

    millions-leather-73634

    02/15/2023, 3:41 PM
    Hello 👋 I was wondering if there was a block other than technical/roadmap to integrate Dagster with DataHub as it is already the case with Airflow? When I read this sentence in a Dagster blog post:
    Dedicated metadata catalogs like Amundsen and Datahub have been developed to meet this need. These are powerful tools, but they have a limitation opposite to that of traditional orchestrators-- their data model has no place for code. Finding the logic that generated an asset is likely to be a complex task with organization-specific idiosyncrasies.
    I wonder if it is just prioritization roadmap question, or there's an overlap between the tools that opposes them instead of bringing them together.
    l
    • 2
    • 1
  • r

    red-solstice-83887

    02/22/2023, 2:32 PM
    Hey everyone, curious if any other company using DataHub has dealt with the need for multi-tenancy? We have specific departments whose metadata should not be visible to the rest of the organisation (either in the preview dropdown of the search bar or on the search result pages) and we’re scratching our heads over how to get this done. cc: @boundless-student-48844
  • b

    bumpy-businessperson-69102

    02/23/2023, 9:57 AM
    Hi everyone! Not sure if it's a correct channel, please refer me correct one if I'm wrong 🙂. Just found small issue in this documentation , Examples for Python SDK for tags and terms are actually the same code. It would be appreciated if you can fix it.
    l
    • 2
    • 1
  • p

    proud-table-38689

    02/27/2023, 11:21 PM
    has anyone implemented a fake data generator to compliment their datahub instance?
    g
    • 2
    • 4
  • w

    witty-journalist-17562

    03/01/2023, 6:19 AM
    any indicative idea on the pricing part
    l
    • 2
    • 1
  • m

    many-manchester-24732

    03/02/2023, 6:32 AM
    Hi, does anyone has any one have any idea on how to deal with unstructured data such as videos and images that is part of tables in lake houses etc. An outline picture of this done is datahub would also be helpful
    p
    • 2
    • 7
  • m

    modern-parrot-70911

    03/07/2023, 2:23 PM
    Kudos, @little-megabyte-1074, for sharing your insightful perspectives on the recent Data Engineering Podcast episode about Data Culture! Your insights on the strategies to establish a data community within an organization were enlightening! Great conversation 👏🏽
    l
    • 2
    • 1
  • m

    many-manchester-24732

    03/09/2023, 12:14 PM
    Is automated data discovery supported by datahub
    a
    l
    m
    • 4
    • 5
  • s

    some-alligator-9844

    03/10/2023, 10:38 AM
    How do I write custom graphql resolvers for Dataset?
  • a

    adventurous-area-49559

    03/10/2023, 11:23 AM
    Hello Are we able to cuztomise the logo's and favicons? To say Company name powered by datahub?
  • e

    elegant-salesmen-99143

    03/13/2023, 10:43 AM
    Hi. I noticed that in documentation there are two pages about Policies, and they have slightly different lists of privileges: https://datahubproject.io/docs/authorization/access-policies-guide has a "Edit Dataset Queries" privilege which the other doesn't have and https://datahubproject.io/docs/authorization/policies/ has "Manage Direct Glossary Children" and "Manage All Glossary Children" which the other doesn't have When I open Manage Permissions tab in Settings in my Datahub, I don't see "Edit Dataset Queries" in available privileges. So what exactly does the first link https://datahubproject.io/docs/authorization/access-policies-guide have to do with?
    b
    • 2
    • 2
  • f

    fierce-dentist-21936

    03/13/2023, 4:02 PM
    Hi! I am looking into data hub and wondering if anyone has used their premium package and if its worth the money. I have reached out to them on pricing but have not heard back.
    e
    • 2
    • 1
  • e

    elegant-salesmen-99143

    03/16/2023, 12:25 PM
    Is it just me, or does it seem like there's been some decrease of activity from Datahub Team in Slack channels here? Is there some kind of holiday, or is the team preparing for something big?
    l
    • 2
    • 1
  • m

    many-manchester-24732

    03/22/2023, 6:21 AM
    How does datahub sit on top different sources and orchestrate sharing of meta-data between tools, lets say the data source is databricks or snowflake and when spark tries to pull data from these data sources , will it be able to share the metadata that is stored in datahub to spark? Is it possible to restrict that transfer from data source to spark if PII data is involved automatically, bottom can the orchestration of data between these tools can be done by datahub?
    m
    • 2
    • 1
Powered by Linen
Title
m

many-manchester-24732

03/22/2023, 6:21 AM
How does datahub sit on top different sources and orchestrate sharing of meta-data between tools, lets say the data source is databricks or snowflake and when spark tries to pull data from these data sources , will it be able to share the metadata that is stored in datahub to spark? Is it possible to restrict that transfer from data source to spark if PII data is involved automatically, bottom can the orchestration of data between these tools can be done by datahub?
m

modern-artist-55754

03/22/2023, 9:43 AM
Datahub has graphql api to query metadata, you can integrate with your orchestrar, spark jobs to handle your business requirements.
View count: 1