https://datahubproject.io logo
Join Slack
Powered by
# getting-started
  • r
    Hi, I'll be implementing DataHub, my infrastructure will contain various data lakes, each data lake will have a different owner, I want that anyone can access the metadata from all of these data lakes (each data lake owner should be able to ingest metadata only to their own section, shouldn't be able to change other data lakes' metadata in datahub and should be able to see all the metadata from all data lakes) so I thought of a centralized DataHub, but I read about Federated Metadata Serving in DataHub website and I am trying to grasp this concept and want to know what are the advantages of implementing this option instead of just ingesting the metadata from all the data lake into one DataHub. Also want to know if there is any information of how to implement this federated metadata serving. Thank you.
  • b
    Hello all! I just start Datahub Quickstart and have some question!
  • b
    https://datahubproject.io/docs/quickstart I install everything but have problem at step 4, datahub docker quickstart
  • b
    does anybody know about that error...? plz..
  • h
    image.png
    plus1 1
  • a
    not able to copy urn
  • d
    Hi everyone, I have a question regarding policies: Is there a way to create a policy for a certain kind of platform? I want to allow certain users to only edit metadata in platform “Redash” for example, but not on other BI platforms.
  • s
    This is weird. I have sqlalchemy plugin enabled Is this something someone can help out?
  • m
    Good Morning One & All! I am trying to perform an ingestion of snowflake with an External Browser type of Authentication and clearly I don’t have the syntax just right. I get the following error:
  • m
    Copy code
    '1 validation error for SnowflakeConfig\n'
           'authentication_type\n'
           '  extra fields not permitted (type=value_error.extra)\n'
  • b
    image.png
  • b
    image.png
  • l
    Hi All, Trying to navigate through docs and have quick question on my use case. Assuming DataHub allows adding triggers like custom entities, does it have support schema registry? Although I looked Kafka schema registry, but in my use case we are looking is when the user creates a new event trigger request for a group of devices then we want to be able to look up the master signal list for each device type and validate that all device types to which this request is being sent produce all the signals specified in the Event Trigger request.
  • w
    Hello--is it possible to add a Feature Table under a specific domain? Any reason why it is not treated as an asset?
  • f
    image.png
  • b
    Hello!
  • b
    (tell me if I need to change channel 😉 )
  • t
    Hi, I am new to this Datahub , deployed it locally with docker on windows with the reference of https://datahubproject.io/docs/quickstart . From UI Ingestion created Oracle as source and execute to fetch the metadata. But getting connection refused . Any help on this
  • t
    source:
    type: oracle
    config:
    host_port: ****:1521
    username: ******
    password: ******
    service_name: ******
    sink:
    type: datahub-rest
    config:
    server: '<http://localhost:9002/api/gms>'
  • r
    Hi there \o
    👋 2
    👋🏻 1
  • b
    Hello everyone, My name is Rafael Durbano and I will explore the DataHub this week ! I would like to know if I can create a execution's lineage for data pipeline (Azure Data Factory), but they are not necessarily nested. It's not a problem to input this values manually. Something like the image below. Each box is one ADF pipeline
    🙂 1
  • b
    Can be understanding as a high level (or group) of data lineage
  • a
    Hi All are all images available on https://gallery.ecr.aws/?page=1 ??
  • f
    Hi, Would like to know if datahub provides REST APIs for adding sources, ingest metadata, ingest lineage data etc. Kinda looking for Metadata Services for Maintaining metadata
  • f
    Getting error while ingesting data from postgres or mysql via UI
  • f
    Copy code
    '[2022-03-31 13:08:38,539] WARNING  {urllib3.connectionpool:810} - Retrying (Retry(total=2, connect=None, read=None, redirect=None, '
           "status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffffa4223280>: Failed to "
           "establish a new connection: [Errno 111] Connection refused')': /api/gms/config\n"
           '[2022-03-31 13:08:42,541] WARNING  {urllib3.connectionpool:810} - Retrying (Retry(total=1, connect=None, read=None, redirect=None, '
           "status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffffa4223670>: Failed to "
           "establish a new connection: [Errno 111] Connection refused')': /api/gms/config\n"
           '[2022-03-31 13:08:50,555] WARNING  {urllib3.connectionpool:810} - Retrying (Retry(total=0, connect=None, read=None, redirect=None, '
           "status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffffa4223850>: Failed to "
           "establish a new connection: [Errno 111] Connection refused')': /api/gms/config\n"
           '[2022-03-31 13:08:50,916] ERROR    {datahub.entrypoints:119} - File '
           '"/tmp/datahub/ingest/venv-3e16d565-2d1a-45bf-94a2-0b4eb1f767c5/lib/python3.9/site-packages/urllib3/connection.py", line 174, in '
           '_new_conn\n'
           '    161  def _new_conn(self):\n'
           ' (...)\n'
           '    170      if self.socket_options:\n'
           '    171          extra_kw["socket_options"] = self.socket_options\n'
           '    172  \n'
           '    173      try:\n'
           '--> 174          conn = connection.create_connection(\n'
           '    175              (self._dns_host, self.port), self.timeout, **extra_kw\n'
  • f
    My Properties
  • f
    this is my localhost postgres
  • f
    its not working too when i put localhost
  • f
    Any help would be helpful here
1...7677787980Latest