witty-kilobyte-6731
04/04/2021, 2:03 PMhigh-hospital-85984
04/06/2021, 9:43 AMfaint-hair-91313
04/06/2021, 10:00 AMwonderful-quill-11255
04/09/2021, 8:52 AMwitty-florist-25216
04/09/2021, 9:24 AMfaint-hair-91313
04/09/2021, 11:55 AMbetter-orange-49102
04/11/2021, 3:58 PMingestion | [2021-04-11 15:51:17,986] ERROR {datahub.ingestion.run.pipeline:41} - failed to write record with workunit public.mytable with ('Unable to emit metadata to DataHub GMS', {'message': "HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /datasets?action=ingest (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14a3261790>: Failed to establish a new connection: [Errno 111] Connection refused'))"}) and info {'message': "HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /datasets?action=ingest (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14a3261790>: Failed to establish a new connection: [Errno 111] Connection refused'))"}
busy-accountant-26554
04/13/2021, 1:53 PMDashboardInfo.pdl
(https://github.com/linkedin/datahub/tree/master/metadata-models/src/main/pegasus/com/linkedin/dashboard) I see coverage for the metadata elements
• Title
• Description
• Last modified
• Last reloaded
+ a few more.
And I guess owner is a common element for all entities in DataHub.
But there seems to be no Dashboard aspect for misc properties and there can be quite a few ot those from my experience with Qlik apps.
Is the possibility to add misc properties (name - value pairs) for Dashboards something that anyone would feel was useful?
Or did I miss anything?broad-flag-97458
04/14/2021, 8:59 PMcalm-sunset-28996
04/15/2021, 8:52 AMicy-easter-2378
04/15/2021, 3:14 PMpip install 'acryl-datahub[mysql]'
but I get this error:icy-easter-2378
04/15/2021, 3:15 PMCollecting acryl-datahub[mysql]
Could not find a version that satisfies the requirement acryl-datahub[mysql] (from versions: )
No matching distribution found for acryl-datahub[mysql]
icy-easter-2378
04/15/2021, 3:22 PMred-journalist-15118
04/19/2021, 4:52 PMcalm-addition-66352
04/19/2021, 11:17 PMdatahub ingest -c ./mssql-recipe_._yml
. And my recipe file looks as below,
source:
type: mssql
config:
username: <user_name>
password: <password>
host_port: <ip>:1433
database: <db_name>
table_pattern:
#deny:
# - "^.*\\.sys_.*" # deny all tables that start with sys_
allow:
- "schema_name_1.*"
sink:
type: "datahub-rest"
config:
server: "<http://localhost:8080>"
But when I try, I get the below error,
sqlalchemy.exc.OperationalError: (pytds.tds_base.OperationalError) Database '<organization_name>\<dba_name>' does not exist. Make sure that the name is entered correctly.
[SQL: use [<organization_name>\<dba_name>]]
• Is there a way to print or get the underlying queries submitted to the database (so I can figure out why it tries to query an object with the dba's name, probably a previously deleted item / user account ect etc...) ?
• Is there a list of permissions that needs to be assigned to the db user that we use for the crawler / ingestion. Current user that I am trying has some additional server level access and was wondering whether it is providing additional metadata of the server that is not expected (eg: dba user names etc etc...)calm-addition-66352
04/20/2021, 6:32 AMcalm-addition-66352
04/21/2021, 8:17 AM{
"auditHeader": null,
"proposedSnapshot": {
"com.linkedin.pegasus2avro.metadata.snapshot.DashboardSnapshot": {
"urn": "urn:li:dashboard:sample",
"aspects": [
{
"com.linkedin.pegasus2avro.dataset.DashboardInfo": {
"title": "Sample Dashboard",
"description": "This is a sample dashboard to test mce events"
}
},
{
"com.linkedin.pegasus2avro.common.Ownership": {
"owners": [
{
"owner": "urn:li:corpuser:bi-analyst",
"type": "DEVELOPER"
}
]
}
}
]
}
},
"proposedDelta": null
}
metadata.yml
source:
type: "file"
config:
filename: ./mce.json
sink:
type: "datahub-rest"
config:
server: "<http://localhost:8080>"
I get an error saying __root__, MetadataFileSourceConfig expected dict not str (type=type_error)
. So I am wondering either this method doesn't support Dashboard/Chart metadata or I am missing a property or a value in my mce json 🙂calm-sunset-28996
04/21/2021, 8:59 AMmodern-nest-69826
04/21/2021, 12:07 PMmodern-nest-69826
04/21/2021, 12:18 PMcalm-addition-66352
04/22/2021, 8:54 PMacryl-datahub[mssql]
to ingest metadata from a mssql database to datahub. Is there a way that I can pass upstream and ownership data at the same time ? Or do I need to pass that separately ? 🙂busy-accountant-26554
04/23/2021, 11:16 AMsteep-pizza-15641
04/23/2021, 2:03 PMhandsome-airplane-62628
04/23/2021, 3:05 PMdelightful-plumber-77060
04/23/2021, 5:04 PMsteep-pizza-15641
04/26/2021, 1:01 PMcalm-addition-66352
04/26/2021, 11:36 PMdatahub
library in my dag, should I install acryl-datahub
python package or a is there a different one for airflow ?
2. I am trying to add the datahub rest connection through the UI, if I have installed acryl-datahub
package - should I see an option as datahub
on the drop down ? or do I need to select a different option ? [screenshot attached]calm-sunset-28996
04/27/2021, 11:57 AMsalmon-translator-27951
04/27/2021, 1:55 PMcom.linkedin.metadata.dao
refers to?curved-magazine-23582
04/27/2021, 2:28 PMProperties
section under chart UI. What data correspond to this section? For dataset entity, a similar section corresponds to DatasetAspect com.linkedin.dataset.DatasetProperties.
But I don't see such aspect for chart. 🤔