https://datahubproject.io logo
Join Slack
Powered by
Hey everyone. I'm having trouble ingesting propert...
# ingestion
b
Hey everyone. I'm having trouble ingesting properties - they don't get added through a successful ingestion. For example ingesting a dashboard with customProperties. (Posting the code of the json I ingest in the thread) The ingestion is successful but I can't find the properties in the UI. Ingesting through rest - this exact code worked on previous versions and is taken from bootstrap_mce.json file from the latest github version.
{
"auditHeader": null,
"proposedSnapshot": {
"com.linkedin.pegasus2avro.metadata.snapshot.DashboardSnapshot": {
"urn": "urn:li:dashboard:(looker,baz)",
"aspects": [
{
"com.linkedin.pegasus2avro.common.Ownership": {
"owners": [
{
"owner": "urn:li:corpGroup:bfoo",
"type": "DATAOWNER",
"source": null
}
],
"lastModified": {
"time": 1581407189000,
"actor": "urn:li:corpuser:jdoe",
"impersonator": null
}
}
},
{
"com.linkedin.pegasus2avro.dashboard.DashboardInfo": {
"title": "Baz Dashboard",
"description": "Baz Dashboard",
"customProperties": {
"prop1": "fakeprop",
"prop2": "pikachu",
"andrew": "andrewtest"
},
"charts": [
"urn:li:chart:(looker,baz1)",
"urn:li:chart:(looker,baz2)"
],
"lastModified": {
"created": {
"time": 0,
"actor": "urn:li:corpuser:jdoe",
"impersonator": null
},
"lastModified": {
"time": 0,
"actor": "urn:li:corpuser:datahub",
"impersonator": null
},
"deleted": null
},
"dashboardUrl": null,
"access": null,
"lastRefreshed": null
}
}
]
}
},
"proposedDelta": null
}
m
@brief-lizard-77958 is this on a release tag or on the latest(head) docker tag?
b
Hey Andrew - thanks for the report. We will take a look!
b
@mammoth-bear-12532 release tag
b
Attempting to repro this now
On the head of master
@brief-lizard-77958 Hey again! Just attempted to ingest the above^ using 
file_to_datahub_rest.yml
example recipe, pointing it to a file that contains the following:
Copy code
[{
  "auditHeader": null,
  "proposedSnapshot": {
    "com.linkedin.pegasus2avro.metadata.snapshot.DashboardSnapshot": {
      "urn": "urn:li:dashboard:(looker,baz)",
      "aspects": [
        {
          "com.linkedin.pegasus2avro.common.Ownership": {
            "owners": [
              {
                "owner": "urn:li:corpGroup:bfoo",
                "type": "DATAOWNER",
                "source": null
              }
            ],
            "lastModified": {
              "time": 1581407189000,
              "actor": "urn:li:corpuser:jdoe",
              "impersonator": null
            }
          }
        },
        {
          "com.linkedin.pegasus2avro.dashboard.DashboardInfo": {
            "title": "Baz Dashboard",
            "description": "Baz Dashboard",
            "customProperties": {
              "prop1": "fakeprop",
              "prop2": "pikachu",
              "andrew": "andrewtest"
            },
            "charts": [
              "urn:li:chart:(looker,baz1)",
              "urn:li:chart:(looker,baz2)"
            ],
            "lastModified": {
              "created": {
                "time": 0,
                "actor": "urn:li:corpuser:jdoe",
                "impersonator": null
              },
              "lastModified": {
                "time": 0,
                "actor": "urn:li:corpuser:datahub",
                "impersonator": null
              },
              "deleted": null
            },
            "dashboardUrl": null,
            "access": null,
            "lastRefreshed": null
          }
        }
      ]
    }
  },
  "proposedDelta": null
}]
And it appears to show
Which version are you using?
c
@brief-lizard-77958 ^^
b
Thx 😛
b
Updated the version and I could see all of the properties without ingesting them again 🙂 thank you all
b
Wonderful!
Open in Slack
PreviousNext
Powered by