Hi all! I have a quick question regarding emittin...
# troubleshootb
busy-airport-23391
08/29/2022, 7:57 PMHi all! I have a quick question regarding emitting validation results. I currently have a loop set up to push assertionInfo, assertionPlatform, and assertionResults to DataHub via a Python Kafka emitter. Everything is working properly, but of the three test assertions I've written, only the latest is showing up in the Validator tab. I'm having trouble finding an example of multiple assertions being pushed to DataHub -- do you have any pointers?
busy-airport-23391
08/29/2022, 8:10 PMbusy-airport-23391
08/30/2022, 6:04 PMI think I figured out the issue -- the assertionUrn needs to change for each assertion (seems obvious now), but for some reason builder.make_assertion_urn from assertion info is not creating a unique urn for subsequent runs. -- still figuring out why
busy-airport-23391
08/30/2022, 7:14 PM^I was passing in an empty json string into the make_assertion_urn method in my latest tests... Marking this one as solved
busy-airport-23391
08/30/2022, 7:37 PMvars(info.datasetAssertion) was returning an empty dictionary, so this was my solution:
Copy code
def assertionUrn(info: AssertionInfo) -> str:
urn = builder.make_assertion_urn(
builder.datahub_guid(
{
"dataset": info.datasetAssertion.dataset,
"scope": info.datasetAssertion.scope,
"fields": info.datasetAssertion.fields,
"aggregation": info.datasetAssertion.aggregation,
"operator": info.datasetAssertion.operator,
}
)
)
print(urn)
return urnl
little-megabyte-1074
08/31/2022, 12:29 AM
Thanks for following up with your fix, @busy-airport-23391!! teamwork
11 Views