wide-ghost-47822
04/07/2023, 10:35 AMaction_list contains Datahub configs and I investigate the traffic with nmap tool on port 8080 which is the gms port, and I can see that every requests are responded with HTTP status code 200. You can see some examples of the output:
{"proposal": {"entityType": "assertion", "entityUrn": "urn:li:assertion:81f38ff71a20153a0291b5f087e75f55", "changeType": "UPSERT", "aspectName": "assertionInfo", "aspect": {"value": "{\"customProperties\": {\"expectation_suite_name\": \"test_suite_3\"}, \"type\": \"DATASET\", \"datasetAssertion\": {\"dataset\": \"urn:li:dataset:(urn:li:dataPlatform:mysql,<my_table>)\", \"scope\": \"DATASET_COLUMN\", \"fields\": [\"urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:mysql,london.logging_bid,PROD),id)\"], \"aggregation\": \"IDENTITY\", \"operator\": \"NOT_NULL\", \"nativeType\": \"expect_column_values_to_not_be_null\", \"nativeParameters\": {\"column\": \"id\", \"mostly\": \"1.0\"}}}", "contentType": "application/json"}}}
HTTP/1.1 200 OK.
Date: Fri, 07 Apr 2023 07:39:55 GMT.
Content-Type: application/json.
X-RestLi-Protocol-Version: 2.0.0.
Content-Length: 61.
Server: Jetty(9.4.46.v20220331).
.
{"value":"urn:li:assertion:81f38ff71a20153a0291b5f087e75f55"}
{"proposal": {"entityType": "assertion", "entityUrn": "urn:li:assertion:81f38ff71a20153a0291b5f087e75f55", "changeType": "UPSERT", "aspectName": "dataPlatformInstance", "aspect": {"value": "{\"platform\": \"urn:li:dataPlatform:great-expectations\"}", "contentType": "application/json"}}}
POST /aspects?action=ingestProposal HTTP/1.1.
Host: <my-host>:8080.
User-Agent: python-requests/2.26.0.
Accept-Encoding: gzip, deflate.
Accept: */*.
Connection: keep-alive.
X-RestLi-Protocol-Version: 2.0.0.
Content-Type: application/json.
Content-Length: 287.
What I expect now is that I should see the expectations result in Datahub UI in tab called Validation . Yet it looks disabled.
So, maybe you can guide me what should be the reason for that.lively-cat-88289
04/07/2023, 10:36 AMastonishing-answer-96712
04/07/2023, 10:09 PMwide-ghost-47822
04/10/2023, 2:07 PMurn:li:dataset:(urn:li:dataPlatform:mariadb,<my-table-name>,PROD) .
Yet when I execute my validation script and listen the traffic in datahub-gms port 8080 I saw something like that.
{
"proposal": {
"entityType": "assertion",
"entityUrn": "urn:li:assertion:a922925e5e864e2567b926ec79a44750",
"changeType": "UPSERT",
"aspectName": "assertionInfo",
"aspect": {
"value": "{\"customProperties\": {\"expectation_suite_name\": \"test_suite\"}, \"type\": \"DATASET\", \"datasetAssertion\": {\"dataset\": \"urn:li:dataset:(urn:li:dataPlatform:mysql,<my-tabĹźe>,PROD)\", \"scope\": \"DATASET_COLUMN\", \"fields\": [\"urn:li:schemaField:(urn:li:dataset:(urn:li:dataPlatform:mysql,<my-table>,PROD),id)\"], \"aggregation\": \"IDENTITY\", \"operator\": \"BETWEEN\", \"parameters\": {\"maxValue\": {\"value\": \"null\", \"type\": \"UNKNOWN\"}, \"minValue\": {\"value\": \"100010\", \"type\": \"UNKNOWN\"}}, \"nativeType\": \"expect_column_values_to_be_between\", \"nativeParameters\": {\"column\": \"id\", \"min_value\": \"100010\"}}}",
"contentType": "application/json"
}
}
}
As you can see urn starts with mysql not mariadb. Do you think that would be the problem?wide-ghost-47822
04/10/2023, 8:44 PMtype from mariadb to mysql .
Now I can see the test results in Datahub. Yet another problem occured 🙂
I couldn’t see any Assertions have run. Trying to figure it out why it gives me this error.astonishing-answer-96712
04/12/2023, 5:44 PMastonishing-answer-96712
04/12/2023, 5:45 PMwide-ghost-47822
04/13/2023, 1:34 PMDataHub CLI version: 0.10.1
Python version: 3.9.2 (default, Feb 28 2021, 17:03:44)
[GCC 10.2.1 20210110]astonishing-answer-96712
04/14/2023, 4:42 PM