hello. I am trying to use datahub in Korea. I am c...
# troubleshootp
polite-actor-701
06/29/2022, 10:55 AMhello. I am trying to use datahub in Korea. I am currently using v0.8.32 installed. When I try to input Korean in urn or description, etc., an error occurs. I tried to install v0.8.36 or v0.8.39, but I couldn't use it with a red error window saying "Oops, an error occurred". I think I need utf8 encoding to input Korean. What should I do?
o
orange-night-91387
06/29/2022, 9:28 PMHi! Where are you inputting the Korean urn? Through the API layer, one of the client SDKs, or from an ingestion source?
orange-night-91387
06/29/2022, 9:29 PMAlso what is the error you are seeing? We should be supporting UTF-8 encoding in urns and have other users using different languages successfully.
p
polite-actor-701
06/29/2022, 11:41 PMHi. I made the following json file for test and ran 'datahub ingest -c file_to_datahub.yml'.
[
{
"auditHeader": null,
"entityType": "dataset",
"entityUrn": "urnlidataset:(urnlidataPlatform:oracle,한글테스트,PROD)",
"entityKeyAspect": null,
"changeType": "UPSERT",
"aspectName": "subTypes",
"aspect": {
"value": "{\"typeNames\": [\"한글\"]}",
"contentType": "application/json"
},
"systemMetadata": null
}
]
polite-actor-701
06/30/2022, 12:03 AMpolite-actor-701
06/30/2022, 12:36 AMAs you said, I can use Korean for urn. However, Korean is not included in document, column name, subtype, etc.
d
dazzling-judge-80093
07/01/2022, 1:33 PMyou can test this with the following curl command:
Copy code
curl -X POST -H 'User-Agent: python-requests/2.27.1' -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'Connection: keep-alive' -H 'X-RestLi-Protocol-Version: 2.0.0' -H 'Content-Type: application/json' --data '{"proposal": {"entityType": "dataset", "entityUrn": "urn:li:dataset:(urn:li:dataPlatform:oracle,\ud55c\uae00\ud14c\uc2a4\ud2b8,PROD)", "changeType": "UPSERT", "aspectName": "subTypes", "aspect": {"value": "{\"typeNames\": [\"\ud55c\uae00\"]}", "contentType": "application/json"}, "systemMetadata": {"lastObserved": 1656682219924, "runId": "file-2022_07_01-15_30_19"}}}' '<http://localhost:8080/aspects?action=ingestProposal>'dazzling-judge-80093
07/01/2022, 1:33 PMthis is what python sends
dazzling-judge-80093
07/01/2022, 1:38 PMok, as I see for urns it works but we fail on the typename
dazzling-judge-80093
07/01/2022, 1:39 PMWith my test this fails:
Copy code
curl -X POST -H 'User-Agent: python-requests/2.27.1' -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'Connection: keep-alive' -H 'X-RestLi-Protocol-Version: 2.0.0' -H 'Content-Type: application/json' --data '{"proposal": {"entityType": "dataset", "entityUrn": "urn:li:dataset:(urn:li:dataPlatform:oracle,\u00e1rv\u00edzt\u0171r\u0151t\u00fck\u00f6rf\u00far\u00f3g\u00e9p,PROD)", "changeType": "UPSERT", "aspectName": "subTypes", "aspect": {"value": "{\"typeNames\": [\"\u00e1rv\u00edyt\u0171r\u0151t\u00fck\u00f6rf\u00far\u00f3g\u00e9p\"]}", "contentType": "application/json"}, "systemMetadata": {"lastObserved": 1656682602517, "runId": "file-2022_07_01-15_36_42"}}}' '<http://localhost:8080/aspects?action=ingestProposal>' Copy code
curl -X POST -H 'User-Agent: python-requests/2.27.1' -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'Connection: keep-alive' -H 'X-RestLi-Protocol-Version: 2.0.0' -H 'Content-Type: application/json' --data '{"proposal": {"entityType": "dataset", "entityUrn": "urn:li:dataset:(urn:li:dataPlatform:oracle,\u00e1rv\u00edzt\u0171r\u0151t\u00fck\u00f6rf\u00far\u00f3g\u00e9p,PROD)", "changeType": "UPSERT", "aspectName": "subTypes", "aspect": {"value": "{\"typeNames\": [\"test_type\"]}", "contentType": "application/json"}, "systemMetadata": {"lastObserved": 1656682679830, "runId": "file-2022_07_01-15_37_59"}}}' '<http://localhost:8080/aspects?action=ingestProposal>'9 Views