hey team, here the example how do I create the URN...
# troubleshoot
s
hey team, here the example how do I create the URN with java api
Copy code
new DatasetUrn(new DataPlatformUrn("delta-lake"), "entities/user", FabricType.TEST)
// or with codes
new DatasetUrn(new DataPlatformUrn("delta-lake"), "entities\u002Fuser", FabricType.TEST)
as a result slash is not considered while if this entity would be pulled directly by datahub the slash would be considered and we would see one more item in the path.
Expected result:
g
Hi @straight-spoon-27189 Could you please post actual and expected result? And more detail on issue you are facing
s
hi @gentle-hamburger-31302 for example, if the delta table
user
is located in the s3 bucket
s3entities
then in the code I put following:
Copy code
new DatasetUrn(new DataPlatformUrn("delta-lake"), "s3entities/user", FabricType.TEST)
and I expect to see:
so datahub distinguish the slash show the table correctly and it works exaclty like this if I do not use java api and import just by default with recipies. The problem only with java api.
g
I think it is because of browse path
Along with urn , connector would also setting the browse path
s
how I can do it in java api?
I'm using the java emitter, MetadataChangeProposalWrapper and DatasetProperties
g
@straight-spoon-27189 I believe you need to implement java code equivalent of https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/source/data_lake_common/data_lake_utils.py#L105 to generate container hierarchy. @dazzling-judge-80093 Please correct me
s
I haven't seen equivalent in java, here the example of usages from datahub documentation https://datahubproject.io/docs/metadata-integration/java/as-a-library#usage
g
@straight-spoon-27189 Might be it is not available for Java, However you can use REST API as well as you can refer Spark Emitter code, It is connector for Spark in Java https://github.com/datahub-project/datahub/tree/master/metadata-integration/java
s
hi, @gentle-hamburger-31302 I'm using the Java api but I found out that such method in java
UrnUtils.toDatasetUrn("hive", "fct_users_deleted", "PROD")
does not work in the same way as similar function in python.
pls check my original question.
g
Hi @gray-shoe-75895 Could you please help @straight-spoon-27189 on original question?