Is there a hardcoded length limit on urns? We’re s...
# ui
h
Is there a hardcoded length limit on urns? We’re seeing some weird errors when loading datasets in the UI and it seems in the logs like the urns are clipped
g
at the moment, yes
this is encoded in the Urn .pdl file
each Urn's field with set a maxLength
h
Ok ill check that in a bit. But shouldnt that be caught at ingestion time?
g
Yes- you're right. Ingestion should be issuing some sort of error instead of truncating
Although I also wonder if these length need to be so restrictive
h
This is the error I'm getting
Copy code
13:57:18 [Thread-334] WARN  n.g.e.SimpleDataFetcherExceptionHandler - Exception while fetching data (/browsePaths) : java.lang.RuntimeException: Failed to retrieve browse paths: entity type DATASET, urn urn:li:dataset:(urn:li:dataPlatform:s3,wolt-datawarehouse-uploads-master
java.util.concurrent.CompletionException: java.lang.RuntimeException: Failed to retrieve browse paths: entity type DATASET, urn urn:li:dataset:(urn:li:dataPlatform:s3,wolt-datawarehouse-uploads-master
        at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
        at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1592)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Failed to retrieve browse paths: entity type DATASET, urn urn:li:dataset:(urn:li:dataPlatform:s3,wolt-datawarehouse-uploads-master
        at com.linkedin.datahub.graphql.resolvers.browse.BrowsePathsResolver.lambda$get$1(BrowsePathsResolver.java:38)
        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
        ... 1 common frames omitted
Caused by: java.lang.RuntimeException: Failed to retrieve dataset with urn urn:li:dataset:(urn:li:dataPlatform:s3,wolt-datawarehouse-uploads-master, invalid urn
        at com.linkedin.datahub.graphql.types.dataset.DatasetUtils.getDatasetUrn(DatasetUtils.java:15)
        at com.linkedin.datahub.graphql.types.dataset.DatasetType.browsePaths(DatasetType.java:147)
        at com.linkedin.datahub.graphql.resolvers.browse.BrowsePathsResolver.lambda$get$1(BrowsePathsResolver.java:35)
        ... 2 common frames omitted
So it's not a length issue
hmm...., now I think i know what the issue is. We just dump the s3 key as is, which includes
/
l
@gray-shoe-75895 ^ we should be escaping
/
b
So this is not a length issue, but a character encoding issue?
g
@high-hospital-85984 can you share the full urn that is deemed invalid?
the urn that is printed out seems to be missing a closing parenthesis
Is that an issue with the urn itself or the error message?
h
@green-football-43791 the urn is
urn:li:dataset:(urn:li:dataPlatform:s3,wolt-datawarehouse-uploads-master/source=user-push-endpoints,PROD)
(in fact, we have multiple similar urns with similar format but only the
user-push-endpoints
changes, and all of them fail). So either the
/
or the
source=
messes with the UI
g
We ran into this when I was playing around with s3 as well - you’ll probably need to replace the / with . in the urn for now
h
okey, good to know I’m not the only one 😅 Does an issue exist for this bug?
g
Not yet but it would be good to track- mind opening an issue?
h
Sure, i’ll do it in a bit!