https://datahubproject.io logo
Join Slack
    d

    damp-telephone-61279

    1 year ago
    Copy to Clipboard
    Hello again! How can I define an urn for a dataset field (used in primary keys and on DatasetFieldMapping)? Thank you for the help
    a

    ambitious-battery-33996

    1 year ago
    Copy to Clipboard
    You can define a dataset field urn using https://github.com/linkedin/datahub/blob/master/li-utils/src/main/javaPegasus/com/linkedin/common/urn/DatasetFieldUrn.java That said, Dataset field is not yet made a first class entity. This is there in our roadmap and we are working towards that. (Refer this https://github.com/linkedin/datahub/blob/master/docs/rfc/active/1841-lineage/field%20level%20lineage.md )
    d

    damp-telephone-61279

    1 year ago
    Copy to Clipboard
    Ok, looking at the RFC I can see that’s something like:
    "urn:li:datasetField:(urn:li:dataset:(urn:li:dataPlatform:billing,bill-invoice,PROD),/invoice_id)"
    Is the ‘/’ really necessary on the fieldPath?
    c

    cool-river-24902

    1 year ago
    Copy to Clipboard
    Probably there is some hardcoded convention in the code, @ambitious-battery-33996 can confirm.
    a

    ambitious-battery-33996

    1 year ago
    Copy to Clipboard
    pathspec defining field paths is not yet open sourced. That is also in our roadmap of filed level lineage. So, today field path is not standardized in open source, soon we will have a plan for that.
    👍 1
View count: 1
Join thread in Slack