n

    nutritious-bird-77396

    1 year ago
    Hi LinkedIn Team….Can you guys help us understand the reasoning behind renaming schema namespaces from
    com.linkedin.mxe
    to
    com.linkedin.pegasus2avro.mxe
    ? For ex: https://github.com/linkedin/datahub/blob/54fef777ac7079e6a44aad40af55e868a5844f57/metadata-events/mxe-utils-avro-1.7/src/main/java/com/linkedin/metadata/EventUtils.java#L84
    a

    ambitious-battery-33996

    1 year ago
    Kafka events are in avro schemas. But, rest of the processing in datahub gms happens in PDL schemas. Hence, we need to convert
    com.linkedin.pegasus2avro.mxe
    to
    com.linkedin.mxe
    m

    microscopic-receptionist-23548

    1 year ago
    So we have a few different schema languages. Our models are written in Pegasus (PDL). But then we auto generate Avro from the PDL (hence
    pegasus2avro
    in the name). If you want to use Kafka, you should use Avro. Otherwise, Pegasus is often nicer to deal with.
    Basically, what @ambitious-battery-33996 said 🙂
    n

    nutritious-bird-77396

    1 year ago
    Thanks!
    b

    bumpy-keyboard-50565

    1 year ago
    More importantly is to avoid class clashes. There are places where both Pegasus & Avro classes are needed. Without renaming it simply won't work.