Hello guys! I’m trying to use airflow with datahub kafka based connection. I’ve configured connection from Airflow UI, i wrote the kafkabroker with <my-broker>:9092. Since i’m using schema registry, i wrote that in “extra” field in json format. It’s something like:

{
  "sink": {
    "config": {
      "connection": {
        "schema_registry_url": "<my-schema-registry>:8081"
      }
    }
  }
}

But it gives me an error like:

sink
  extra fields not permitted (type=value_error.extra)

Also above lines in log, i see

Host: <my-broker>:9092, Port: None, Schema: , Login: , Password: None, extra: {'sink': {'config': {'connection': {'schema_registry_url': '<my-schema-registry>:8081'}}}}

So, it gets my host but can not get port? I didn’t get it. Also, how can i resolve the extra fields problem. Thanks!

Hi can you try the same without the “sink” part in the json?

{
  "config": {
    "connection": {
      "schema_registry_url": "<my-schema-registry>:8081"
    }
  }
}

Hi, i’ve resolved the problem. It should be starting from connection. Like

{
  "connection": {
    "schema_registry_url": "<my-schema-registry>:8081"
  }
}

Thanks for sharing your solution!

Your welcome!