Is it safe to limit kafka topic message sizes to 1M instead of 5M?? Getting this error caused by the limitation of our managed kafka provider:

Error while executing config command with args '--command-config /tmp/connection.properties --bootstrap-server <redacted> --entity-type topics --entity-name PlatformEvent_v1 --alter --add-config max.message.bytes=5242880'
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.InvalidConfigurationException: Exceeding max allowed message size for topic 'PlatformEvent_v1'. Limit: 1048576, requested: 5242880

I see where I can change it in the helm chart, but didn’t know if making too small might break functionality.

kafka:
    maxMessageBytes: "5242880"  # 5MB
    producer:
      compressionType: none
      maxRequestSize: "5242880"  # 5MB
    consumer:
      maxPartitionFetchBytes: "5242880"  # 5MB
      stopContainerOnDeserializationError: true

Helm versions:

$ helm list -n dev
NAME                            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                           APP VERSION
datahub                         dev             9               2024-01-11 19:30:31.882348541 +0000 UTC deployed        datahub-0.3.23                  0.12.1     
prerequisites                   dev             2               2024-01-10 17:12:03.308544108 +0000 UTC deployed        datahub-prerequisites-0.1.6

You can definitely try. Wether it works or not will depend on the size of the metadata being emitted/ingested. You will see errors during ingestion along the lines of

org.apache.kafka.common.errors.RecordTooLargeException: The message is XXX bytes when serialized which is larger than YYY, which is the value of the max.request.size configuration

if there are problems. You can also try to enable compression to hopefully reduce the size of the records.

Making it too small can definitely impact functionality - we'd definitely recommend providing more headroom than less!