i have this error --- [3:44 PM] [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 6.1.4-ccs [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: c9124241a6ff43bc [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1667849230200 WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both. Created topic MetadataAuditEvent_v4. WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both. Created topic MetadataChangeEvent_v4. WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both. Created topic FailedMetadataChangeEvent_v4. WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both. Created topic MetadataChangeLog_Versioned_v1. WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both. Created topic MetadataChangeLog_Timeseries_v1. WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both. Created topic MetadataChangeProposal_v1. WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both. Created topic FailedMetadataChangeProposal_v1. WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both. Created topic PlatformEvent_v1. WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both. Created topic DataHubUsageEvent_v1. Error while executing config command with args '--command-config /tmp/connection.properties --bootstrap-server b-1.prodproddatahubekspro.5w4ibt.c17.kafka.us-east-1.amazonaws.com:9092,b-2.prodproddatahubekspro.5w4ibt.c17.kafka.us-east-1.amazonaws.com:9092 --entity-type topics --entity-name _schemas --alter --add-config cleanup.policy=compact' java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.UnknownTopicOrPartitionException: at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45) at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32) at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:104) at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:272) at kafka.admin.ConfigCommand$.getResourceConfig(ConfigCommand.scala:552) at kafka.admin.ConfigCommand$.alterConfig(ConfigCommand.scala:322) at kafka.admin.ConfigCommand$.processCommand(ConfigCommand.scala:302) at kafka.admin.ConfigCommand$.main(ConfigCommand.scala:97) at kafka.admin.ConfigCommand.main(ConfigCommand.scala) Caused by: org.apache.kafka.common.errors.UnknownTopicOrPartitionException: [3:45 PM] is when i want to use kafka of aws

Hello Gabriel, Please do not cross-post messages across different channels. This makes it harder for the core DataHub team to keep track of unanswered & unresolved questions. Thanks.

hi. i followed this link https://datahubproject.io/docs/deploy/aws and they do no put credentials

That guide is specific to aws only in regards to Elastic Kubernetes Service (EKS) not MSK (AWS’s kafka offering).

Yes, I know. but what is the solution to the problem?

You need to add credentials to the kafka job, this will depending on your own setup.

i put full access without key right now. that is say the docs

How are you connecting to kafka by hand? How are the consumers configured?

create with terra in the some vpc

I mean the client, when you are connecting to the kafka cluster

this

And how are you connecting locally where it works?

I tried by hand and kafka works