https://datahubproject.io logo
Join Slack
Powered by
Hi Team, recently our Kafka cluster crashed and af...
# all-things-deployment
b
Hi Team, recently our Kafka cluster crashed and after recreating it we are facing these issues in starting datahub. GMS is waiting for MAE consumer to complete it’s execution . But in MAE consumer we are getting below error
Copy code
│ Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing key/value for partition MetadataChangeLog_Versioned_v1-0 at offset 6191. If needed, please seek past the record to continue consu │
│ Caused by: org.apache.kafka.common.errors.SerializationException: Error retrieving Avro unknown schema for id 4                                                     │
│ Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Schema 4 not found; error code: 40403                                             │
│   at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:292)

│ ERROR KafkaMessageListenerContainer$ListenerConsumer Consumer exception                                                                                                                                                 │
│  java.lang.IllegalStateException: This error handler cannot process 'SerializationException's directly; please consider configuring an 'ErrorHandlingDeserializer' in the value and/or key deserializer  

│ Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing key/value for partition DataHubUpgradeHistory_v1-0 at offset 0. If needed, please seek past the record to continue consumption.   │
│ Caused by: org.apache.kafka.common.errors.SerializationException: Error retrieving Avro unknown schema for id 9                                                                                                         │
│ Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Schema 9 not found; error code: 40403                                                                                          │
│     at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:292)
and schema-registry throwing below error.
Copy code
[2023-06-28 07:45:49,091] INFO 172.18.52.255 - - [28/Jun/2023:07:45:49 +0000] "GET /schemas/ids/9?fetchMaxId=false HTTP/1.1" 404 51 1 (io.confluent.rest-utils.requests)              
[2023-06-28 07:45:49,091] ERROR Request Failed with exception (io.confluent.rest.exceptions.DebuggableExceptionMapper)                                      io.confluent.rest.exceptions.RestNotFoundException: Schema 9 not found                                                                 at io.confluent.kafka.schemaregistry.rest.exceptions.Errors.schemaNotFoundException(Errors.java:106)                                                 at io.confluent.kafka.schemaregistry.rest.resources.SchemasResource.getSchema(SchemasResource.java:85)                                               at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)                                                              at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl
can you please help us to restore our setup.
GMS is getting terminated after sometime with the below error.
Copy code
│ 2023-06-30 08:22:55,460 [pool-10-thread-1] WARN org.elasticsearch.client.RestClient:65 - request [POST <http://elasticsearch-master:9200/datahubpolicyindex_v2/_search?typed_keys=true&max_concurrent_shard_requests=5&> │
│ 2023-06-30 08:22:55,734 [pool-10-thread-1] WARN org.elasticsearch.client.RestClient:65 - request [POST <http://elasticsearch-master:9200/datahubpolicyindex_v2/_search?typed_keys=true&max_concurrent_shard_requests=5&> │
│ 2023-06-30 08:24:55,292 [pool-10-thread-1] WARN org.elasticsearch.client.RestClient:65 - request [POST <http://elasticsearch-master:9200/datahubpolicyindex_v2/_search?typed_keys=true&max_concurrent_shard_requests=5&> │
│ 2023-06-30 08:24:55,368 [pool-10-thread-1] WARN org.elasticsearch.client.RestClient:65 - request [POST <http://elasticsearch-master:9200/datahubpolicyindex_v2/_search?typed_keys=true&max_concurrent_shard_requests=5&> │
│ 2023-06-30 08:26:55,291 [pool-10-thread-1] WARN org.elasticsearch.client.RestClient:65 - request [POST <http://elasticsearch-master:9200/datahubpolicyindex_v2/_search?typed_keys=true&max_concurrent_shard_requests=5&> │
│ 2023-06-30 08:26:55,351 [pool-10-thread-1] WARN org.elasticsearch.client.RestClient:65 - request [POST <http://elasticsearch-master:9200/datahubpolicyindex_v2/_search?typed_keys=true&max_concurrent_shard_requests=5&> │
│ ANTLR Tool version 4.5 used for code generation does not match the current runtime version 4.7.2ANTLR Runtime version 4.5 used for parser compilation does not match the current runtime version 4.7.2ANTLR Tool versio │
│ 2023/06/30 08:26:57 Command exited with error: wait: no child processes                                                                         │
│ panic: interface conversion: error is *os.SyscallError, not *exec.ExitError                                                                       │
│                                                                                                             │
│ goroutine 23 [running]:                                                                                                 │
│ main.runCmd({0x800ef0?, 0xc000180460}, 0xc000184250, {0x7fff9eb349ea, 0x4}, {0xc000007620?, 0x37?, 0xc000061f98?})                                                   │
│   /go/pkg/mod/github.com/jwilder/dockerize@v0.6.1/exec.go:54 +0x454                                                                          │
│ created by main.main                                                                                                  │
│   /go/pkg/mod/github.com/jwilder/dockerize@v0.6.1/main.go:299 +0xf17
d
Hi, what version are you using?
b
@delightful-ram-75848 we are using version v0.10.0.6
3 Views
Open in Slack
PreviousNext
Powered by