https://datahubproject.io logo
Join Slack
Powered by
Following the DataHub-on-Kubernetes instructions o...
# all-things-deployment
g
Following the DataHub-on-Kubernetes instructions outlined here: https://datahubproject.io/docs/deploy/kubernetes/ Kubernetes is RedHat Openshift on AWS (ROSA), I am using the default prerequisites helm install:
Copy code
helm install prerequisites datahub/datahub-prerequisites
results in prerequisites-cp-schema-registry crashing. Should Kafka be installed by default?? Error log from crashing pod in the replies to this comment....
Copy code
===> User
uid=1001000000(1001000000) gid=0(root) groups=0(root),1001000000
===> Configuring ...
===> Running preflight checks ... 
===> Check if Kafka is healthy ...
[main] INFO org.apache.kafka.clients.admin.AdminClientConfig - AdminClientConfig values: 
        bootstrap.servers = [prerequisites-kafka:9092]
        client.dns.lookup = use_all_dns_ips
        client.id = 
        <http://connections.max.idle.ms|connections.max.idle.ms> = 300000
        <http://default.api.timeout.ms|default.api.timeout.ms> = 60000
        <http://metadata.max.age.ms|metadata.max.age.ms> = 300000
        metric.reporters = []
        metrics.num.samples = 2
        metrics.recording.level = INFO
        <http://metrics.sample.window.ms|metrics.sample.window.ms> = 30000
        receive.buffer.bytes = 65536
        <http://reconnect.backoff.max.ms|reconnect.backoff.max.ms> = 1000
        <http://reconnect.backoff.ms|reconnect.backoff.ms> = 50
        <http://request.timeout.ms|request.timeout.ms> = 30000
        retries = 2147483647
        <http://retry.backoff.ms|retry.backoff.ms> = 100
        sasl.client.callback.handler.class = null
        sasl.jaas.config = null
        sasl.kerberos.kinit.cmd = /usr/bin/kinit
        sasl.kerberos.min.time.before.relogin = 60000
        sasl.kerberos.service.name = null
        sasl.kerberos.ticket.renew.jitter = 0.05
        sasl.kerberos.ticket.renew.window.factor = 0.8
        sasl.login.callback.handler.class = null
        sasl.login.class = null
        sasl.login.refresh.buffer.seconds = 300
        sasl.login.refresh.min.period.seconds = 60
        sasl.login.refresh.window.factor = 0.8
        sasl.login.refresh.window.jitter = 0.05
        sasl.mechanism = GSSAPI
        security.protocol = PLAINTEXT
        security.providers = null
        send.buffer.bytes = 131072
        ssl.cipher.suites = null
        ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
        ssl.endpoint.identification.algorithm = https
        ssl.engine.factory.class = null
        ssl.key.password = null
        ssl.keymanager.algorithm = SunX509
        ssl.keystore.location = null
        ssl.keystore.password = null
        ssl.keystore.type = JKS
        ssl.protocol = TLSv1.3
        ssl.provider = null
        ssl.secure.random.implementation = null
        ssl.trustmanager.algorithm = PKIX
        ssl.truststore.location = null
        ssl.truststore.password = null
        ssl.truststore.type = JKS

[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 6.0.1-ccs
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: 9c1fbb3db1e0d69d
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1652815006219
[kafka-admin-client-thread | adminclient-1] INFO org.apache.kafka.clients.admin.internals.AdminMetadataManager - [AdminClient clientId=adminclient-1] Metadata update failed
org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1652815036222, tries=1, nextAllowedTryMs=1652815036323) timed out at 1652815036223 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call.
[main] ERROR io.confluent.admin.utils.ClusterStatus - Error while getting broker list.
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1652815046224, tries=1, nextAllowedTryMs=1652815046325) timed out at 1652815046225 after 1 attempt(s)
        at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
        at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
        at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
        at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
        at io.confluent.admin.utils.ClusterStatus.isKafkaReady(ClusterStatus.java:149)
        at io.confluent.admin.utils.cli.KafkaReadyCommand.main(KafkaReadyCommand.java:150)
Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1652815046224, tries=1, nextAllowedTryMs=1652815046325) timed out at 1652815046225 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
[main] INFO io.confluent.admin.utils.ClusterStatus - Expected 1 brokers but found only 0. Trying to query Kafka for metadata again ...
[main] ERROR io.confluent.admin.utils.ClusterStatus - Expected 1 brokers but found only 0. Brokers found [].
(event_subscriber-E14yQQjC)
i
Hello Joe, DataHub requires Kafka to function. We are in the proposing of making these components optional but it will take a while. What pod crashed? The logs you sent, is it from GMS?
g
Pedro - thanks for the help...
prerequisites-cp-schema-registry
is failing after installing the stock 
helm install prerequisites datahub/datahub-prerequisites
This seems to be a namespace issue. My error occurs while trying to 
helm install
into a namespace
i
Could you share the logs and values.yaml?
g
logs are above, I'm not using a specific values.yaml
i
So the log you shared is for schema registry?
Was Kafka deployed successfully?
g
yes log is for schema registry. Kafka did not deploy at all, causing schema registry to crash with the above. I was deploying all helm charts in a custom namespace, which caused the crash(es). Running the helm installs in the default namespace seems to work fine...everything deploys
👍 1
2 Views
Open in Slack
PreviousNext
Powered by