modern-belgium-81337
06/09/2022, 11:15 PMtall-fall-45442
06/10/2022, 12:17 AMhelm upgrade --install datahub/datahub --values values.yaml --debug
(where I've placed all the AWS configuration details in the values.yaml file) I get the following output
history.go:56: [debug] getting history for release datahub
upgrade.go:139: [debug] preparing upgrade for datahub
upgrade.go:147: [debug] performing update for datahub
upgrade.go:319: [debug] creating upgraded release for datahub
client.go:299: [debug] Starting delete for "datahub-elasticsearch-setup-job" Job
client.go:128: [debug] creating 1 resource(s)
client.go:528: [debug] Watching for changes to Job datahub-elasticsearch-setup-job with timeout of 5m0s
client.go:556: [debug] Add/Modify event for datahub-elasticsearch-setup-job: ADDED
client.go:595: [debug] datahub-elasticsearch-setup-job: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
client.go:556: [debug] Add/Modify event for datahub-elasticsearch-setup-job: MODIFIED
client.go:299: [debug] Starting delete for "datahub-kafka-setup-job" Job
client.go:128: [debug] creating 1 resource(s)
client.go:528: [debug] Watching for changes to Job datahub-kafka-setup-job with timeout of 5m0s
client.go:556: [debug] Add/Modify event for datahub-kafka-setup-job: ADDED
client.go:595: [debug] datahub-kafka-setup-job: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
upgrade.go:430: [debug] warning: Upgrade "datahub" failed: pre-upgrade hooks failed: timed out waiting for the condition
Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition
helm.go:88: [debug] pre-upgrade hooks failed: timed out waiting for the condition
UPGRADE FAILED
main.newUpgradeCmd.func2
<http://helm.sh/helm/v3/cmd/helm/upgrade.go:202|helm.sh/helm/v3/cmd/helm/upgrade.go:202>
<http://github.com/spf13/cobra.(*Command).execute|github.com/spf13/cobra.(*Command).execute>
<http://github.com/spf13/cobra@v1.2.1/command.go:856|github.com/spf13/cobra@v1.2.1/command.go:856>
<http://github.com/spf13/cobra.(*Command).ExecuteC|github.com/spf13/cobra.(*Command).ExecuteC>
<http://github.com/spf13/cobra@v1.2.1/command.go:974|github.com/spf13/cobra@v1.2.1/command.go:974>
<http://github.com/spf13/cobra.(*Command).Execute|github.com/spf13/cobra.(*Command).Execute>
<http://github.com/spf13/cobra@v1.2.1/command.go:902|github.com/spf13/cobra@v1.2.1/command.go:902>
main.main
<http://helm.sh/helm/v3/cmd/helm/helm.go:87|helm.sh/helm/v3/cmd/helm/helm.go:87>
runtime.main
runtime/proc.go:255
runtime.goexit
runtime/asm_amd64.s:1581
Where can I find information about the pre-upgrade hooks? I don't seem to see it mentioned in the deployment documentation.quick-megabyte-61846
06/10/2022, 7:16 AMtall-fall-45442
06/10/2022, 5:26 PMcreamy-artist-77191
06/10/2022, 9:20 PMtall-fall-45442
06/12/2022, 4:55 AMvalues.yaml
file:
global:
springKafkaConfigurationOverrides:
security.protocol: SASL_SSL
sasl.mechanism: SCRAM-SHA-512
sasl.jaas.config: org.apache.kafka.common.security.scram.ScramLoginModule required username="username" password="password":
However, with only these changes the datahub-kafka-setup-job
fails, and I see the following in the logs.
org.apache.kafka.common.KafkaException: Failed to create new KafkaAdminClient
at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:535)
at org.apache.kafka.clients.admin.Admin.create(Admin.java:75)
at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:49)
at io.confluent.admin.utils.ClusterStatus.isKafkaReady(ClusterStatus.java:138)
at io.confluent.admin.utils.cli.KafkaReadyCommand.main(KafkaReadyCommand.java:150)
Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
at org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:82)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:134)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:73)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105)
at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:508)
... 4 more
Does anyone know if there is something that I am missing or did incorrectly?breezy-controller-54597
06/13/2022, 12:48 AMelasticsearch-setup-job
cannot connect to elasticsearch-master
.
2022/06/13 00:38:01 Waiting for: <http://elasticsearch-master:9200|http://elasticsearch-master:9200>
2022/06/13 00:38:05 Problem with request: Get <http://elasticsearch-master:9200:|http://elasticsearch-master:9200:> dial tcp: lookup elasticsearch-master on 10.43.0.10:53: server misbehaving. sleeping 1s
…
2022/06/13 01:08:48 Timeout after 2m0s waiting on dependencies to become available: [<http://elasticseach-master:9200|http://elasticseach-master:9200>]
mammoth-fountain-32989
06/13/2022, 5:24 AMbreezy-controller-54597
06/13/2022, 5:51 AMcreamy-van-28626
06/13/2022, 7:39 AMnumerous-bird-27004
06/13/2022, 5:22 PMbig-carpet-38439
06/14/2022, 4:37 PMfancy-thailand-73281
06/14/2022, 9:38 PMbetter-orange-49102
06/15/2022, 3:48 AMdatahub delete
command is accessible by the other parties as well. Just wondering how you guys would set out to block or at least logging people who send hard delete commands? Some kind of IP whitelist/blacklist for the Ingress and endpoint combination? Is it possible?fresh-napkin-5247
06/15/2022, 12:53 PMbland-orange-13353
06/15/2022, 5:43 PMmysterious-portugal-30527
06/15/2022, 8:16 PMrapid-book-98432
06/16/2022, 10:27 AMmammoth-fountain-32989
06/16/2022, 10:28 AMcreamy-van-28626
06/16/2022, 2:09 PMfaint-translator-23365
06/16/2022, 8:48 PMbright-cpu-56427
06/17/2022, 5:24 AMgray-architect-29447
06/17/2022, 7:49 AMcreamy-church-10353
06/17/2022, 9:37 AM7.10
, and also using it as a backend for GMS graph service by setting global.graph_service_impl = elasticsearch
in the helm chart.
I'm getting errors while accessing data on Datahub UI, for e.g Analytics
page. I have attached the corresponding errors produced by GMS service. See datahub-gms.log
Any help or hint in this case, please.bitter-lizard-32293
06/17/2022, 2:14 PMhelpful-processor-71693
06/19/2022, 10:17 AMkafka:
bootstrap:
server: "bootstrap1:9096,bootstrap2:9096,bootstrap3:9096"
zookeeper:
server: "zk1:2181,zk2:2181,zk3:2181"
global:
credentialsAndCertsSecrets:
name: sasl-jass-config
secureEnv:
sasl.jaas.config: sasl_jaas_config
springKafkaConfigurationOverrides:
security.protocol: SASL_SSL
sasl.mechanism: SCRAM-SHA-512
My secrets file is containing the jaas.config file content as follows.
org.apache.kafka.common.security.scram.ScramLoginModule required username="xxxxxxxxxx" password="xxxxxxxxxxxx";
and i verified that my MSK is already configured with SASL/SCRAM but still datahub-kafka-setup-job
is failing with following error:
[main] ERROR io.confluent.admin.utils.cli.KafkaReadyCommand - Error while running kafka-ready.
org.apache.kafka.common.KafkaException: Failed to create new KafkaAdminClient
at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:535)
at org.apache.kafka.clients.admin.Admin.create(Admin.java:75)
at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:49)
at io.confluent.admin.utils.ClusterStatus.isKafkaReady(ClusterStatus.java:138)
at io.confluent.admin.utils.cli.KafkaReadyCommand.main(KafkaReadyCommand.java:150)
Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
at org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:82)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:134)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:73)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105)
at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:508)
Can someone please help if anything specific I am missing in the configuration ?prehistoric-knife-90526
06/19/2022, 7:47 PMbland-easter-53873
06/20/2022, 7:13 AMbland-easter-53873
06/20/2022, 7:49 AMgray-architect-29447
06/20/2022, 9:38 AM