https://datahubproject.io logo
#all-things-deployment
Title
# all-things-deployment
t

thousands-solstice-2498

09/26/2022, 9:12 AM
Hi team, Please advise here. URI [/datahubpolicyindex_v2/_search?typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 404 Not Found] {"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [datahubpolicyindex_v2]","resource.type":"index_or_alias","resource.id":"datahubpolicyindex_v2","index_uuid":"_na_","index":"datahubpolicyindex_v2"}],"type":"index_not_found_exception","reason":"no such index [datahubpolicyindex_v2]","resource.type":"index_or_alias","resource.id":"datahubpolicyindex_v2","index_uuid":"_na_","index":"datahubpolicyindex_v2"},"status":404}
b

bulky-electrician-72362

09/26/2022, 7:06 PM
hey @thousands-solstice-2498, how is the service deployed?(quick-start, k8s with helm, etc) where is the elasticsearch instance running? (aws, k8s, local docker..)
t

thousands-solstice-2498

09/27/2022, 3:31 AM
we've deployed in our cloud native infra. s0g09ba@m-c02f5axamd6n sg-rcube-datahub % kubectl get pods -n p1978837828 NAME READY STATUS RESTARTS AGE prerequisites-cp-schema-registry-96495795-t289v 2/2 Running 0 22h sg-rcube-datahub-datahub-upgrade-job-45mkh 0/1 Error 0 12h sg-rcube-datahub-datahub-upgrade-job-fzdtz 0/1 Error 0 12h sg-rcube-datahub-datahub-upgrade-job-ms6rk 0/1 Error 0 12h sg-rcube-datahub-datahub-upgrade-job-rlwwg 0/1 Error 0 12h sg-rcube-datahub-datahub-upgrade-job-tvqf9 0/1 Error 0 12h sg-rcube-datahub-datahub-upgrade-job-tvz85 0/1 Error 0 12h sg-rcube-datahub-datahub-upgrade-job-xm75b 0/1 Error 0 12h sg-rcube-datahub-elasticsearch-setup-job-wfzmz 0/1 Completed 0 12h sg-rcube-datahub-kafka-setup-job-q6l8w 0/1 Completed 0 12h sg-rcube-datahub-postgresql-setup-job-lfvqz 0/1 Completed 0 12h tiller-deploy-66fbc9bdb9-qncnx 1/1 Running 0 23h s0g09ba@m-c02f5axamd6n sg-rcube-datahub %
by using helm we've deployed.
b

bulky-electrician-72362

09/27/2022, 1:52 PM
hey, could you share the helm values file and the version you are using/ we have seen reports from the recent changes in Elastic Setup jobs are broken. could you use an older version.
could you check the logs in this pod?
sg-rcube-datahub-elasticsearch-setup-job-wfzmz   0/1    Completed  0         12
this job supposed to setup the indecies on your elasticsearch, which is missing for you. can you see any indecies on your ES instance?
t

thousands-solstice-2498

09/27/2022, 3:19 PM
s0g09ba@m-c02f5axamd6n prerequisites % kubectl logs sg-rcube-datahub-elasticsearch-setup-job-mf6dh -n p1978837828 2022/09/27 125645 Waiting for: http://elasticsearch.azure-stage.prod.us.walmart.net:9200 2022/09/27 125645 Received 200 from http://elasticsearch.azure-stage.prod.us.walmart.net:9200 datahub_usage_event_policy exists datahub_usage_event_index_template exists 2022/09/27 125645 Command finished successfully. s0g09ba@m-c02f5axamd6n prerequisites %
But upgrade job we could below error. Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://elasticsearch.azure-stage.prod.us.walmart.net:9200], URI [/datahubpolicyindex_v2/_search?typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 400 Bad Request]
f

freezing-oxygen-20989

06/16/2023, 2:23 PM
hi @bulky-electrician-72362, we’re having the same issues.
datahub-system-update-job
fails with this error:
Copy code
Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [<https://vpc-data>-**********.<http://eu-west-2.es.amazonaws.com:443|eu-west-2.es.amazonaws.com:443>], URI [/datahubpolicyindex_v2/_search?typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 404 Not Found]
{"error":{"root_cause":[{"type":"index_not_found_exception"
while
datahub-elasticsearch-setup-job
completed with the following logs:
Copy code
2023/06/16 11:13:09 Waiting for: <https://vpc-data>-*******.<http://eu-west-2.es.amazonaws.com:443|eu-west-2.es.amazonaws.com:443>
2023/06/16 11:13:10 Received 200 from <https://vpc-data-7v2suywptznvqzaiiabiozgiua.eu-west-2.es.amazonaws.com:443>
going to use protocol: https
going to use elastic headers based on username and password
not using any prefix

 datahub_analytics_enabled: true

>>> GET _opendistro/_ism/policies/datahub_usage_event_policy response code is 200
>>> _opendistro/_ism/policies/datahub_usage_event_policy already exists ✓

>>> GET _template/datahub_usage_event_index_template response code is 200
>>> _template/datahub_usage_event_index_template already exists ✓

>>> GET datahub_usage_event-000001 response code is 200
>>> datahub_usage_event-000001 already exists ✓
2023/06/16 11:13:10 Command finished successfully.
do you have any suggestions what to do by any chance?
3 Views