https://datahubproject.io logo
Join Slack
Powered by
I know this question regarding use of OpenSearch(1...
# getting-started
i
I know this question regarding use of OpenSearch(1.2) as an alternative to a hosted ElasticSearch have been asked before. But it doesn’t fully work for us -
Copy code
....
elasticsearchSetupJob:
      enabled: true
      image:
        repository: linkedin/datahub-elasticsearch-setup
        tag: "v0.8.40"
      extraEnvs:
        - name: USE_AWS_ELASTICSEARCH
          value: "true"    
....
it triggers a job that finishes successfully The data ingestion works fine updating ES index. But analytics fail with
Copy code
type	:	illegal_argument_exception
reason	:	Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [browserId] in order to load field data by uninverting the inverted index. Note that this can use significant memory.
now I can successfully send POST to -
Copy code
<https://opensearch-domain-endpoint:443/datahub_usage_event/_search?typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true>
Because of above GMS k8s pod is getting evicted periodically. Which affects all datahub setup. Any suggestions?
b
hi roman , you can follow this thread for your query - https://datahubspace.slack.com/archives/C029A3M079U/p1646431113049059
key steps to fix this would be • Delete the datahub_usage_event index from es cluster • Run “curl -XDELETE <<url>>/_opendistro/_ism/policies/datahub_datahub_usage_event_policy” • Run “curl -XDELETE <<url>>/_template/datahub_datahub_usage_event_index_template” • Rerun the elasticsearch-setup-job • Check your setup job logs, if you have no errors, thats it! NOTE: Do not interact with website before all steps are done!! understand that deleting the indices will delete the data from elasticsearch and you follow above steps if you are encountering the issue while setting datahub for the first time
9 Views
Open in Slack
PreviousNext
Powered by