Hi team, I am getting below error in gms logs and also not able to see datasets only platform is added . Logs are here...... ERROR c.l.m.s.e.update.BulkListener:56 - Error feeding bulk request. No retries left. Request: Failed to perform bulk request: index [dat asetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGM%2CPROD%29];Failed to perform bulk request: index [ datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGMGRP%2CPROD%29];Failed to perform bulk request: i ndex [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGMGRP%2CPROD%29];Failed to perform bulk requ est: index [system_metadata_service_v1], optype: [UPDATE], type [_doc], id [MulrbF+m7pOX8FZWKhFvfA==];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], typ e [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTG%2CPROD%29];Failed to perform bulk request: index [system_metadata_service_v1], optype: [UPDATE], type [_doc], id [+dVcgKkZo0b54/VULBLKVA==];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3A dataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGL%2CPROD%29];Failed to perform bulk request: index [graph_service_v1], optype: [UPDATE], type [_doc], id [q8v+RyJ/4GuJiUEeie9BYQ==];Faile d to perform bulk request: index [system_metadata_service_v1], optype: [UPDATE], type [_doc], id [n3GQtcgTWWWXYCsCtIxcIA==];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGL%2CPROD%29];Failed to perform bulk request: index [system_metadat a_service_v1], optype: [UPDATE], type [_doc], id [1Pe69BbgsIXutJ+qQx46Sg==];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Ad ataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGL%2CPROD%29];Failed to perform bulk request: index [system_metadata_service_v1], optype: [UPDATE], type [_doc], id [0EvAuIBMmx2PfhUA010aFQ==];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CC TGS.STNET.DTS2.CTGL%2CPROD%29];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere% 2CCTGS.STNET.DTS2.CTGRP%2CPROD%29];Failed to perform bulk request: index [system_metadata_service_v1], optype: [UPDATE], type [_doc], id [yvOt1wkWazchptZYmJtRzg==];Failed to perfor m bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGL%2CPROD%29];Failed to per form bulk request: index [system_metadata_service_v1], optype: [UPDATE], type [_doc], id [OvHgSEtTPUICetikdlM0+g==];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGL%2CPROD%29];Failed to perform bulk request: index [system_metadata_servic e_v1], optype: [UPDATE], type [_doc], id [7c9EvHBwA7Xss4O7rNsE2g==];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3 A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGL%2CPROD%29];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adatase t%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGRP%2CPROD%29];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Ada taset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTITLE%2CPROD%29];Failed to perform bulk request: index [system_metadata_service_v1], optype: [UPDATE], type [_doc], i d [sQtpzVgs7goDUM/HXNvkGw==];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2C CTGS.STNET.DTS2.CTGL%2CPROD%29];Failed to perform bulk request: index [system_metadata_service_v1], optype: [UPDATE], type [_doc], id [qIM8ghhj7xP2qzARnIDXwQ==];Failed to perform b ulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGM%2CPROD%29];Failed to perfor m bulk request: index [graph_service_v1], optype: [UPDATE], type [_doc], id [q7vdPq8m0+zZpY5saCJV6A==];Failed to perform bulk request: index [system_metadata_service_v1], optype: [ UPDATE], type [_doc], id [6OV/6/4OgKUF/Y5Rp0hbEg==];Failed to perform bulk request: index [datasetindex_v2], optype: [UPDATE], type [_doc], id [urn%3Ali%3Adataset%3A%28urn%3Ali%3Ad ataPlatform%3AAmpere%2CCTGS.STNET.DTS2.CTGM%2CPROD%29] java.io.IOException: Unable to parse response body for Response{requestLine=POST /_bulk?timeout=1m HTTP/1.1, host=http://logging-es-http.monitoring.svc.cluster.local:9200, response =HTTP/1.1 200 OK} at org.elasticsearch.client.RestHighLevelClient$1.onSuccess(RestHighLevelClient.java:1783) at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onSuccess(RestClient.java:636) at org.elasticsearch.client.RestClient$1.completed(RestClient.java:376) at org.elasticsearch.client.RestClient$1.completed(RestClient.java:370) at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:181) at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448) at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338) at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.NullPointerException: null at java.base/java.util.Objects.requireNonNull(Objects.java:221) at org.elasticsearch.action.DocWriteResponse.<init>(DocWriteResponse.java:127) at org.elasticsearch.action.update.UpdateResponse.<init>(UpdateResponse.java:65) at org.elasticsearch.action.update.UpdateResponse$Builder.build(UpdateResponse.java:172) at org.elasticsearch.action.update.UpdateResponse$Builder.build(UpdateResponse.java:160) at org.elasticsearch.action.bulk.BulkItemResponse.fromXContent(BulkItemResponse.java:159) at org.elasticsearch.action.bulk.BulkResponse.fromXContent(BulkResponse.java:188) at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1911) at org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAsyncAndParseEntity$10(RestHighLevelClient.java:1699) at org.elasticsearch.client.RestHighLevelClient$1.onSuccess(RestHighLevelClient.java:1781) ... 18 common frames omitted 2023-09-07 145508,612 [qtp71399214-18] INFO c.l.m.r.platform.PlatformResource:61 - Emitting platform event. name: entityChangeEvent, key: entityChangeEvent-urnlidataset:(urn:l idataPlatformAmpere,CTGS.STNET.DTS2.CTGM,PROD) Any idea on why it is failing while making bulk request to elastic search? Context : using helm deployment of datahub and ingesting a custom data source.

The error indicates it cannot parse the response from Elasticsearch, would you be able to check the Elasticsearch logs? Is there any indication that Elasticsearch is encountering an issue?

Elastic search version is 8.2.3

Please downgrade to 7.x

hi @brainy-tent-14503 I encountered similar issue as mentioned by @orange-gpu-90973. May I know whether the build from this PR is available for use ? I would like to give it a try. the below log in gms pod is recurring when I refresh the frontend landing page.

│ 2023-09-28 04:04:35,395 [I/O dispatcher 1] ERROR c.l.m.s.e.update.BulkListener:56 - Error feeding bulk request. No retries left. Request: Failed to perform bulk request: index [datahub_usage_event], optype: [CREATE], type [_doc], id [Recommendation │ java.io.IOException: Unable to parse response body for Response{requestLine=POST /_bulk?timeout=1m HTTP/1.1, host=https://<aws-opensearch-endpoint>:443, response=HTTP/1.1 200 OK}             │

besides , The already ingested entities, like domain , tags, and even user groups have not been shown up properly as expected. I wonder whether it also has to do with version conflict. info: OpenSearch 2.5 helm chart used : datahub/datahub (0.2.188) thanks

While the PR has been merged to master, we’ve not yet collected and tested all the changes for the next release. Assuming the next release version will likely be v0.11.1 and will include the OS 2.x support in the changelog for that release.