Hey all, I'm having a failure on Datahub. It's shown like the attached picture in the UI. When I check the logs it says the following:

exception: java.util.concurrent.CompletionException: java.lang.RuntimeException: Failed to retrieve entities of type Dataset

Caused by: java.lang.RuntimeException: Failed to batch load Datasets

Caused by: com.linkedin.r2.RemoteInvocationException: com.linkedin.r2.RemoteInvocationException: Failed to get response from server for URI <https://datahub-gms.net:443/entities>
	at com.linkedin.restli.internal.client.ExceptionUtil.wrapThrowable(ExceptionUtil.java:135)

Caused by: io.netty.handler.codec.TooLongFrameException: Response entity too large: HttpObjectAggregator$AggregatedFullHttpResponse(decodeResult: success, version: HTTP/1.1, content: CompositeByteBuf(ridx: 0, widx: 2096929, cap: 2096929, components=335))

So the entities are too large, causing. the lookup to fail. To give a bit of context: this is only happening with really specific searches, where it has to retrieve multiple datasets which have a huge amount of columns (1000k+). Then it times out. If I search for these individual entities it's fine, the same when I go to their respective pages. Any idea on how to fix this? I'm looking for some netty settings atm like

maxResponseKB

which I could potentially set. It's probably the same error as https://github.com/linkedin/datahub/issues/3106

Thanks for reporting this @calm-sunset-28996: @green-football-43791 might be able to help you with this.

Hey @calm-sunset-28996! Thanks for reporting. I'll look into this and get back to you.

Cool, thanks. Let me know if you need help with something. I can look into it further tomorrow as well.

Hey @calm-sunset-28996 - I have identified a fix for this. We are going to migrate our graphql layer to call the batchload directly rather than thru http. this will solve the maxResponseKB issue.

Sorry I was on holidays so I missed the chat. But super, that sounds excellent! Thank you for fixing it in such a short notice!