Hi DataHub Team I just encountered a few Snowflake ingestion DataHub #troubleshoot

Hi DataHub-Team, I just encountered a few Snowfla...

rich-pager-68736

02/17/2023, 9:44 AM

Hi DataHub-Team, I just encountered a few Snowflake ingestion errors, namely 413 client errors. Is there any way to tweak this, so bigger requests are allowed?

Copy code

'failures': [{'error': 'Unable to emit metadata to DataHub GMS',
               'info': {'message': '413 Client Error: Request Entity Too Large for url: '
                                   '<https://xxxxxxxxx/api/gms/aspects?action=ingestProposal>',
                        'id': 'urn:li:dataset:(urn:li:dataPlatform:snowflake,YYYYYYYYYYYYYYYY,PROD)'}},
              {'error': 'Unable to emit metadata to DataHub GMS',
               'info': {'message': '413 Client Error: Request Entity Too Large for url: '
                                   '<https://xxxxxxxxx/api/gms/aspects?action=ingestProposal>',
                        'id': 'urn:li:dataset:(urn:li:dataPlatform:snowflake,YYYYYYYYYYYYYYYY,PROD)'}},

✅ 1

rich-pager-68736

02/17/2023, 1:43 PM

Usually, I just run the CLI by port-forwarding our gms port to my local machine. This time I used our external address, so we guess this error happened due to our Ingress not allowing that body size. We set

<http://nginx.ingress.kubernetes.io/proxy-body-size|nginx.ingress.kubernetes.io/proxy-body-size>: 8m

and I hope this resolves it. Will let you know once the CLI job has finished again.

rich-pager-68736

02/17/2023, 4:25 PM

Confirmed. No 413 error msgs anymore after annotating our nginx.

5 Views

Open in Slack

Previous Next