https://bentoml.com logo
Join Slack
Powered by
This message was deleted.
# ask-for-help
s
This message was deleted.
j
Hey @Aytunc Tunay, can you share the full stack trace, and the component and cloud you are using to deploy yatai?
a
Copy code
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1076, in _send_output
    self.send(chunk)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 998, in send
    self.sock.sendall(data)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1237, in sendall
    v = self.send(byte_view[count:])
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1206, in send
    return self._sslobj.write(data)
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2396)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='<http://yatai-dev.dev-solity.net|yatai-dev.dev-solity.net>', port=443): Max retries exceeded with url: /api/v1/model_repositories/summarization/models/pdgwjysl6celxvxi/upload (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2396)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/bin/bentoml", line 8, in <module>
    sys.exit(cli())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml_cli/utils.py", line 362, in wrapper
    return func(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml_cli/utils.py", line 333, in wrapper
    return_value = func(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml_cli/utils.py", line 290, in wrapper
    return func(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml_cli/bentos.py", line 287, in push
    cloud_client.push_bento(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/bentocloud.py", line 78, in push_bento
    self._do_push_bento(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/bentocloud.py", line 119, in _do_push_bento
    list(futures)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/bentocloud.py", line 110, in push_model
    self._do_push_model(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/bentocloud.py", line 699, in _do_push_model
    raise e
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/bentocloud.py", line 690, in _do_push_model
    yatai_rest_client.upload_model(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/client.py", line 377, in upload_model
    resp = self.session.put(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 647, in put
    return self.request("PUT", url, data=data, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/adapters.py", line 563, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='<http://yatai-dev.dev-solity.net|yatai-dev.dev-solity.net>', port=443): Max retries exceeded with url: /api/v1/model_repositories/summarization/models/pdgwjysl6celxvxi/upload (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2396)')))
I deployed Yatai in kubernetes cluster that hosted in aws. I’m also using vpn to access it by the way. Thank you in advance.
@Jian Shen Yap
j
@Xipeng Guan do you have any insights on this?
x
This SSL certificate issue is unrelated to the Yatai code. Here is a debugging approach: 1. Get the current Yatai endpoint:
Copy code
echo $(bentoml yatai current-context | jq -r '.endpoint')/api/v1/info
2. Manually request the Yatai endpoint:
Copy code
curl $(bentoml yatai current-context | jq -r '.endpoint')/api/v1/info
If the second step request fails, it means there is an issue with the SSL certificate configuration.
a
Copy code
➜  bentoml bentoml yatai login --api-token afagagagaasdsa --endpoint <https://yatai-dev.dev-solity.net>
Overriding existing cloud context config: default
Successfully logged in to Cloud for dev-admin in default
➜  bentoml echo $(bentoml yatai current-context | jq -r '.endpoint')/api/v1/info
<https://yatai-dev.dev-solity.net/api/v1/info>
➜  bentoml curl $(bentoml yatai current-context | jq -r '.endpoint')/api/v1/info
{"is_saas":false,"saas_domain_suffix":""}%                                        
➜  bentoml
This is the result @Xipeng Guan
x
It looks like there is no problem with the SSL certificate. You can try executing "bentoml push" again.
a
But unfortunately I’m getting the same error
Copy code
Uploading model "summarization:pdgwjysl6celxvxi" ━━━━ 3.0% • 0.0/… • 1.1  • 0:18:…
                                                             GB      MB/s         
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1076, in _send_output
    self.send(chunk)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 998, in send
    self.sock.sendall(data)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1237, in sendall
    v = self.send(byte_view[count:])
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1206, in send
    return self._sslobj.write(data)
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2396)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='<http://yatai-dev.dev-solity.net|yatai-dev.dev-solity.net>', port=443): Max retries exceeded with url: /api/v1/model_repositories/summarization/models/pdgwjysl6celxvxi/upload (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2396)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/bin/bentoml", line 8, in <module>
    sys.exit(cli())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml_cli/utils.py", line 362, in wrapper
    return func(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml_cli/utils.py", line 333, in wrapper
    return_value = func(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml_cli/utils.py", line 290, in wrapper
    return func(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/click/decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml_cli/bentos.py", line 287, in push
    cloud_client.push_bento(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/bentocloud.py", line 78, in push_bento
    self._do_push_bento(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/bentocloud.py", line 119, in _do_push_bento
    list(futures)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/bentocloud.py", line 110, in push_model
    self._do_push_model(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/bentocloud.py", line 699, in _do_push_model
    raise e
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/bentocloud.py", line 690, in _do_push_model
    yatai_rest_client.upload_model(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/bentoml/_internal/cloud/client.py", line 377, in upload_model
    resp = self.session.put(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 647, in put
    return self.request("PUT", url, data=data, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/adapters.py", line 563, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='<http://yatai-dev.dev-solity.net|yatai-dev.dev-solity.net>', port=443): Max retries exceeded with url: /api/v1/model_repositories/summarization/models/pdgwjysl6celxvxi/upload (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2396)')))
x
Is the command executed on the same machine (virtual machine)? If so, please try executing this command: 
curl -I <https://yatai-dev.dev-solity.net/api/v1/model_repositories/summarization/models/pdgwjysl6celxvxi/upload>
a
Yes, all commands executed on the same machine and here is the response of this command
Copy code
➜  bentoml curl -I <https://yatai-dev.dev-solity.net/api/v1/model_repositories/summarization/models/pdgwjysl6celxvxi/upload>
HTTP/2 404 
date: Mon, 11 Sep 2023 10:15:38 GMT
content-type: application/json; charset=utf-8
content-length: 52
strict-transport-security: max-age=15724800; includeSubDomains
@Xipeng Guan
x
What do you get when you add -X POST to the curl command?
a
Copy code
bentoml curl -X POST  <https://yatai-dev.dev-solity.net/api/v1/model_repositories/summarization/models/pdgwjysl6celxvxi/upload>
{"message":"not found this router with method POST"}%
x
It looks like Python handles SSL certificates differently than curl, so I'll look into it.
a
Okay, thank you. I’m looking forward to hear back from you to solve the problem that I experience.
Is there any update on this ? @Xipeng Guan
x
‌‌‌This is a problem with self-signed certificates, you can try this solution. https://stackoverflow.com/a/43935319
a
But I’m not using self signed certificate. I’m using Let’s Encrypt as CA.
x
‌‌‌‌Will executing the command "bentoml yatai login --api-token yyyy --endpoint xxxxx" result in an error?
a
x
‌‌‌‌‌‌‌Can you provide a detailed introduction to your certificate configuration? Did you add an nginx in front of the yatai http server for SSL verification?
a
Yes of course, • Dns created at route53 in aws which is connected to nginx in kubernetes cluster.
x
‌Will executing this command result in an error? The previous similar command's HTTP method was mistaken, it should be PUT.
Copy code
curl -X PUT <https://yatai-dev.dev-solity.net/api/v1/model_repositories/summarization/models/pdgwjysl6celxvxi/upload>
a
Yes of course, • Dns created at route53 in aws which is connected to nginx in kubernetes cluster.
x
‌‌Is nginx an ingress controller or a self-maintained k8s service?
a
This is the result of the command.
Copy code
bentoml curl -X PUT <https://yatai-dev.dev-solity.net/api/v1/model_repositories/summarization/models/pdgwjysl6celxvxi/upload>
{"message":"username in cookie is empty"}%
x
‌‌‌You can try adding the API token in the header.
Copy code
curl -H 'X-YATAI-API-TOKEN: xxxx' -X PUT <https://yatai-dev.dev-solity.net/api/v1/model_repositories/summarization/models/pdgwjysl6celxvxi/upload>
a
Detail configuration about connection and certification as follows: Dns record created at route53 at aws which is connected to the ingress nginx instance in kubernetes cluster. Ingress resource uses the certificate that is created by let’s encrypt for this record in the cluster.
I run the command with api token, it didnt return anything
x
‌‌‌‌You can try uploading a real file.
Copy code
curl -H 'X-YATAI-API-TOKEN: xxxx' -H 'Content-Type: application/octet-stream' --data-binary @model.tar -f -X PUT <https://yatai-dev.dev-solity.net/api/v1/model_repositories/summarization/models/pdgwjysl6celxvxi/upload>
a
I tried with this by using docker save i created .tar version of the image. But couldnt upload because curl give out of memory error so I changed --data-binary to -T, but this type i got http 413 which also indicates payload is too much
x
a
Ok, I can try that, but do you have any idea why I can’t do that with bentoml push ?
x
‌‌‌‌‌‌Sorry, I cannot reproduce this issue on my end, so I asked you to help execute some commands to remotely locate the problem.
a
Understood, thank you
It’s strange but now I tried again with bentoml push and i didnt get any error 🙂
x
‌‌‌‌‌‌‌Haha, could it be caused by the 413 status code?
a
I’m not really sure, i didnt change anything i just wrote bentoml push image again and it worked!
x
‌🤔 How amazing
Open in Slack
PreviousNext
Powered by