https://bentoml.com logo
Join Slack
Powered by
This message was deleted.
# ask-for-help
s
This message was deleted.
j
yes
a
@Jiang Could you kindly point to any resource on the docs which illustrates this? We are trying to use the bentoml service as a fastApi wrapper for now, and we are not using any locally trained model. Have created a service like this without any runners:
Copy code
svc = bentoml.Service("my_service")
We are using poetry for our python dependencies. In 
bentofile.yaml
we have supplied the service field with the corresponding service, provided the exclude list. We thought the issue might be with poetry so we even specified a requirements.txt file like:
Copy code
python:
  requirements_txt: "requirements.txt"
  lock_packages: false
  trusted_host:
    - "<http://pypi.python.org|pypi.python.org>"
While running 
bentoml build --verbose
, it doesn't proceed beyond 
Building BentoML service...
Do let me know if want any additional details/logs.
j
I see. Do you have large files in your project directory?
a
None that we haven't added to 
exclude
j
Ah. Is there any downloading logic in the module level of the 
service.py
?
a
No downloading logic
j
From my pov I thing it may not be caused by lack of runners. Do you use conda? Or would you mind share me the service.py and bentofile.yaml?
a
This is the method defined in the 
service.py
Copy code
fastapi_app = FastAPI()

def get_service() -> bentoml.Service:
    svc = bentoml.Service("my_service")

svc.mount_asgi_app(fastapi_app)

    @svc.api(input=JSON(pydantic_model=ServiceInput), output=JSON(), route="/ml-svcs/v0")
    async def get_data(user_input: ServiceInput) -> Dict[str, Any]:
        response = models[user_input.model].get_user_data(user_input.bio)
        return response

    return svc
Importing this into root level 
main.py
Copy code
app = get_service()
And here is 
bentofile.yaml
Copy code
service: "project.main:app"
labels:
  owner: XYZ
  stage: dev
exclude:
  - "tests/"
python:
  requirements_txt: "requirements.txt"
  lock_packages: false
  trusted_host:
    - "<http://pypi.python.org|pypi.python.org>"
The service runs fine using 
bentoml serve project.main:app --development --reload
The issue is in building the bento. Also we are not using Conda
j
nvm
I guess the indent is
Copy code
fastapi_app = FastAPI()

def get_service() -> bentoml.Service:
    svc = bentoml.Service("my_service")
    svc.mount_asgi_app(fastapi_app)

    @svc.api(input=JSON(pydantic_model=ServiceInput), output=JSON(), route="/ml-svcs/v0")
    async def get_data(user_input: ServiceInput) -> Dict[str, Any]:
        response = models[user_input.model].get_user_data(user_input.bio)
        return response

    return svc
a
Yeah, that is the existing indent. Got messed up on Slack probably.
j
Would you mind sharing the lines about 
models
?
a
Imported models like:
Copy code
from project.model.llm_model import models
The 
llml_model
file has two model classes that are being exported:
Copy code
models = {"chat": ChatModelInference(), "text": LLMInferenceModel("text-davinci-003")}
r
in this classes we are basically calling the open ai api for now to get answers to the prompt ..
j
Sure. I see. So the build step just stucked at the second line 
Building BentoML service "yolo_v5_demo:dnsrvuaxik7fsusu" from build context
like this ?
a
Yes but we are not getting the rest of the logs that we should be getting either as you have in the image. It's just the single line 
Building BentoML service "yolo_v5_demo:dnsrvuaxik7fsusu" from build context
That's why we have no idea what might be causing the behaviour.
j
Which Python version are you using?
a
3.11.2
j
That's why we have no idea what might be causing the behaviour.
Yes, it also caused me trouble in locating. Would you like sharing me the req.txt?
a
Sure, here is the 
pyproject.toml
file for poetry. We generated the 
requirements.txt
using 
poetry export -f requirements.txt --output requirements.txt
Copy code
[tool.poetry]
name = "expertise-generation"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]
readme = "README.md"
packages = [{ include = "expertise_generation" }]

[tool.poetry.dependencies]
python = "3.11.2"
langchain = "0.0.200"
openai = "0.27.6"
fastapi = { extras = ["all"], version = "0.97.0" }
uvicorn = { extras = ["standard"], version = "0.22.0" }
invoke = "2.1.3"
bentoml = "1.0.22"

[tool.poetry.group.dev.dependencies]
ruff = "0.0.270"
black = "23.3.0"
pytest = "7.3.1"
isort = "5.12.0"
pre-commit = "3.3.2"
watchdog = "3.0.0"
streamlit = "1.22.0"

[tool.black]
line-length = 120
target-version = ['py37']
include = '\.pyi?$'
exclude = '''
(
  /(
      \.eggs         # exclude a few common directories in the
    | \.git          # root of the project
    | \.hg
    | \.mypy_cache
    | \.tox
    | \.venv
    | _build
    | buck-out
    | build
    | dist
  )/

)
'''

[tool.ruff]
# Enable pycodestyle (`E`) and Pyflakes (`F`) codes by default.
select = ["E", "F"]
ignore = []

# Allow autofix for all enabled rules (when `--fix`) is provided.
fixable = ["A", "B", "C", "D", "E", "F", "G", "I", "N", "Q", "S", "T", "W", "ANN", "ARG", "BLE", "COM", "DJ", "DTZ", "EM", "ERA", "EXE", "FBT", "ICN", "INP", "ISC", "NPY", "PD", "PGH", "PIE", "PL", "PT", "PTH", "PYI", "RET", "RSE", "RUF", "SIM", "SLF", "TCH", "TID", "TRY", "UP", "YTT"]
unfixable = []

# Exclude a variety of commonly ignored directories.
exclude = [
    ".bzr",
    ".direnv",
    ".eggs",
    ".git",
    ".git-rewrite",
    ".hg",
    ".mypy_cache",
    ".nox",
    ".pants.d",
    ".pytype",
    ".ruff_cache",
    ".svn",
    ".tox",
    ".venv",
    "__pypackages__",
    "_build",
    "buck-out",
    "build",
    "dist",
    "node_modules",
    "venv",
]

# Same as Black.
line-length = 120

# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"

# Assume Python 3.11.
target-version = "py311"

[tool.ruff.mccabe]
# Unlike Flake8, default to a complexity level of 10.
max-complexity = 10

[tool.isort]
profile = "black"
src_paths = ["expertise_generation", "tests"]
skip = [".gitignore"]
line_length = 120


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
👀 1
j
I can not reproduce with the same req.txt, python and bentoml version. I will ask in the team channel for other's help.
gratitude thank you 1
r
Hi Jiang, just realised , 
.venv
folder was missing from excluded list in the 
bentofile.yaml
. After adding 
.venv
in the exclude section, then build is getting created.
thank you for support 😄 and sorry for the confusion 🙏
➕ 1
2 Views
Open in Slack
PreviousNext
Powered by