I am using bentoml 1.0.10 and datadog 1.6.3

Is it using ddtrace library? I.E adding ddtrace-run in the entry point command

I am using

BentoML 1.10.0

.

QQ have you tried ddtrace-run earlier? Or is it directly this approach

This was working for me in the prior versions that is before the major release

@Shihgian Lee attaching the files for dd integration which worked for me with bentoml==1.0.8 bentofile.yaml

service: "serving.py:svc" 

docker:
  env:
    BENTOML_CONFIG: "src/configuration.yaml"
  dockerfile_template: "Dockerfile.template"

Dockerfile.template

{% extends bento_base_template %}
{% block SETUP_BENTO_ENTRYPOINT %}
{{ super() }}
ENTRYPOINT [ "{{ bento__entrypoint }}", "ddtrace-run", "bentoml", "serve", "{{ bento__path }}", "--production" ]
{% endblock %}

hope this helps

@Chaoyu In the containerization documentation, we have the following:

{% extends bento_base_template %}
{% block SETUP_BENTO_ENTRYPOINT %}
{{ super() }}

...
ENTRYPOINT [ "{{ bento__entrypoint }}", "python", "-m", "awslambdaric" ]
{% endblock %}

Can I define the entry point as follows instead?:

{% extends bento_base_template %}
{% block SETUP_BENTO_ENTRYPOINT %}
{{ super() }}

...
ENTRYPOINT [ "{{ bento__entrypoint }}" ]

CMD ["ddtrace-run", "bentoml", "serve", "{{ bento__path }}", "--production" ]
{% endblock %}

Cc @Navya Dalavayi

Hi @Navya Dalavayi Thank you for your instructions! I managed to use

ddtrace-run

with

bentoml

. My

Dockerfile.template

template is a bit different but should not impact the end result :

{% extends bento_base_template %}
{% block SETUP_BENTO_ENTRYPOINT %}
{{ super() }}
CMD ["ddtrace-run", "bentoml", "serve", "{{ bento__path }}", "--production" ]
{% endblock %}

The generated Dockerfile has the following content:

...

USER bentoml

ENTRYPOINT [ "/home/bentoml/bento/env/docker/entrypoint.sh" ]



CMD ["ddtrace-run", "bentoml", "serve", "/home/bentoml/bento", "--production" ]

I prefer the

ddtrace-run

approach over

patch_all()

because I don't have to worry about initialize

patch-all()

as early as possible in the code. However, the

ddtrace-run

didn't solve my problem. In my runner, I use Feast to fetch features from Google Datastore. In

Bentoml 0.13.x

, I could see

grpc

trace. After I upgraded to

Bentoml 1.0.10

, the grpc trace showed up a couple times and then disappeared forever. Datadog has grpc integration. I suspect the data retrieval for Google Datastore maybe too fast for Datadog to capture the trace. I am curious if you access any datastore in your runner? If you do, do you see any traces for your datastore?

It seems not work in your case, can you provide further details, like what you observerd?

OTLP ingestion is not part of our infrastructure. We will have to look into it and discuss it with our DevOps. Thanks for the suggestion! However, this won't be a near term fix for us because there are factors that are not within our team's control.

We moved feast code out of runner and into service api. The Datadog trace shows up for grpc again. Our code is also much simpler. The lesson learned is to make runner dead simple (dumb); only making inferences and nothing else.