Hi there, very new to BentoML. I have a question regarding configuring the BentoML dockerfile to Instrument my application with Datadog. We have our BentoML projects running on Google Cloud Run. I am following the below two docs for Datadog & Configuring Dockerfile for BentoML • https://docs.datadoghq.com/serverless/google_cloud_run#how-serverless-init-works • https://docs.bentoml.com/en/v1.1.11/guides/containerization.html#entrypoint I have updated the Dockerfile.template to below

{% extends bento_base_template %}
{% block SETUP_BENTO_COMPONENTS %}
{{ super() }}

COPY --from=datadog/serverless-init:1 /datadog-init /app/datadog-init

RUN pip install --target /dd_tracer/python/ ddtrace

{% endblock %}
{% block SETUP_BENTO_ENTRYPOINT %}
{{ super() }}

ENTRYPOINT ["/app/datadog-init"]

CMD [ "{{ bento__entrypoint }}", "bentoml", "serve", "{{ bento__path }}"]

{% endblock %}

I am able to run the app after containerizing, but whenever I go to localhost:3000, the app breaks on rendering the swagger UI. Additionally, when I deploy the container to GCR. I see the error log

ERROR 2024-05-21T18:57:45.879894719Z terminated: Application failed to start: failed to load /app/datadog-init: exec format error
  {
    "textPayload": "terminated: Application failed to start: failed to load /app/datadog-init: exec format error",
    "insertId": "664ceea9000d6d1bd6bd334b",
    "resource": {
      ...
    },
    "timestamp": "2024-05-21T18:57:45.879894719Z",
    "severity": "ERROR",
    "logName": "...",
    "receiveTimestamp": "2024-05-21T18:57:45.882099443Z"
  }

I think the issue could be that

/app/datadog-init

not being able to wrap the Bento app as a subprocess. > The

serverless-init

application wraps your process and executes it as a subprocess. It starts a DogStatsD listener for metrics and a Trace Agent listener for traces. It collects logs by wrapping the stdout/stderr streams of your application. After bootstrapping, serverless-init then launches your command as a subprocess. Has anyone been able to successfully integrate Datadog to BentoML? Thank you!

I don’t think DataDog can automatically configure itself for ASGI apps. I don’t really understand the considerations, but following Chaoyu’s suggestion here, I was able to get it working if you add the

TraceMiddleware

to the service in your model service code.

Thank you so much @Adam Liter! In my service.py, I am running below to add TraceMiddleware

svc.add_asgi_middleware(TraceMiddleware)

And running ddtrace-run when running bentoml serve

CMD [ddtrace-run, "bentoml", "serve", "{{ bento__path }}"]

Does this align with what you had set up? Additionally, a bit of basic question, where did you configure the Tracer with the DD_API_KEY, DD_SERVICE, DD_ENV, DD_VERSION?

We deploy to K8s, and our DevOps team has all K8s deployments configured to use the

ddtrace

K8s injection library as an init container. So we don’t include

ddtrace-run

in the entrypoint. Moreover, all of those things are set as part of the environment variables in the templated Helm/K8s deployment. Not sure if that helps. The answers are unfortunately pretty specific to the idiosyncrasies of the workflows developed by our DevOps team for our organization.