Hi guys! I'm struggling to use the BentoML monitoring to send logs to Elastic APM using OpenTelemetry (OTLP). I've open an issue. I would appreciate any help! 🤗

There is an internal context that you can use

from bentoml._internal.context import trace_context

...
    @bentoml.api
    def predict(self, inputs: str) -> str:
        print(trace_context.span_id)
        print(trace_context.trace_id)

see if this works for you

Hi guys, I'm still stuck on setting up model monitoring (OTLP Agent to Elastic APM). I've been trying manual OpenTelemetry instrumentation like so:

env_variables = load_service_env_vars()

logging.basicConfig(level=<http://logging.INFO|logging.INFO>)
logger = logging.getLogger(__name__)

otel_exporter_otlp_endpoint = os.environ.get('OTEL_EXPORTER_OTLP_ENDPOINT') + "/v1/traces"
otel_exporter_otlp_headers = os.environ.get('OTEL_EXPORTER_OTLP_HEADERS')

key, value = otel_exporter_otlp_headers.split('=')
otel_headers_dict = {key: value}

exporter = OTLPSpanExporter(endpoint=otel_exporter_otlp_endpoint, headers=otel_headers_dict)

resource_attributes = os.environ.get('OTEL_RESOURCE_ATTRIBUTES')

key_value_pairs = resource_attributes.split(',')
result_dict = {}

for pair in key_value_pairs:
    key, value = pair.split('=')
    result_dict[key] = value

OTEL_SERVICE_NAME = "reconciliation-ranker"
resource_attributes = {
    "service.name": OTEL_SERVICE_NAME,
    "service.version": result_dict['service.version'],
    "deployment.environment": result_dict['deployment.environment'],
}

resource = Resource.create(resource_attributes)

provider = TracerProvider(resource=resource)
processor = BatchSpanProcessor(exporter)
provider.add_span_processor(processor)

# Sets the global default tracer provider
trace.set_tracer_provider(provider)

# Creates a tracer from the global tracer provider
tracer = trace.get_tracer(OTEL_SERVICE_NAME)

Then in my BentoML method:

@bentoml.api(input_spec=InferenceInput)  # pylint: disable=no-member
    def inference(self, **params: Any) -> Dict:
        """
        Predicts the reconciliation between the site and the provider tables
        """
        with tracer.start_as_current_span("reconciliation-ranker-test-span") as span:
            span.set_attribute("custom.attribute", "test-value2")

            site_ids = params['site_ids']
            provider_ids = params['provider_ids']

            site_ids_array = np.array(site_ids)
            provider_ids_array = np.array(provider_ids)

            df_site = self.fetch_online_features(
                site_ids_array, feature_view_id=self.feature_site_id
            )
            df_provider = self.fetch_online_features(
                provider_ids_array, feature_view_id=self.feature_provider_id
            )

            df_site['encoded_date'] = encode_str_date(df_site.date.tolist())
            df_provider['encoded_date'] = encode_str_date(df_provider.date.tolist())

            response_dict = self.model.predict(
                df_site,
                df_provider,
                top_k=params['top_k'],
                remove_matched=params['remove_matched'],
                only_recommend=params['only_recommend'],
            )

        provider.force_flush()
        return response_dict

This works when I run in python (i.e

python service.py

). I'm able to see the logs in Elastic APM as well as my custom span. However, when I try to

bentoml build

and make requests to the Bento, I can see some BigQuery related spans (in Elastic APM) but not my custom span. Any idea why?

The id that is printed from the Context is in base10, where else in the access logs is 0x32 and 0x16 respectively, https://github.com/bentoml/BentoML/blob/main/src/bentoml/_internal/configuration/v2/default_configuration.yaml#L34-L35

Thanks Jian! I managed to make the

span_id

and

trace_id

concord. Do you have any clue or guidance on why my manual instrumentalization works when running the service (

python service.py

) and not when running

bentoml serve

? Can it be because BentoML is running its own tracer and this provoke some sort of conflict?

This may be related to my thread a few days ago. We are having issues with tracing as well https://bentoml.slack.com/archives/CKRANBHPH/p1723847426065379?thread_ts=1723847426.065379&cid=CKRANBHPH

CC: @Frost Ming

Hi @Oren Lederman! Yeah I saw your issue and it seems related. I'm going crazy as when I run

bentoml serve

(or the Docker container), I can see some spans (BigQuery related ones for example) but not my custom spans. I don't really know what to do at this point.

@Axel Mendoza We don’t have a solution yet either 😕