https://bentoml.com logo
Join Slack
Powered by
This message was deleted.
# ask-for-help
s
This message was deleted.
f
I tried the following code, but it doesn't register the nested span:
Copy code
def new_tracing_span(
    name: str, attributes: trace.types.Attributes = None
) -> Iterator[trace.Span]:
    return (
        trace.get_tracer_provider()
        .get_tracer(__name__)
        .start_as_current_span(name, attributes=attributes)
    )


with new_tracing_span("preprocessing_fn"):
    # ...
Maybe I would need the same 
tracer
instance? From reading the code, it doesn't seem like I have access to it.
Not ideal (since it uses internal API from BentoML), but it works:
Copy code
def new_tracing_span(
    name: str, attributes: trace.types.Attributes = None
) -> Iterator[trace.Span]:
    tracer = BentoMLContainer.tracer_provider.get().get_tracer(
        otel_asgi.__name__,
        otel_asgi.__version__,
    )
    return tracer.start_as_current_span(
        name,
        context=context.get_current(),
        kind=trace.SpanKind.INTERNAL,
        attributes=attributes,
    )
j
Glad that you found a solution @Fernando Camargo. Do you mind sharing your use case of using a nested span in your ML workflow?
f
Sure. Basically, when my BentoML service receives a request, it will receive an URL and possibly the HTML of the page. If the HTML is 
None
(the usual case), we fetch the page. We then preprocess this HTML to feed it into our MarkupLM model (which will extract text and XPaths and then tokenize). For last, I invoke my model. So, I want to know how longs it's taking to fetch pages and preprocess them. That's why I'm registering spans for each of them.
a
@Fernando Camargo, Where do you import otel_asgi from? thanks!
f
Copy code
from bentoml._internal.configuration.containers import BentoMLContainer
from opentelemetry import context, trace
from opentelemetry.instrumentation import asgi as otel_asgi

def new_tracing_span(
    name: str, attributes: trace.types.Attributes = None
) -> Iterator[trace.Span]:
    tracer = BentoMLContainer.tracer_provider.get().get_tracer(
        otel_asgi.__name__,
        otel_asgi.__version__,
    )
    return tracer.start_as_current_span(
        name,
        context=context.get_current(),
        kind=trace.SpanKind.INTERNAL,
        attributes=attributes,
    )
Not ideal, as I'm using internal modules from BentoML, but it works, @Amit Gelber
a
@Fernando Camargo, thank you for the quick response! not sure what I'm doing wrong, but it does not seem to work. moreover, if I'm adding these lines, it does not show the ASGI traces I had before
f
I'm using that function as follows:
Copy code
with new_tracing_span("/predict preprocessing"):
    # Code for preprocessing
with new_tracing_span("/predict runner"):
    probas = await runner(batch)
The idea is that I have one parent trace for the 
/predict
and then I want to break down into smaller spans.
I also have these two functions:
Copy code
def register_input(api_input: ApiInput):
    span = trace.get_current_span()

    for i, instance in enumerate(api_input.instances):
        span.set_attribute("input.instances[%d].url" % i, instance.url)
    span.set_attribute("input.parameters.threshold", api_input.parameters.threshold)

def register_output(api_output: ApiOutput):
    span = trace.get_current_span()

    for i, prediction in enumerate(api_output.predictions):
        span.set_attribute("output.predictions[%d].url" % i, prediction.url)
        for j, tag in enumerate(prediction.tags):
            span.set_attribute(
                "output.predictions[%d].tags[%d].label" % (i, j), tag.label
            )
            span.set_attribute(
                "output.predictions[%d].tags[%d].proba" % (i, j), tag.proba
            )
I invoke them in the beginning and the end of my service function.
🙏 1
a
still not sure why it's complicated on BentoML. only thing i needed to add on FastAPI is
Copy code
with tracer.start_as_current_span("Inference") as span:
that's it. I wish if someone from BentoML could clarify this
j
@Jiang could you shed some light here regarding our tracer design!
🙏 1
a
@Jiang?
@Jian Shen Yap is it possible to reach @Jiang on some other channel? Currently I'm stuck. Even "can't be done" answer is something. Thank you
j
@Amit Gelber could you help us to provide a reproducible case for your case? it will help the engineer to quickly understand your case here. meanwhile i'll dive deeper into this
a
seems that my problem was the sample rate 🙂 thanks, everyone! @Jian Shen Yap I still think it should be documented better how to expose custom traces.
j
Glad that you found out what it is. If you would like to share your finding, we'd be happy to have you contributing to the docs or examples since BentoML its an OSS project!
Open in Slack
PreviousNext
Powered by