This message was deleted.

Hi @Ashish Singh, what is the goal for writing unit tests against

bentoml.metrics

? Do you want to ensure metrics gets recorded and emitted correctly? In that case, would it be better to write integration test against the

/metrics

endpoint?

I agree, it can fall under integration test, so its a simple unit test around a function that on error inc a counter metric with x, y ,z metric. Unit test previously used looked something like this

def test_chi_squared_gauge_metric():
    label = {}
    function_that_will_inc_metric()
    )
    after = get_metric_value("metric", label)
    assert after == 1.0

get_metric_value

looks like this

def get_metric_value(
    prom_client: PrometheusClient,
    event: str,
    result_label: Dict[str, Any],
) -> int | float:
    return REGISTRY.get_sample_value(event, result_label) or 0

and change I have done because

REGISTRY

is not available directly that works locally, is add a fixture

@pytest.fixture()
def fixture_metrics_client() -> PrometheusClient:
    """This fixtures return a PrometheusClient instance that can be used for testing."""
    return BentoMLContainer.metrics_client.get()

But after this change all tests work locally, but all metric test failed on circleci, log points to this code in prom client

@property
    def prometheus_client(self):
        if self.multiproc and not self._imported:
            # step 1: check environment
            assert (
                "prometheus_client" not in sys.modules
            ), "prometheus_client is already imported, multiprocessing will not work properly"
    
            assert (
                self.multiproc_dir
            ), f"Invalid prometheus multiproc directory: {self.multiproc_dir}"
>           assert os.path.isdir(self.multiproc_dir)
E           AssertionError

Not able to assert

multiproc_dir

Thanks for sharing the findings!