Hi all, thanks for your work on the langkit integration. it is working for me on localhost. however when i deploy to heroku i get an error for slug size too large at 2.4 gb (see attached screenshot). I'm using: langchain==0.0.187 langkit==0.0.1b4 I've uninstalled langkit and it deploys fine. is anyone else using heroku or running into this issue? thanks!

has renamed the channel from "langkit-feedback" to "langkit-support"

set the channel topic: Questions and feedback for LangKit https://github.com/whylabs/langkit/

Hi all; Apologies I keep sending my message on accident! I have a very simple problem that may have an easy solution:

import whylogs as why
from whylogs.api.writer.whylabs import WhyLabsWriter
from langkit.whylogs.rolling_logger import RollingLogger
from langkit import all_metrics


schema = all_metrics.init()
results = why.log({"David": "Hello,", "Emily": "Hey there!"}, schema=all_metrics).profile()

In the previous example, I am doing a simple upload of this text query using the schema from all_metrics to upload to whylab. My question is, how can I get the 'raw' output values from each UDF for every given input string? I know I can get the summary stats using view:

#See some initial metrics tabulated
view = results.view()
view.to_pandas()

But rather than summary statistics, I would like basically tabular data where each row is an input string, and there is a column for every UDF metric I've defined. Reason being I want to calculate the correlation between my metrics across all input samples .Please let me know if I should clarify more!

hello I am having issues with injections detection tutorial. My kernel keeps crashing when trying to run it. Is the injections embedding model large?

what went wrong? https://github.com/whylabs/langkit/blob/main/langkit/examples/Intro_to_Langkit.ipynb

Hi, I am getting error "No module named 'PIL'" while trying to import llm_metrics or themes from langkit. Before I installed the new version of langkit, I was able to import themes. I tried importing pillow, but that doesnt help. Please suggest what can be done to resolve this error.

Trace below:

--------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) File c:\Users\<>\WhylabsDemo\whylabsenv1\Lib\site-packages\langkit\llm_metrics.py:11 10 from langkit import textstat ---> 11 from langkit import themes 12 from langkit import toxicity File c:\Users\<>\WhylabsDemo\whylabsenv1\Lib\site-packages\langkit\themes.py:5 3 from typing import Callable, Optional, Dict, List ----> 5 from sentence_transformers import util 6 from torch import Tensor File c\Users\101861\WhylabsDemo\whylabsenv1\Lib\site packages\sentence transformers\ init .py3 2 __MODEL_HUB_ORGANIZATION__ = 'sentence-transformers' ----> 3 from .datasets import SentencesDataset, ParallelSentencesDataset 4 from .LoggingHandler import LoggingHandler File c:\Users\<>\WhylabsDemo\whylabsenv1\Lib\site-packages\sentence_transformers\datasets\__init__.py:3 2 from .NoDuplicatesDataLoader import NoDuplicatesDataLoader ----> 3 from .ParallelSentencesDataset import ParallelSentencesDataset 4 from .SentencesDataset import SentencesDataset File c:\Users\<>\WhylabsDemo\whylabsenv1\Lib\site-packages\sentence_transformers\datasets\ParallelSentencesDataset.py:4 3 import gzip ----> 4 from .. import SentenceTransformer 5 from ..readers import InputExample File c:\Users\<>\WhylabsDemo\whylabsenv1\Lib\site-packages\sentence_transformers\SentenceTransformer.py:27 26 from .util import import_from_string, batch_to_device, fullname, snapshot_download ---> 27 from .models import Transformer, Pooling, Dense 28 from .model_card_templates import ModelCardTemplate File c:\Users\<>\WhylabsDemo\whylabsenv1\Lib\site-packages\sentence_transformers\models\__init__.py:14 13 from .WordWeights import WordWeights ---> 14 from .CLIPModel import CLIPModel File c:\Users\<>\WhylabsDemo\whylabsenv1\Lib\site-packages\sentence_transformers\models\CLIPModel.py:4 3 import torch ----> 4 from PIL import Image 7 class CLIPModel(nn.Module): ModuleNotFoundError: No module named 'PIL' During handling of the above exception, another exception occurred: ImportError Traceback (most recent call last) Cell In[23], line 1 ----> 1 from langkit import llm_metrics File c:\Users\<>\WhylabsDemo\whylabsenv1\Lib\site-packages\langkit\llm_metrics.py:15 13 from langkit import input_output 14 except ImportError: ---> 15 raise ImportError( 16 "To use

llm_metrics

please install it with

pip install langkit[all]

." 17 ) 20 def init() -> DeclarativeSchema: 21 regexes.init() ImportError: To use

llm_metrics

please install it with

pip install langkit[all]

.

I have a question. Is the non-llm prompt injection detection done locally?

I am trying to push LLM monitoring metrics to Platform. I am facing this error even though langkit has been installed. Could you please help ? No module named : langkit.callback.handler. Lankit version is 0.0.2 and the error seems to be due to the code - whylabs = WhyLabsCallbackHandler.from_params() (edited)

I'm having hanging issues of initializing the injections when running to create an API using fast API.

Hi community, 👋🏽 Is there any resource I could use to learn how to integrate LangKit and OpenLLM directly—or with a few hacks? (I saw a WhyLabs-BentoML integration: whylogs/python/examples/integrations/bentoml at mainline · whylabs/whylogs · GitHub and trying it out.)

HI Team just simple question does langkit use any LLM as a provider to come with all these metrices?

Hi team, I was running hallucination metrics and wanted to extract response.hallucination metrics score by using profile.get_column(‘response.hallucination’).to_summer_dict(), unfortunately I got ‘dataset profile’ object has no attributes ‘get_colum’ error message. Do I made something wrong or there’s other way to do it? Thank you!